From serwis at poliman.pl Wed Feb 1 06:18:35 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 07:18:35 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> Message-ID: This is debug log files in syslog: Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5ldD4= Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: CONT Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( do_not_reply at example.com,12.173.211.32): query: SELECT email as user, password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, '/', IF(maildir_format='maildir','Maildir',maildir_format)) as userdb_mail, uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, but we have only CRYPT Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: FAIL#0112#011user=do_not_reply at example.com Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 authentication failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5ldD4= Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: AUTH#0113#011CRAM-MD5#011service=smtp#011nologin#011lip=173.72.31.7#011rip=12.173.211.32#011secured Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoLm5ldD4= Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: CONT Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( do_not_reply at example.com,12.173.211.32): query: SELECT email as user, password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, '/', IF(maildir_format='maildir','Maildir',maildir_format)) as userdb_mail, uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, but we have only CRYPT Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: FAIL#0113#011user=do_not_reply at example.com ##################### I added in dovecot.conf lines in passdb block: driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd and commented out default lines #args = /etc/dovecot/dovecot-sql.conf #driver = sql When I try set again default lines I got above error 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > > > On 31.01.2017 09:06, Poliman - Serwis wrote: > > I set up cram-md5 using this tutorial > > https://wiki2.dovecot.org/HowTo/CRAM-MD5 in /etc/dovecot/dovecot.conf in > > passdb code block: > > listen = *,[::] > > protocols = imap pop3 > > #auth_mechanisms = plain login cram-md5 > > auth_mechanisms = cram-md5 plain login > > #dodana nizej linia > > ssl = required > > disable_plaintext_auth = yes > > log_timestamp = "%Y-%m-%d %H:%M:%S " > > mail_privileged_group = vmail > > postmaster_address = postmaster at vps342401.ovh.net > > ssl_cert = > ssl_key = > ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > > ssl_cipher_list = > > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: > > :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > > ssl_prefer_server_ciphers = yes > > ssl_dh_parameters_length = 2048 > > > > > > mail_max_userip_connections = 100 > > passdb { > > # args = /etc/dovecot/dovecot-sql.conf > > # driver = sql > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > } > > userdb { > > driver = prefetch > > } > > userdb { > > args = /etc/dovecot/dovecot-sql.conf > > driver = sql > > } > > Of course I created cram-md5.pwd file. All mails go out and come nicely. > > But after I want to do default settings by commented out these two lines: > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > and uncomment > > # args = /etc/dovecot/dovecot-sql.conf > > # driver = sql > > I can't send emails - I use Thunderbird - get error "logging on server > > mail.example.com not work out". Error in logs: > > dovecot: auth-worker(22698): Error: Auth worker sees different > > passdbs/userdbs than auth server. > > dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > > > > Is it possible that hashed password from cram-md5.pwd file was written to > > database (if yes then where - I have ISPconfig)? I wasn't change any > userdb > > {} block and this second userdb block has this same lines like default > > settings in passdb block. > > > Try > > auth_debug=yes > auth_verbose=yes > > and see if it gives any more reasonable messages. > > Aki > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From aki.tuomi at dovecot.fi Wed Feb 1 07:27:51 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 09:27:51 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> Message-ID: On 01.02.2017 08:18, Poliman - Serwis wrote: > This is debug log files in syslog: > Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: > CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5ldD4= > Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: CONT > Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( > do_not_reply at example.com,12.173.211.32): query: SELECT email as user, > password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, > '/', IF(maildir_format='maildir','Maildir',maildir_format)) as userdb_mail, > uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS > userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' > Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, but we > have only CRYPT > Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: > FAIL#0112#011user=do_not_reply at example.com > Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 authentication > failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5ldD4= > Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > AUTH#0113#011CRAM-MD5#011service=smtp#011nologin#011lip=173.72.31.7#011rip=12.173.211.32#011secured > Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: > CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoLm5ldD4= > Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: CONT > Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( > do_not_reply at example.com,12.173.211.32): query: SELECT email as user, > password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, > '/', IF(maildir_format='maildir','Maildir',maildir_format)) as userdb_mail, > uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS > userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' > Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, but we > have only CRYPT > Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: > FAIL#0113#011user=do_not_reply at example.com > > > > ##################### > I added in dovecot.conf lines in passdb block: > driver = passwd-file > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > and commented out default lines > #args = /etc/dovecot/dovecot-sql.conf > #driver = sql > When I try set again default lines I got above error Can you run doveconf -n with the configuration that causes the above error? Also it clearly does SQL lookup, so that error is happening with SQL passdb. You need to remember to restart dovecot between configuration changes. Aki > > 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >> >> On 31.01.2017 09:06, Poliman - Serwis wrote: >>> I set up cram-md5 using this tutorial >>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in /etc/dovecot/dovecot.conf in >>> passdb code block: >>> listen = *,[::] >>> protocols = imap pop3 >>> #auth_mechanisms = plain login cram-md5 >>> auth_mechanisms = cram-md5 plain login >>> #dodana nizej linia >>> ssl = required >>> disable_plaintext_auth = yes >>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> mail_privileged_group = vmail >>> postmaster_address = postmaster at vps342401.ovh.net >>> ssl_cert = >> ssl_key = >> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>> ssl_cipher_list = >>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: >>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>> ssl_prefer_server_ciphers = yes >>> ssl_dh_parameters_length = 2048 >>> >>> >>> mail_max_userip_connections = 100 >>> passdb { >>> # args = /etc/dovecot/dovecot-sql.conf >>> # driver = sql >>> driver = passwd-file >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> } >>> userdb { >>> driver = prefetch >>> } >>> userdb { >>> args = /etc/dovecot/dovecot-sql.conf >>> driver = sql >>> } >>> Of course I created cram-md5.pwd file. All mails go out and come nicely. >>> But after I want to do default settings by commented out these two lines: >>> driver = passwd-file >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> and uncomment >>> # args = /etc/dovecot/dovecot-sql.conf >>> # driver = sql >>> I can't send emails - I use Thunderbird - get error "logging on server >>> mail.example.com not work out". Error in logs: >>> dovecot: auth-worker(22698): Error: Auth worker sees different >>> passdbs/userdbs than auth server. >>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>> >>> Is it possible that hashed password from cram-md5.pwd file was written to >>> database (if yes then where - I have ISPconfig)? I wasn't change any >> userdb >>> {} block and this second userdb block has this same lines like default >>> settings in passdb block. >>> >> Try >> >> auth_debug=yes >> auth_verbose=yes >> >> and see if it gives any more reasonable messages. >> >> Aki >> > > From serwis at poliman.pl Wed Feb 1 07:33:07 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 08:33:07 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> Message-ID: I always restart dovecot after change config. ;) Sure, I commented out added two lines by me, restarted dovecot and here it is: # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS auth_mechanisms = plain login cram-md5 listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster at example.com protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = : > > > On 01.02.2017 08:18, Poliman - Serwis wrote: > > This is debug log files in syslog: > > Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: > > CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > m5ldD4= > > Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: CONT > > Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( > > do_not_reply at example.com,12.173.211.32): query: SELECT email as user, > > password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, > > '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > userdb_mail, > > uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS > > userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > > mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > > do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' > > Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > > do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, > but we > > have only CRYPT > > Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: > > FAIL#0112#011user=do_not_reply at example.com > > Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > > host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 authentication > > failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5ldD4= > > Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > > (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo > > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > > Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > > (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > > Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > > AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > 011lip=173.72.31.7#011rip=12.173.211.32#011secured > > Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: > > CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > m5ldD4= > > Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: CONT > > Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( > > do_not_reply at example.com,12.173.211.32): query: SELECT email as user, > > password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, > > '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > userdb_mail, > > uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS > > userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > > mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > > do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' > > Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > > do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, but > we > > have only CRYPT > > Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: > > FAIL#0113#011user=do_not_reply at example.com > > > > > > > > ##################### > > I added in dovecot.conf lines in passdb block: > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > and commented out default lines > > #args = /etc/dovecot/dovecot-sql.conf > > #driver = sql > > When I try set again default lines I got above error > > Can you run doveconf -n with the configuration that causes the above > error? Also it clearly does SQL lookup, so that error is happening with > SQL passdb. You need to remember to restart dovecot between > configuration changes. > > Aki > > > > > 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > > > >> > >> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>> I set up cram-md5 using this tutorial > >>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in /etc/dovecot/dovecot.conf > in > >>> passdb code block: > >>> listen = *,[::] > >>> protocols = imap pop3 > >>> #auth_mechanisms = plain login cram-md5 > >>> auth_mechanisms = cram-md5 plain login > >>> #dodana nizej linia > >>> ssl = required > >>> disable_plaintext_auth = yes > >>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>> mail_privileged_group = vmail > >>> postmaster_address = postmaster at vps342401.ovh.net > >>> ssl_cert = >>> ssl_key = >>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>> ssl_cipher_list = > >>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: > >>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>> ssl_prefer_server_ciphers = yes > >>> ssl_dh_parameters_length = 2048 > >>> > >>> > >>> mail_max_userip_connections = 100 > >>> passdb { > >>> # args = /etc/dovecot/dovecot-sql.conf > >>> # driver = sql > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> } > >>> userdb { > >>> driver = prefetch > >>> } > >>> userdb { > >>> args = /etc/dovecot/dovecot-sql.conf > >>> driver = sql > >>> } > >>> Of course I created cram-md5.pwd file. All mails go out and come > nicely. > >>> But after I want to do default settings by commented out these two > lines: > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> and uncomment > >>> # args = /etc/dovecot/dovecot-sql.conf > >>> # driver = sql > >>> I can't send emails - I use Thunderbird - get error "logging on server > >>> mail.example.com not work out". Error in logs: > >>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>> passdbs/userdbs than auth server. > >>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>> > >>> Is it possible that hashed password from cram-md5.pwd file was written > to > >>> database (if yes then where - I have ISPconfig)? I wasn't change any > >> userdb > >>> {} block and this second userdb block has this same lines like default > >>> settings in passdb block. > >>> > >> Try > >> > >> auth_debug=yes > >> auth_verbose=yes > >> > >> and see if it gives any more reasonable messages. > >> > >> Aki > >> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From tlx at leuxner.net Wed Feb 1 07:34:18 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 1 Feb 2017 08:34:18 +0100 Subject: Sieve removeflag Action In-Reply-To: References: <20170113081514.GA60507@nihlus.leuxner.net> <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> Message-ID: <20170201073417.GA55296@nihlus.leuxner.net> * Stephan Bosch 2017.01.31 21:31: > This slipped my attention for the moment. Will look at this soon... Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From aki.tuomi at dovecot.fi Wed Feb 1 07:34:59 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 09:34:59 +0200 Subject: quota-status returns quota_status_success when email would put user over quota In-Reply-To: <26E5C771-BFF5-4982-BB79-69998F19FC80@valo.at> References: <58870F64.2030906@asom-net.dk> <5890AF03.30308@asom-net.dk> <26E5C771-BFF5-4982-BB79-69998F19FC80@valo.at> Message-ID: Steps to setup quota with status: mail_plugins = $mail_plugins quota protocol imap { mail_plugins = $mail_plugins imap_quota } service quota-status { executable = quota-status -p postfix inet_listener { port = 12340 # You can choose any port you want } client_limit = 1 } plugin { quota = count:User quota # or some other backend quota_rule = *:storage=10M # or from userdb quota_grace = 10%% # 10% is the default quota_status_success = DUNNO quota_status_nouser = DUNNO quota_status_overquota = "552 5.2.2 Mailbox is full" quota_vsizes = yes } On 01.02.2017 00:20, Christian Kivalo wrote: > > Am 31. J?nner 2017 16:36:35 MEZ schrieb Kristian Pedersen : >> Hi list, >> >> We still did not manage to get quota-status working. >> We're hoping someone can provide some feedback/ideas on how we may >> investigate this issue further? >> Is it likely to be a bug fixed in a newer version? >> >> Regards, >> >> Kristian >> >> > [...] > >>> Quota-status will return unknown user if that is the case: >>> root at mail:~# printf >>> "recipient=kptest2 at asom-net.dk\nsize=1000000000\n\n" | nc -q1 >>> localhost 12340 >>> action=551 5.5.1 User not found >>> > Tried this and works here. Doveconf -n output with regards to quota settings is very similar, i use a quota dict, not maildir, spotted one difference i commented in your doveconf -n and i'm using version 2.2.27 from source > ... >>> dovecot -n: >>> # 2.2.13: /etc/dovecot/dovecot.conf >>> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 ext4 >>> auth_default_realm = vejen-net.dk >>> auth_mechanisms = plain login >>> auth_verbose = yes >>> disable_plaintext_auth = no >>> first_valid_uid = 110 >>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> mail_debug = yes >>> mail_location = maildir:/data/vmail/%d/%n/ > The one line i'm missing here from your doveconf -n output is mail_plugins = " quota" set in conf.d/10-mail.conf > > Have you added quota to the global mail plugins setting? http://wiki2.dovecot.org/Quota > >>> mail_privileged_group = mail >>> namespace inbox { >>> inbox = yes >>> location = >>> mailbox Drafts { >>> special_use = \Drafts >>> } >>> mailbox Junk { >>> special_use = \Junk >>> } >>> mailbox Sent { >>> special_use = \Sent >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } >>> mailbox Trash { >>> special_use = \Trash >>> } >>> prefix = >>> } >>> passdb { >>> args = /etc/dovecot/local-sql.conf >>> driver = sql >>> } >>> plugin { >>> quota = maildir:User quota >>> quota_rule = *:storage=200M >>> quota_status_nouser = 551 5.5.1 User not found >>> quota_status_overquota = 552 5.2.2 Mailbox is full >>> quota_status_success = DUNNO >>> } >>> protocols = imap pop3 >>> service auth { >>> unix_listener /var/spool/postfix/private/auth { >>> group = postfix >>> mode = 0660 >>> user = postfix >>> } >>> unix_listener auth-master { >>> mode = 0600 >>> user = vmail >>> } >>> user = root >>> } >>> service imap-login { >>> client_limit = 1024 >>> process_limit = 256 >>> process_min_avail = 8 >>> service_count = 0 >>> vsz_limit = 512 M >>> } >>> service imap { >>> process_limit = 10240 >>> } >>> service pop3-login { >>> client_limit = 512 >>> process_limit = 256 >>> process_min_avail = 8 >>> service_count = 0 >>> vsz_limit = 512 M >>> } >>> service pop3 { >>> process_limit = 10240 >>> } >>> service quota-status { >>> client_limit = 1 >>> executable = /usr/lib/dovecot/quota-status -p postfix >>> inet_listener { >>> port = 12340 >>> } >>> } >>> ssl_cert = >> ssl_key = >> ssl_prefer_server_ciphers = yes >>> userdb { >>> args = /etc/dovecot/local-sql.conf >>> driver = sql >>> } >>> protocol lda { >>> auth_socket_path = /var/run/dovecot/auth-master >>> mail_plugins = quota >>> postmaster_address = postmaster at asom-net.dk >>> } >>> protocol imap { >>> mail_plugins = quota imap_quota >>> } >>> protocol pop3 { >>> mail_plugins = quota >>> pop3_uidl_format = %08Xu%08Xv >>> } >>> >>> >>> local-sql.conf: >>> driver = mysql >>> connect = host=xyz dbname=xyz user=xyz password=xyz >>> default_pass_scheme = CRYPT >>> password_query = SELECT email as user, password FROM virtual_users >>> WHERE email='%u'; >>> user_query = SELECT >>> >> CONCAT('/data/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) >> >>> AS home, 110 AS uid, 110 AS gid, CONCAT('*:storage=',mailquota,'M') >> AS >>> quota_rule FROM virtual_users WHERE email='%u'; >>> >>> If I do a strace on the quota-status PID, it seems to do a stat on >> the >>> directory and then give up? Not sure Im interpreting it correct (only >>> including the last few lines): >>> ... >>> lseek(14, 833, SEEK_SET) = 833 >>> munmap(0x7f165d32a000, 833) = 0 >>> close(14) = 0 >>> geteuid() = 0 >>> getegid() = 110 >>> getgid() = 110 >>> getegid() = 110 >>> setgroups(1, [110]) = 0 >>> setresuid(-1, 110, -1) = 0 >>> prctl(PR_SET_DUMPABLE, 1) = 0 >>> stat("/data/vmail/asom-net.dk/kptest", {st_mode=S_IFDIR|0700, >>> st_size=4096, ...}) = 0 >>> prctl(PR_SET_DUMPABLE, 1) = 0 >>> setsockopt(12, SOL_TCP, TCP_CORK, [1], 4) = 0 >>> write(12, "action=DUNNO\n\n", 14) = 14 >>> setsockopt(12, SOL_TCP, TCP_CORK, [0], 4) = 0 >>> epoll_wait(11, {{EPOLLIN, {u32=1593554016, u64=139734059562080}}}, 5, >>> 59999) = 1 >>> read(12, "", 8146) = 0 >>> epoll_ctl(11, EPOLL_CTL_DEL, 12, 7fff0be817a0) = 0 >>> close(12) = 0 >>> epoll_wait(11, {}, 5, 1000) = 0 >>> write(5, "\35q\1\0007\10\0\0\1\0\0\0", 12) = 12 >>> epoll_wait(11, >>> ... >>> >>> It seems like sort of permission issue? >>> root at mail:~# su - vmail >>> No directory, logging in with HOME=/ >>> $ id -a >>> uid=110(vmail) gid=110(vmail) groups=110(vmail) >>> $ cat /data/vmail/asom-net.dk/kptest/maildirsize >>> 524288000S >>> 685 1 >>> 690 1 >>> >>> /data/vmail/asom-net.dk is actually a symlink, maybe that could be of >>> importance?: >>> root at mail:~# ls -ld /data/vmail/asom-net.dk >>> lrwxrwxrwx 1 root root 19 Jan 9 11:18 /data/vmail/asom-net.dk -> >>> ../mnt1/asom-net.dk >>> >>> root at mail:~# ls -ld /data/mnt1/asom-net.dk/ >>> drwxrwx--- 45 vmail vmail 4096 Dec 15 10:54 /data/mnt1/asom-net.dk/ >>> >>> root at mail:~# ls -ld /data/mnt1/asom-net.dk/kptest/ >>> drwx------ 9 vmail vmail 4096 Jan 23 08:55 >> /data/mnt1/asom-net.dk/kptest/ >>> root at mail:~# ls -ld /data/mnt1/asom-net.dk/kptest/maildirsize >>> -rw------- 1 vmail vmail 23 Jan 12 16:50 >>> /data/mnt1/asom-net.dk/kptest/maildirsize >>> >>> Anyone have any idea what might be wrong here? >>> >>> Regards, >>> From aki.tuomi at dovecot.fi Wed Feb 1 07:36:09 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 09:36:09 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> Message-ID: <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Because cram-md5 needs the user's password for calculating responses, it cannot work with hashed passwords (one-way encrypted). The only supported password schemes are PLAIN and CRAM-MD5. Aki On 01.02.2017 09:33, Poliman - Serwis wrote: > I always restart dovecot after change config. ;) Sure, I commented out > added two lines by me, restarted dovecot and here it is: > > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > auth_mechanisms = plain login cram-md5 > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_max_userip_connections = 100 > mail_plugins = " quota" > mail_privileged_group = vmail > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > sieve_max_redirects = 25 > } > postmaster_address = postmaster at example.com > protocols = imap pop3 > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service imap-login { > client_limit = 1000 > process_limit = 512 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > ssl = required > ssl_cert = ssl_cipher_list = > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > ssl_dh_parameters_length = 2048 > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = sieve quota > postmaster_address = webmaster at localhost > } > protocol lmtp { > mail_plugins = quota sieve > postmaster_address = webmaster at localhost > } > > > 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >> >> On 01.02.2017 08:18, Poliman - Serwis wrote: >>> This is debug log files in syslog: >>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: >>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >> m5ldD4= >>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: CONT >>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( >>> do_not_reply at example.com,12.173.211.32): query: SELECT email as user, >>> password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, >>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >> userdb_mail, >>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS >>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' >>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( >>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, >> but we >>> have only CRYPT >>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: >>> FAIL#0112#011user=do_not_reply at example.com >>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 authentication >>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5ldD4= >>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo >>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo >>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: >>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >> m5ldD4= >>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: CONT >>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( >>> do_not_reply at example.com,12.173.211.32): query: SELECT email as user, >>> password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, >>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >> userdb_mail, >>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') AS >>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' >>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( >>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, but >> we >>> have only CRYPT >>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: >>> FAIL#0113#011user=do_not_reply at example.com >>> >>> >>> >>> ##################### >>> I added in dovecot.conf lines in passdb block: >>> driver = passwd-file >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> and commented out default lines >>> #args = /etc/dovecot/dovecot-sql.conf >>> #driver = sql >>> When I try set again default lines I got above error >> Can you run doveconf -n with the configuration that causes the above >> error? Also it clearly does SQL lookup, so that error is happening with >> SQL passdb. You need to remember to restart dovecot between >> configuration changes. >> >> Aki >> >>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>> >>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>> I set up cram-md5 using this tutorial >>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in /etc/dovecot/dovecot.conf >> in >>>>> passdb code block: >>>>> listen = *,[::] >>>>> protocols = imap pop3 >>>>> #auth_mechanisms = plain login cram-md5 >>>>> auth_mechanisms = cram-md5 plain login >>>>> #dodana nizej linia >>>>> ssl = required >>>>> disable_plaintext_auth = yes >>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>> mail_privileged_group = vmail >>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>> ssl_cert = >>>> ssl_key = >>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>> ssl_cipher_list = >>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: >>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>> ssl_prefer_server_ciphers = yes >>>>> ssl_dh_parameters_length = 2048 >>>>> >>>>> >>>>> mail_max_userip_connections = 100 >>>>> passdb { >>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>> # driver = sql >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> } >>>>> userdb { >>>>> driver = prefetch >>>>> } >>>>> userdb { >>>>> args = /etc/dovecot/dovecot-sql.conf >>>>> driver = sql >>>>> } >>>>> Of course I created cram-md5.pwd file. All mails go out and come >> nicely. >>>>> But after I want to do default settings by commented out these two >> lines: >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> and uncomment >>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>> # driver = sql >>>>> I can't send emails - I use Thunderbird - get error "logging on server >>>>> mail.example.com not work out". Error in logs: >>>>> dovecot: auth-worker(22698): Error: Auth worker sees different >>>>> passdbs/userdbs than auth server. >>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>> >>>>> Is it possible that hashed password from cram-md5.pwd file was written >> to >>>>> database (if yes then where - I have ISPconfig)? I wasn't change any >>>> userdb >>>>> {} block and this second userdb block has this same lines like default >>>>> settings in passdb block. >>>>> >>>> Try >>>> >>>> auth_debug=yes >>>> auth_verbose=yes >>>> >>>> and see if it gives any more reasonable messages. >>>> >>>> Aki >>>> >>> > > From aki.tuomi at dovecot.fi Wed Feb 1 07:37:18 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 09:37:18 +0200 Subject: quota-status returns quota_status_success when email would put user over quota In-Reply-To: References: <58870F64.2030906@asom-net.dk> <5890AF03.30308@asom-net.dk> <26E5C771-BFF5-4982-BB79-69998F19FC80@valo.at> Message-ID: <19c1a131-476e-a228-9709-928094000e3a@dovecot.fi> Forgot to add the postfix config: smtpd_recipient_restrictions = ... check_policy_service inet:localhost:12340 Aki On 01.02.2017 09:34, Aki Tuomi wrote: > Steps to setup quota with status: > > mail_plugins = $mail_plugins quota > > protocol imap { > mail_plugins = $mail_plugins imap_quota > } > > service quota-status { > executable = quota-status -p postfix > inet_listener { > port = 12340 # You can choose any port you want > } > client_limit = 1 > } > > plugin { > quota = count:User quota # or some other backend > quota_rule = *:storage=10M # or from userdb > quota_grace = 10%% > # 10% is the default > quota_status_success = DUNNO > quota_status_nouser = DUNNO > quota_status_overquota = "552 5.2.2 Mailbox is full" > quota_vsizes = yes > } > > > On 01.02.2017 00:20, Christian Kivalo wrote: >> Am 31. J?nner 2017 16:36:35 MEZ schrieb Kristian Pedersen : >>> Hi list, >>> >>> We still did not manage to get quota-status working. >>> We're hoping someone can provide some feedback/ideas on how we may >>> investigate this issue further? >>> Is it likely to be a bug fixed in a newer version? >>> >>> Regards, >>> >>> Kristian >>> >>> >> [...] >> >>>> Quota-status will return unknown user if that is the case: >>>> root at mail:~# printf >>>> "recipient=kptest2 at asom-net.dk\nsize=1000000000\n\n" | nc -q1 >>>> localhost 12340 >>>> action=551 5.5.1 User not found >>>> >> Tried this and works here. Doveconf -n output with regards to quota settings is very similar, i use a quota dict, not maildir, spotted one difference i commented in your doveconf -n and i'm using version 2.2.27 from source >> ... >>>> dovecot -n: >>>> # 2.2.13: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 ext4 >>>> auth_default_realm = vejen-net.dk >>>> auth_mechanisms = plain login >>>> auth_verbose = yes >>>> disable_plaintext_auth = no >>>> first_valid_uid = 110 >>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>> mail_debug = yes >>>> mail_location = maildir:/data/vmail/%d/%n/ >> The one line i'm missing here from your doveconf -n output is mail_plugins = " quota" set in conf.d/10-mail.conf >> >> Have you added quota to the global mail plugins setting? http://wiki2.dovecot.org/Quota >> >>>> mail_privileged_group = mail >>>> namespace inbox { >>>> inbox = yes >>>> location = >>>> mailbox Drafts { >>>> special_use = \Drafts >>>> } >>>> mailbox Junk { >>>> special_use = \Junk >>>> } >>>> mailbox Sent { >>>> special_use = \Sent >>>> } >>>> mailbox "Sent Messages" { >>>> special_use = \Sent >>>> } >>>> mailbox Trash { >>>> special_use = \Trash >>>> } >>>> prefix = >>>> } >>>> passdb { >>>> args = /etc/dovecot/local-sql.conf >>>> driver = sql >>>> } >>>> plugin { >>>> quota = maildir:User quota >>>> quota_rule = *:storage=200M >>>> quota_status_nouser = 551 5.5.1 User not found >>>> quota_status_overquota = 552 5.2.2 Mailbox is full >>>> quota_status_success = DUNNO >>>> } >>>> protocols = imap pop3 >>>> service auth { >>>> unix_listener /var/spool/postfix/private/auth { >>>> group = postfix >>>> mode = 0660 >>>> user = postfix >>>> } >>>> unix_listener auth-master { >>>> mode = 0600 >>>> user = vmail >>>> } >>>> user = root >>>> } >>>> service imap-login { >>>> client_limit = 1024 >>>> process_limit = 256 >>>> process_min_avail = 8 >>>> service_count = 0 >>>> vsz_limit = 512 M >>>> } >>>> service imap { >>>> process_limit = 10240 >>>> } >>>> service pop3-login { >>>> client_limit = 512 >>>> process_limit = 256 >>>> process_min_avail = 8 >>>> service_count = 0 >>>> vsz_limit = 512 M >>>> } >>>> service pop3 { >>>> process_limit = 10240 >>>> } >>>> service quota-status { >>>> client_limit = 1 >>>> executable = /usr/lib/dovecot/quota-status -p postfix >>>> inet_listener { >>>> port = 12340 >>>> } >>>> } >>>> ssl_cert = >>> ssl_key = >>> ssl_prefer_server_ciphers = yes >>>> userdb { >>>> args = /etc/dovecot/local-sql.conf >>>> driver = sql >>>> } >>>> protocol lda { >>>> auth_socket_path = /var/run/dovecot/auth-master >>>> mail_plugins = quota >>>> postmaster_address = postmaster at asom-net.dk >>>> } >>>> protocol imap { >>>> mail_plugins = quota imap_quota >>>> } >>>> protocol pop3 { >>>> mail_plugins = quota >>>> pop3_uidl_format = %08Xu%08Xv >>>> } >>>> >>>> >>>> local-sql.conf: >>>> driver = mysql >>>> connect = host=xyz dbname=xyz user=xyz password=xyz >>>> default_pass_scheme = CRYPT >>>> password_query = SELECT email as user, password FROM virtual_users >>>> WHERE email='%u'; >>>> user_query = SELECT >>>> >>> CONCAT('/data/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) >>> >>>> AS home, 110 AS uid, 110 AS gid, CONCAT('*:storage=',mailquota,'M') >>> AS >>>> quota_rule FROM virtual_users WHERE email='%u'; >>>> >>>> If I do a strace on the quota-status PID, it seems to do a stat on >>> the >>>> directory and then give up? Not sure Im interpreting it correct (only >>>> including the last few lines): >>>> ... >>>> lseek(14, 833, SEEK_SET) = 833 >>>> munmap(0x7f165d32a000, 833) = 0 >>>> close(14) = 0 >>>> geteuid() = 0 >>>> getegid() = 110 >>>> getgid() = 110 >>>> getegid() = 110 >>>> setgroups(1, [110]) = 0 >>>> setresuid(-1, 110, -1) = 0 >>>> prctl(PR_SET_DUMPABLE, 1) = 0 >>>> stat("/data/vmail/asom-net.dk/kptest", {st_mode=S_IFDIR|0700, >>>> st_size=4096, ...}) = 0 >>>> prctl(PR_SET_DUMPABLE, 1) = 0 >>>> setsockopt(12, SOL_TCP, TCP_CORK, [1], 4) = 0 >>>> write(12, "action=DUNNO\n\n", 14) = 14 >>>> setsockopt(12, SOL_TCP, TCP_CORK, [0], 4) = 0 >>>> epoll_wait(11, {{EPOLLIN, {u32=1593554016, u64=139734059562080}}}, 5, >>>> 59999) = 1 >>>> read(12, "", 8146) = 0 >>>> epoll_ctl(11, EPOLL_CTL_DEL, 12, 7fff0be817a0) = 0 >>>> close(12) = 0 >>>> epoll_wait(11, {}, 5, 1000) = 0 >>>> write(5, "\35q\1\0007\10\0\0\1\0\0\0", 12) = 12 >>>> epoll_wait(11, >>>> ... >>>> >>>> It seems like sort of permission issue? >>>> root at mail:~# su - vmail >>>> No directory, logging in with HOME=/ >>>> $ id -a >>>> uid=110(vmail) gid=110(vmail) groups=110(vmail) >>>> $ cat /data/vmail/asom-net.dk/kptest/maildirsize >>>> 524288000S >>>> 685 1 >>>> 690 1 >>>> >>>> /data/vmail/asom-net.dk is actually a symlink, maybe that could be of >>>> importance?: >>>> root at mail:~# ls -ld /data/vmail/asom-net.dk >>>> lrwxrwxrwx 1 root root 19 Jan 9 11:18 /data/vmail/asom-net.dk -> >>>> ../mnt1/asom-net.dk >>>> >>>> root at mail:~# ls -ld /data/mnt1/asom-net.dk/ >>>> drwxrwx--- 45 vmail vmail 4096 Dec 15 10:54 /data/mnt1/asom-net.dk/ >>>> >>>> root at mail:~# ls -ld /data/mnt1/asom-net.dk/kptest/ >>>> drwx------ 9 vmail vmail 4096 Jan 23 08:55 >>> /data/mnt1/asom-net.dk/kptest/ >>>> root at mail:~# ls -ld /data/mnt1/asom-net.dk/kptest/maildirsize >>>> -rw------- 1 vmail vmail 23 Jan 12 16:50 >>>> /data/mnt1/asom-net.dk/kptest/maildirsize >>>> >>>> Anyone have any idea what might be wrong here? >>>> >>>> Regards, >>>> From serwis at poliman.pl Wed Feb 1 07:41:13 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 08:41:13 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: Default it was: "auth_mechanisms = plain login" and I added cram-md5. After restart all work perfectly. But after I added: driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd I can't set default lines because I got error. Please tell me which lines should be changed to resolve this issue. Should I remove "login" from auth_mechanism ("login" was default setting and I would like to move back to default settings)? 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > Because cram-md5 needs the user's password for calculating responses, it > cannot work with hashed passwords (one-way encrypted). The only > supported password schemes are PLAIN and CRAM-MD5. > > Aki > > On 01.02.2017 09:33, Poliman - Serwis wrote: > > I always restart dovecot after change config. ;) Sure, I commented out > > added two lines by me, restarted dovecot and here it is: > > > > # 2.2.9: /etc/dovecot/dovecot.conf > > # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > > auth_mechanisms = plain login cram-md5 > > listen = *,[::] > > log_timestamp = "%Y-%m-%d %H:%M:%S " > > mail_max_userip_connections = 100 > > mail_plugins = " quota" > > mail_privileged_group = vmail > > passdb { > > args = /etc/dovecot/dovecot-sql.conf > > driver = sql > > } > > plugin { > > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > > sieve = /var/vmail/%d/%n/.sieve > > sieve_max_redirects = 25 > > } > > postmaster_address = postmaster at example.com > > protocols = imap pop3 > > service auth { > > unix_listener /var/spool/postfix/private/auth { > > group = postfix > > mode = 0660 > > user = postfix > > } > > unix_listener auth-userdb { > > group = vmail > > mode = 0600 > > user = vmail > > } > > user = root > > } > > service imap-login { > > client_limit = 1000 > > process_limit = 512 > > } > > service lmtp { > > unix_listener /var/spool/postfix/private/dovecot-lmtp { > > group = postfix > > mode = 0600 > > user = postfix > > } > > } > > ssl = required > > ssl_cert = > ssl_cipher_list = > > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > > ssl_dh_parameters_length = 2048 > > ssl_key = > ssl_prefer_server_ciphers = yes > > ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > > userdb { > > driver = prefetch > > } > > userdb { > > args = /etc/dovecot/dovecot-sql.conf > > driver = sql > > } > > protocol imap { > > mail_plugins = quota imap_quota > > } > > protocol pop3 { > > mail_plugins = quota > > pop3_uidl_format = %08Xu%08Xv > > } > > protocol lda { > > mail_plugins = sieve quota > > postmaster_address = webmaster at localhost > > } > > protocol lmtp { > > mail_plugins = quota sieve > > postmaster_address = webmaster at localhost > > } > > > > > > 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > > > >> > >> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>> This is debug log files in syslog: > >>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: > >>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >> m5ldD4= > >>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: CONT > >>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>> do_not_reply at example.com,12.173.211.32): query: SELECT email as user, > >>> password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, > >>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >> userdb_mail, > >>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') > AS > >>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' > >>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > >>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, > >> but we > >>> have only CRYPT > >>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: > >>> FAIL#0112#011user=do_not_reply at example.com > >>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 authentication > >>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > dD4= > >>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo > >>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > >>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: > >>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >> m5ldD4= > >>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: CONT > >>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>> do_not_reply at example.com,12.173.211.32): query: SELECT email as user, > >>> password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, > >>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >> userdb_mail, > >>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') > AS > >>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' > >>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > >>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, > but > >> we > >>> have only CRYPT > >>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: > >>> FAIL#0113#011user=do_not_reply at example.com > >>> > >>> > >>> > >>> ##################### > >>> I added in dovecot.conf lines in passdb block: > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> and commented out default lines > >>> #args = /etc/dovecot/dovecot-sql.conf > >>> #driver = sql > >>> When I try set again default lines I got above error > >> Can you run doveconf -n with the configuration that causes the above > >> error? Also it clearly does SQL lookup, so that error is happening with > >> SQL passdb. You need to remember to restart dovecot between > >> configuration changes. > >> > >> Aki > >> > >>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>> > >>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>> I set up cram-md5 using this tutorial > >>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > /etc/dovecot/dovecot.conf > >> in > >>>>> passdb code block: > >>>>> listen = *,[::] > >>>>> protocols = imap pop3 > >>>>> #auth_mechanisms = plain login cram-md5 > >>>>> auth_mechanisms = cram-md5 plain login > >>>>> #dodana nizej linia > >>>>> ssl = required > >>>>> disable_plaintext_auth = yes > >>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>> mail_privileged_group = vmail > >>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>> ssl_cert = >>>>> ssl_key = >>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>> ssl_cipher_list = > >>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: > >>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>> ssl_prefer_server_ciphers = yes > >>>>> ssl_dh_parameters_length = 2048 > >>>>> > >>>>> > >>>>> mail_max_userip_connections = 100 > >>>>> passdb { > >>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>> # driver = sql > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> } > >>>>> userdb { > >>>>> driver = prefetch > >>>>> } > >>>>> userdb { > >>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>> driver = sql > >>>>> } > >>>>> Of course I created cram-md5.pwd file. All mails go out and come > >> nicely. > >>>>> But after I want to do default settings by commented out these two > >> lines: > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> and uncomment > >>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>> # driver = sql > >>>>> I can't send emails - I use Thunderbird - get error "logging on > server > >>>>> mail.example.com not work out". Error in logs: > >>>>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>>>> passdbs/userdbs than auth server. > >>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>> > >>>>> Is it possible that hashed password from cram-md5.pwd file was > written > >> to > >>>>> database (if yes then where - I have ISPconfig)? I wasn't change any > >>>> userdb > >>>>> {} block and this second userdb block has this same lines like > default > >>>>> settings in passdb block. > >>>>> > >>>> Try > >>>> > >>>> auth_debug=yes > >>>> auth_verbose=yes > >>>> > >>>> and see if it gives any more reasonable messages. > >>>> > >>>> Aki > >>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From aki.tuomi at dovecot.fi Wed Feb 1 07:45:09 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 09:45:09 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: You are probably wanting to do passdb { driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd } passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } Why you want to use cram-md5 is beyond me, because using SSL is much more safer. Aki On 01.02.2017 09:41, Poliman - Serwis wrote: > Default it was: "auth_mechanisms = plain login" and I added cram-md5. > After restart all work perfectly. But after I added: > driver = passwd-file > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > I can't set default lines because I got error. Please tell me which lines > should be changed to resolve this issue. Should I remove "login" from > auth_mechanism ("login" was default setting and I would like to move back > to default settings)? > > 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > >> Because cram-md5 needs the user's password for calculating responses, it >> cannot work with hashed passwords (one-way encrypted). The only >> supported password schemes are PLAIN and CRAM-MD5. >> >> Aki >> >> On 01.02.2017 09:33, Poliman - Serwis wrote: >>> I always restart dovecot after change config. ;) Sure, I commented out >>> added two lines by me, restarted dovecot and here it is: >>> >>> # 2.2.9: /etc/dovecot/dovecot.conf >>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>> auth_mechanisms = plain login cram-md5 >>> listen = *,[::] >>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> mail_max_userip_connections = 100 >>> mail_plugins = " quota" >>> mail_privileged_group = vmail >>> passdb { >>> args = /etc/dovecot/dovecot-sql.conf >>> driver = sql >>> } >>> plugin { >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>> sieve = /var/vmail/%d/%n/.sieve >>> sieve_max_redirects = 25 >>> } >>> postmaster_address = postmaster at example.com >>> protocols = imap pop3 >>> service auth { >>> unix_listener /var/spool/postfix/private/auth { >>> group = postfix >>> mode = 0660 >>> user = postfix >>> } >>> unix_listener auth-userdb { >>> group = vmail >>> mode = 0600 >>> user = vmail >>> } >>> user = root >>> } >>> service imap-login { >>> client_limit = 1000 >>> process_limit = 512 >>> } >>> service lmtp { >>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>> group = postfix >>> mode = 0600 >>> user = postfix >>> } >>> } >>> ssl = required >>> ssl_cert = >> ssl_cipher_list = >>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>> ssl_dh_parameters_length = 2048 >>> ssl_key = >> ssl_prefer_server_ciphers = yes >>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>> userdb { >>> driver = prefetch >>> } >>> userdb { >>> args = /etc/dovecot/dovecot-sql.conf >>> driver = sql >>> } >>> protocol imap { >>> mail_plugins = quota imap_quota >>> } >>> protocol pop3 { >>> mail_plugins = quota >>> pop3_uidl_format = %08Xu%08Xv >>> } >>> protocol lda { >>> mail_plugins = sieve quota >>> postmaster_address = webmaster at localhost >>> } >>> protocol lmtp { >>> mail_plugins = quota sieve >>> postmaster_address = webmaster at localhost >>> } >>> >>> >>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>> >>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>>>> This is debug log files in syslog: >>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: >>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>>> m5ldD4= >>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: CONT >>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( >>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as user, >>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, >>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>> userdb_mail, >>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') >> AS >>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' >>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( >>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, >>>> but we >>>>> have only CRYPT >>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: >>>>> FAIL#0112#011user=do_not_reply at example.com >>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 authentication >>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >> dD4= >>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo >>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo >>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: >>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>>> m5ldD4= >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: CONT >>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( >>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as user, >>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', maildir, >>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>> userdb_mail, >>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, 'B') >> AS >>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = '1' >>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( >>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, >> but >>>> we >>>>> have only CRYPT >>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: >>>>> FAIL#0113#011user=do_not_reply at example.com >>>>> >>>>> >>>>> >>>>> ##################### >>>>> I added in dovecot.conf lines in passdb block: >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> and commented out default lines >>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>> #driver = sql >>>>> When I try set again default lines I got above error >>>> Can you run doveconf -n with the configuration that causes the above >>>> error? Also it clearly does SQL lookup, so that error is happening with >>>> SQL passdb. You need to remember to restart dovecot between >>>> configuration changes. >>>> >>>> Aki >>>> >>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>>>> >>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>>>> I set up cram-md5 using this tutorial >>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >> /etc/dovecot/dovecot.conf >>>> in >>>>>>> passdb code block: >>>>>>> listen = *,[::] >>>>>>> protocols = imap pop3 >>>>>>> #auth_mechanisms = plain login cram-md5 >>>>>>> auth_mechanisms = cram-md5 plain login >>>>>>> #dodana nizej linia >>>>>>> ssl = required >>>>>>> disable_plaintext_auth = yes >>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>> mail_privileged_group = vmail >>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>>>> ssl_cert = >>>>>> ssl_key = >>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>> ssl_cipher_list = >>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: >>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>>>> ssl_prefer_server_ciphers = yes >>>>>>> ssl_dh_parameters_length = 2048 >>>>>>> >>>>>>> >>>>>>> mail_max_userip_connections = 100 >>>>>>> passdb { >>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>> # driver = sql >>>>>>> driver = passwd-file >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>> } >>>>>>> userdb { >>>>>>> driver = prefetch >>>>>>> } >>>>>>> userdb { >>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>> driver = sql >>>>>>> } >>>>>>> Of course I created cram-md5.pwd file. All mails go out and come >>>> nicely. >>>>>>> But after I want to do default settings by commented out these two >>>> lines: >>>>>>> driver = passwd-file >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>> and uncomment >>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>> # driver = sql >>>>>>> I can't send emails - I use Thunderbird - get error "logging on >> server >>>>>>> mail.example.com not work out". Error in logs: >>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different >>>>>>> passdbs/userdbs than auth server. >>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>>>> >>>>>>> Is it possible that hashed password from cram-md5.pwd file was >> written >>>> to >>>>>>> database (if yes then where - I have ISPconfig)? I wasn't change any >>>>>> userdb >>>>>>> {} block and this second userdb block has this same lines like >> default >>>>>>> settings in passdb block. >>>>>>> >>>>>> Try >>>>>> >>>>>> auth_debug=yes >>>>>> auth_verbose=yes >>>>>> >>>>>> and see if it gives any more reasonable messages. >>>>>> >>>>>> Aki >>>>>> >>> > > From serwis at poliman.pl Wed Feb 1 07:48:17 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 08:48:17 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: It was only for testing purposes. That's why I want change it back to default settings. ;) I will check above lines and give response asap. 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > You are probably wanting to do > passdb { > driver = passwd-file > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > } > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf > } > > Why you want to use cram-md5 is beyond me, because using SSL is much > more safer. > > Aki > > On 01.02.2017 09:41, Poliman - Serwis wrote: > > Default it was: "auth_mechanisms = plain login" and I added cram-md5. > > After restart all work perfectly. But after I added: > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > I can't set default lines because I got error. Please tell me which lines > > should be changed to resolve this issue. Should I remove "login" from > > auth_mechanism ("login" was default setting and I would like to move back > > to default settings)? > > > > 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > > > >> Because cram-md5 needs the user's password for calculating responses, it > >> cannot work with hashed passwords (one-way encrypted). The only > >> supported password schemes are PLAIN and CRAM-MD5. > >> > >> Aki > >> > >> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>> I always restart dovecot after change config. ;) Sure, I commented out > >>> added two lines by me, restarted dovecot and here it is: > >>> > >>> # 2.2.9: /etc/dovecot/dovecot.conf > >>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>> auth_mechanisms = plain login cram-md5 > >>> listen = *,[::] > >>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>> mail_max_userip_connections = 100 > >>> mail_plugins = " quota" > >>> mail_privileged_group = vmail > >>> passdb { > >>> args = /etc/dovecot/dovecot-sql.conf > >>> driver = sql > >>> } > >>> plugin { > >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>> sieve = /var/vmail/%d/%n/.sieve > >>> sieve_max_redirects = 25 > >>> } > >>> postmaster_address = postmaster at example.com > >>> protocols = imap pop3 > >>> service auth { > >>> unix_listener /var/spool/postfix/private/auth { > >>> group = postfix > >>> mode = 0660 > >>> user = postfix > >>> } > >>> unix_listener auth-userdb { > >>> group = vmail > >>> mode = 0600 > >>> user = vmail > >>> } > >>> user = root > >>> } > >>> service imap-login { > >>> client_limit = 1000 > >>> process_limit = 512 > >>> } > >>> service lmtp { > >>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>> group = postfix > >>> mode = 0600 > >>> user = postfix > >>> } > >>> } > >>> ssl = required > >>> ssl_cert = >>> ssl_cipher_list = > >>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>> ssl_dh_parameters_length = 2048 > >>> ssl_key = >>> ssl_prefer_server_ciphers = yes > >>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>> userdb { > >>> driver = prefetch > >>> } > >>> userdb { > >>> args = /etc/dovecot/dovecot-sql.conf > >>> driver = sql > >>> } > >>> protocol imap { > >>> mail_plugins = quota imap_quota > >>> } > >>> protocol pop3 { > >>> mail_plugins = quota > >>> pop3_uidl_format = %08Xu%08Xv > >>> } > >>> protocol lda { > >>> mail_plugins = sieve quota > >>> postmaster_address = webmaster at localhost > >>> } > >>> protocol lmtp { > >>> mail_plugins = quota sieve > >>> postmaster_address = webmaster at localhost > >>> } > >>> > >>> > >>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>> > >>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>> This is debug log files in syslog: > >>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>> m5ldD4= > >>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > CONT > >>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > user, > >>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > maildir, > >>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>> userdb_mail, > >>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > 'B') > >> AS > >>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = > '1' > >>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > >>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, > >>>> but we > >>>>> have only CRYPT > >>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > authentication > >>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >> dD4= > >>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do > echo > >>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > >>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>> m5ldD4= > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > CONT > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > user, > >>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > maildir, > >>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>> userdb_mail, > >>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > 'B') > >> AS > >>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = > '1' > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > >>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, > >> but > >>>> we > >>>>> have only CRYPT > >>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>> > >>>>> > >>>>> > >>>>> ##################### > >>>>> I added in dovecot.conf lines in passdb block: > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> and commented out default lines > >>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>> #driver = sql > >>>>> When I try set again default lines I got above error > >>>> Can you run doveconf -n with the configuration that causes the above > >>>> error? Also it clearly does SQL lookup, so that error is happening > with > >>>> SQL passdb. You need to remember to restart dovecot between > >>>> configuration changes. > >>>> > >>>> Aki > >>>> > >>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>> I set up cram-md5 using this tutorial > >>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >> /etc/dovecot/dovecot.conf > >>>> in > >>>>>>> passdb code block: > >>>>>>> listen = *,[::] > >>>>>>> protocols = imap pop3 > >>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>> #dodana nizej linia > >>>>>>> ssl = required > >>>>>>> disable_plaintext_auth = yes > >>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>> mail_privileged_group = vmail > >>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>> ssl_cert = >>>>>>> ssl_key = >>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>> ssl_cipher_list = > >>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: > >>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>> > >>>>>>> > >>>>>>> mail_max_userip_connections = 100 > >>>>>>> passdb { > >>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>> # driver = sql > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> } > >>>>>>> userdb { > >>>>>>> driver = prefetch > >>>>>>> } > >>>>>>> userdb { > >>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>> driver = sql > >>>>>>> } > >>>>>>> Of course I created cram-md5.pwd file. All mails go out and come > >>>> nicely. > >>>>>>> But after I want to do default settings by commented out these two > >>>> lines: > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> and uncomment > >>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>> # driver = sql > >>>>>>> I can't send emails - I use Thunderbird - get error "logging on > >> server > >>>>>>> mail.example.com not work out". Error in logs: > >>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>>>>>> passdbs/userdbs than auth server. > >>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>> > >>>>>>> Is it possible that hashed password from cram-md5.pwd file was > >> written > >>>> to > >>>>>>> database (if yes then where - I have ISPconfig)? I wasn't change > any > >>>>>> userdb > >>>>>>> {} block and this second userdb block has this same lines like > >> default > >>>>>>> settings in passdb block. > >>>>>>> > >>>>>> Try > >>>>>> > >>>>>> auth_debug=yes > >>>>>> auth_verbose=yes > >>>>>> > >>>>>> and see if it gives any more reasonable messages. > >>>>>> > >>>>>> Aki > >>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From serwis at poliman.pl Wed Feb 1 07:51:05 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 08:51:05 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: It still use: passdb { driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd } When I delete above and delete "cram-md5" in auth_mechanisms it still not working. 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > You are probably wanting to do > passdb { > driver = passwd-file > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > } > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf > } > > Why you want to use cram-md5 is beyond me, because using SSL is much > more safer. > > Aki > > On 01.02.2017 09:41, Poliman - Serwis wrote: > > Default it was: "auth_mechanisms = plain login" and I added cram-md5. > > After restart all work perfectly. But after I added: > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > I can't set default lines because I got error. Please tell me which lines > > should be changed to resolve this issue. Should I remove "login" from > > auth_mechanism ("login" was default setting and I would like to move back > > to default settings)? > > > > 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > > > >> Because cram-md5 needs the user's password for calculating responses, it > >> cannot work with hashed passwords (one-way encrypted). The only > >> supported password schemes are PLAIN and CRAM-MD5. > >> > >> Aki > >> > >> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>> I always restart dovecot after change config. ;) Sure, I commented out > >>> added two lines by me, restarted dovecot and here it is: > >>> > >>> # 2.2.9: /etc/dovecot/dovecot.conf > >>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>> auth_mechanisms = plain login cram-md5 > >>> listen = *,[::] > >>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>> mail_max_userip_connections = 100 > >>> mail_plugins = " quota" > >>> mail_privileged_group = vmail > >>> passdb { > >>> args = /etc/dovecot/dovecot-sql.conf > >>> driver = sql > >>> } > >>> plugin { > >>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>> sieve = /var/vmail/%d/%n/.sieve > >>> sieve_max_redirects = 25 > >>> } > >>> postmaster_address = postmaster at example.com > >>> protocols = imap pop3 > >>> service auth { > >>> unix_listener /var/spool/postfix/private/auth { > >>> group = postfix > >>> mode = 0660 > >>> user = postfix > >>> } > >>> unix_listener auth-userdb { > >>> group = vmail > >>> mode = 0600 > >>> user = vmail > >>> } > >>> user = root > >>> } > >>> service imap-login { > >>> client_limit = 1000 > >>> process_limit = 512 > >>> } > >>> service lmtp { > >>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>> group = postfix > >>> mode = 0600 > >>> user = postfix > >>> } > >>> } > >>> ssl = required > >>> ssl_cert = >>> ssl_cipher_list = > >>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>> ssl_dh_parameters_length = 2048 > >>> ssl_key = >>> ssl_prefer_server_ciphers = yes > >>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>> userdb { > >>> driver = prefetch > >>> } > >>> userdb { > >>> args = /etc/dovecot/dovecot-sql.conf > >>> driver = sql > >>> } > >>> protocol imap { > >>> mail_plugins = quota imap_quota > >>> } > >>> protocol pop3 { > >>> mail_plugins = quota > >>> pop3_uidl_format = %08Xu%08Xv > >>> } > >>> protocol lda { > >>> mail_plugins = sieve quota > >>> postmaster_address = webmaster at localhost > >>> } > >>> protocol lmtp { > >>> mail_plugins = quota sieve > >>> postmaster_address = webmaster at localhost > >>> } > >>> > >>> > >>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>> > >>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>> This is debug log files in syslog: > >>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>> m5ldD4= > >>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > CONT > >>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > user, > >>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > maildir, > >>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>> userdb_mail, > >>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > 'B') > >> AS > >>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = > '1' > >>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > >>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, > >>>> but we > >>>>> have only CRYPT > >>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > authentication > >>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >> dD4= > >>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do > echo > >>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > >>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>> m5ldD4= > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > CONT > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > user, > >>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > maildir, > >>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>> userdb_mail, > >>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > 'B') > >> AS > >>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = > '1' > >>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > >>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, > >> but > >>>> we > >>>>> have only CRYPT > >>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: > >>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>> > >>>>> > >>>>> > >>>>> ##################### > >>>>> I added in dovecot.conf lines in passdb block: > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> and commented out default lines > >>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>> #driver = sql > >>>>> When I try set again default lines I got above error > >>>> Can you run doveconf -n with the configuration that causes the above > >>>> error? Also it clearly does SQL lookup, so that error is happening > with > >>>> SQL passdb. You need to remember to restart dovecot between > >>>> configuration changes. > >>>> > >>>> Aki > >>>> > >>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>> I set up cram-md5 using this tutorial > >>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >> /etc/dovecot/dovecot.conf > >>>> in > >>>>>>> passdb code block: > >>>>>>> listen = *,[::] > >>>>>>> protocols = imap pop3 > >>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>> #dodana nizej linia > >>>>>>> ssl = required > >>>>>>> disable_plaintext_auth = yes > >>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>> mail_privileged_group = vmail > >>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>> ssl_cert = >>>>>>> ssl_key = >>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>> ssl_cipher_list = > >>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: > >>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>> > >>>>>>> > >>>>>>> mail_max_userip_connections = 100 > >>>>>>> passdb { > >>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>> # driver = sql > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> } > >>>>>>> userdb { > >>>>>>> driver = prefetch > >>>>>>> } > >>>>>>> userdb { > >>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>> driver = sql > >>>>>>> } > >>>>>>> Of course I created cram-md5.pwd file. All mails go out and come > >>>> nicely. > >>>>>>> But after I want to do default settings by commented out these two > >>>> lines: > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> and uncomment > >>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>> # driver = sql > >>>>>>> I can't send emails - I use Thunderbird - get error "logging on > >> server > >>>>>>> mail.example.com not work out". Error in logs: > >>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>>>>>> passdbs/userdbs than auth server. > >>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>> > >>>>>>> Is it possible that hashed password from cram-md5.pwd file was > >> written > >>>> to > >>>>>>> database (if yes then where - I have ISPconfig)? I wasn't change > any > >>>>>> userdb > >>>>>>> {} block and this second userdb block has this same lines like > >> default > >>>>>>> settings in passdb block. > >>>>>>> > >>>>>> Try > >>>>>> > >>>>>> auth_debug=yes > >>>>>> auth_verbose=yes > >>>>>> > >>>>>> and see if it gives any more reasonable messages. > >>>>>> > >>>>>> Aki > >>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From aki.tuomi at dovecot.fi Wed Feb 1 07:59:08 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 09:59:08 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: Are you still trying to authenticate using cram-md5? Aki On 01.02.2017 09:51, Poliman - Serwis wrote: > It still use: > passdb { > driver = passwd-file > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > } > > When I delete above and delete "cram-md5" in auth_mechanisms it still not > working. > > 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > >> You are probably wanting to do >> passdb { >> driver = passwd-file >> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> } >> >> passdb { >> driver = sql >> args = /etc/dovecot/dovecot-sql.conf >> } >> >> Why you want to use cram-md5 is beyond me, because using SSL is much >> more safer. >> >> Aki >> >> On 01.02.2017 09:41, Poliman - Serwis wrote: >>> Default it was: "auth_mechanisms = plain login" and I added cram-md5. >>> After restart all work perfectly. But after I added: >>> driver = passwd-file >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> I can't set default lines because I got error. Please tell me which lines >>> should be changed to resolve this issue. Should I remove "login" from >>> auth_mechanism ("login" was default setting and I would like to move back >>> to default settings)? >>> >>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >>> >>>> Because cram-md5 needs the user's password for calculating responses, it >>>> cannot work with hashed passwords (one-way encrypted). The only >>>> supported password schemes are PLAIN and CRAM-MD5. >>>> >>>> Aki >>>> >>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >>>>> I always restart dovecot after change config. ;) Sure, I commented out >>>>> added two lines by me, restarted dovecot and here it is: >>>>> >>>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>>>> auth_mechanisms = plain login cram-md5 >>>>> listen = *,[::] >>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>> mail_max_userip_connections = 100 >>>>> mail_plugins = " quota" >>>>> mail_privileged_group = vmail >>>>> passdb { >>>>> args = /etc/dovecot/dovecot-sql.conf >>>>> driver = sql >>>>> } >>>>> plugin { >>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>>> sieve = /var/vmail/%d/%n/.sieve >>>>> sieve_max_redirects = 25 >>>>> } >>>>> postmaster_address = postmaster at example.com >>>>> protocols = imap pop3 >>>>> service auth { >>>>> unix_listener /var/spool/postfix/private/auth { >>>>> group = postfix >>>>> mode = 0660 >>>>> user = postfix >>>>> } >>>>> unix_listener auth-userdb { >>>>> group = vmail >>>>> mode = 0600 >>>>> user = vmail >>>>> } >>>>> user = root >>>>> } >>>>> service imap-login { >>>>> client_limit = 1000 >>>>> process_limit = 512 >>>>> } >>>>> service lmtp { >>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>>> group = postfix >>>>> mode = 0600 >>>>> user = postfix >>>>> } >>>>> } >>>>> ssl = required >>>>> ssl_cert = >>>> ssl_cipher_list = >>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>>>> ssl_dh_parameters_length = 2048 >>>>> ssl_key = >>>> ssl_prefer_server_ciphers = yes >>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>> userdb { >>>>> driver = prefetch >>>>> } >>>>> userdb { >>>>> args = /etc/dovecot/dovecot-sql.conf >>>>> driver = sql >>>>> } >>>>> protocol imap { >>>>> mail_plugins = quota imap_quota >>>>> } >>>>> protocol pop3 { >>>>> mail_plugins = quota >>>>> pop3_uidl_format = %08Xu%08Xv >>>>> } >>>>> protocol lda { >>>>> mail_plugins = sieve quota >>>>> postmaster_address = webmaster at localhost >>>>> } >>>>> protocol lmtp { >>>>> mail_plugins = quota sieve >>>>> postmaster_address = webmaster at localhost >>>>> } >>>>> >>>>> >>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>>>> >>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>>>>>> This is debug log files in syslog: >>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>>>>> m5ldD4= >>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >> CONT >>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( >>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as >> user, >>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >> maildir, >>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>> userdb_mail, >>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, >> 'B') >>>> AS >>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = >> '1' >>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( >>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 scheme, >>>>>> but we >>>>>>> have only CRYPT >>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>> FAIL#0112#011user=do_not_reply at example.com >>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >> authentication >>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NTkyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >>>> dD4= >>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do >> echo >>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo >>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>>>>> m5ldD4= >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >> CONT >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( >>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as >> user, >>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >> maildir, >>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>> userdb_mail, >>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, >> 'B') >>>> AS >>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = >> '1' >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( >>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 scheme, >>>> but >>>>>> we >>>>>>> have only CRYPT >>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>> FAIL#0113#011user=do_not_reply at example.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> ##################### >>>>>>> I added in dovecot.conf lines in passdb block: >>>>>>> driver = passwd-file >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>> and commented out default lines >>>>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>>>> #driver = sql >>>>>>> When I try set again default lines I got above error >>>>>> Can you run doveconf -n with the configuration that causes the above >>>>>> error? Also it clearly does SQL lookup, so that error is happening >> with >>>>>> SQL passdb. You need to remember to restart dovecot between >>>>>> configuration changes. >>>>>> >>>>>> Aki >>>>>> >>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>>>>>> >>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>>>>>> I set up cram-md5 using this tutorial >>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >>>> /etc/dovecot/dovecot.conf >>>>>> in >>>>>>>>> passdb code block: >>>>>>>>> listen = *,[::] >>>>>>>>> protocols = imap pop3 >>>>>>>>> #auth_mechanisms = plain login cram-md5 >>>>>>>>> auth_mechanisms = cram-md5 plain login >>>>>>>>> #dodana nizej linia >>>>>>>>> ssl = required >>>>>>>>> disable_plaintext_auth = yes >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>> mail_privileged_group = vmail >>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>>>>>> ssl_cert = >>>>>>>> ssl_key = >>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>> ssl_cipher_list = >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: >>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>> >>>>>>>>> >>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>> passdb { >>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>> # driver = sql >>>>>>>>> driver = passwd-file >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>> } >>>>>>>>> userdb { >>>>>>>>> driver = prefetch >>>>>>>>> } >>>>>>>>> userdb { >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>> driver = sql >>>>>>>>> } >>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and come >>>>>> nicely. >>>>>>>>> But after I want to do default settings by commented out these two >>>>>> lines: >>>>>>>>> driver = passwd-file >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>> and uncomment >>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>> # driver = sql >>>>>>>>> I can't send emails - I use Thunderbird - get error "logging on >>>> server >>>>>>>>> mail.example.com not work out". Error in logs: >>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different >>>>>>>>> passdbs/userdbs than auth server. >>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>>>>>> >>>>>>>>> Is it possible that hashed password from cram-md5.pwd file was >>>> written >>>>>> to >>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't change >> any >>>>>>>> userdb >>>>>>>>> {} block and this second userdb block has this same lines like >>>> default >>>>>>>>> settings in passdb block. >>>>>>>>> >>>>>>>> Try >>>>>>>> >>>>>>>> auth_debug=yes >>>>>>>> auth_verbose=yes >>>>>>>> >>>>>>>> and see if it gives any more reasonable messages. >>>>>>>> >>>>>>>> Aki >>>>>>>> >>> > > From serwis at poliman.pl Wed Feb 1 08:02:24 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 09:02:24 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: When I used backup copy of the dovecot.conf file I have this same error. So I think that maybe something was written to database? I really would point out that I only added passdb { driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd } and comment out from above block default lines #args = /etc/dovecot/dovecot-sql.conf #driver = sql And in auth_mechanisms add line cram-md5. Nothing more in any other file. I don't want to use cram-md5. I need move back to default settings. Cram-md5 was only for testing purposes. :) But I supposed that I can move back to default by commenting out added lines. But unfortunately it isn't that simple. 2017-02-01 8:59 GMT+01:00 Aki Tuomi : > Are you still trying to authenticate using cram-md5? > > Aki > > > On 01.02.2017 09:51, Poliman - Serwis wrote: > > It still use: > > passdb { > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > } > > > > When I delete above and delete "cram-md5" in auth_mechanisms it still not > > working. > > > > 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > > > >> You are probably wanting to do > >> passdb { > >> driver = passwd-file > >> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >> } > >> > >> passdb { > >> driver = sql > >> args = /etc/dovecot/dovecot-sql.conf > >> } > >> > >> Why you want to use cram-md5 is beyond me, because using SSL is much > >> more safer. > >> > >> Aki > >> > >> On 01.02.2017 09:41, Poliman - Serwis wrote: > >>> Default it was: "auth_mechanisms = plain login" and I added cram-md5. > >>> After restart all work perfectly. But after I added: > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> I can't set default lines because I got error. Please tell me which > lines > >>> should be changed to resolve this issue. Should I remove "login" from > >>> auth_mechanism ("login" was default setting and I would like to move > back > >>> to default settings)? > >>> > >>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > >>> > >>>> Because cram-md5 needs the user's password for calculating responses, > it > >>>> cannot work with hashed passwords (one-way encrypted). The only > >>>> supported password schemes are PLAIN and CRAM-MD5. > >>>> > >>>> Aki > >>>> > >>>> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>>>> I always restart dovecot after change config. ;) Sure, I commented > out > >>>>> added two lines by me, restarted dovecot and here it is: > >>>>> > >>>>> # 2.2.9: /etc/dovecot/dovecot.conf > >>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>>>> auth_mechanisms = plain login cram-md5 > >>>>> listen = *,[::] > >>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>> mail_max_userip_connections = 100 > >>>>> mail_plugins = " quota" > >>>>> mail_privileged_group = vmail > >>>>> passdb { > >>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>> driver = sql > >>>>> } > >>>>> plugin { > >>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>>>> sieve = /var/vmail/%d/%n/.sieve > >>>>> sieve_max_redirects = 25 > >>>>> } > >>>>> postmaster_address = postmaster at example.com > >>>>> protocols = imap pop3 > >>>>> service auth { > >>>>> unix_listener /var/spool/postfix/private/auth { > >>>>> group = postfix > >>>>> mode = 0660 > >>>>> user = postfix > >>>>> } > >>>>> unix_listener auth-userdb { > >>>>> group = vmail > >>>>> mode = 0600 > >>>>> user = vmail > >>>>> } > >>>>> user = root > >>>>> } > >>>>> service imap-login { > >>>>> client_limit = 1000 > >>>>> process_limit = 512 > >>>>> } > >>>>> service lmtp { > >>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>>>> group = postfix > >>>>> mode = 0600 > >>>>> user = postfix > >>>>> } > >>>>> } > >>>>> ssl = required > >>>>> ssl_cert = >>>>> ssl_cipher_list = > >>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>>>> ssl_dh_parameters_length = 2048 > >>>>> ssl_key = >>>>> ssl_prefer_server_ciphers = yes > >>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>> userdb { > >>>>> driver = prefetch > >>>>> } > >>>>> userdb { > >>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>> driver = sql > >>>>> } > >>>>> protocol imap { > >>>>> mail_plugins = quota imap_quota > >>>>> } > >>>>> protocol pop3 { > >>>>> mail_plugins = quota > >>>>> pop3_uidl_format = %08Xu%08Xv > >>>>> } > >>>>> protocol lda { > >>>>> mail_plugins = sieve quota > >>>>> postmaster_address = webmaster at localhost > >>>>> } > >>>>> protocol lmtp { > >>>>> mail_plugins = quota sieve > >>>>> postmaster_address = webmaster at localhost > >>>>> } > >>>>> > >>>>> > >>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>>>> This is debug log files in syslog: > >>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: > >>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > >> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>>>> m5ldD4= > >>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > >> CONT > >>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > >> user, > >>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >> maildir, > >>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>> userdb_mail, > >>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > >> 'B') > >>>> AS > >>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = > >> '1' > >>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > >>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 > scheme, > >>>>>> but we > >>>>>>> have only CRYPT > >>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: > >>>>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > >> authentication > >>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT > kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >>>> dD4= > >>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do > >> echo > >>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do > echo > >>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: > >>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > >> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>>>> m5ldD4= > >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >> CONT > >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( > >>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > >> user, > >>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >> maildir, > >>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>> userdb_mail, > >>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > >> 'B') > >>>> AS > >>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM > >>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = > >> '1' > >>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > >>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 > scheme, > >>>> but > >>>>>> we > >>>>>>> have only CRYPT > >>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: > >>>>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> ##################### > >>>>>>> I added in dovecot.conf lines in passdb block: > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> and commented out default lines > >>>>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>>>> #driver = sql > >>>>>>> When I try set again default lines I got above error > >>>>>> Can you run doveconf -n with the configuration that causes the above > >>>>>> error? Also it clearly does SQL lookup, so that error is happening > >> with > >>>>>> SQL passdb. You need to remember to restart dovecot between > >>>>>> configuration changes. > >>>>>> > >>>>>> Aki > >>>>>> > >>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>>>> > >>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>>>> I set up cram-md5 using this tutorial > >>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >>>> /etc/dovecot/dovecot.conf > >>>>>> in > >>>>>>>>> passdb code block: > >>>>>>>>> listen = *,[::] > >>>>>>>>> protocols = imap pop3 > >>>>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>>>> #dodana nizej linia > >>>>>>>>> ssl = required > >>>>>>>>> disable_plaintext_auth = yes > >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>> mail_privileged_group = vmail > >>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>>>> ssl_cert = >>>>>>>>> ssl_key = >>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>> ssl_cipher_list = > >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: > >>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>> passdb { > >>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> # driver = sql > >>>>>>>>> driver = passwd-file > >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>> } > >>>>>>>>> userdb { > >>>>>>>>> driver = prefetch > >>>>>>>>> } > >>>>>>>>> userdb { > >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> driver = sql > >>>>>>>>> } > >>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and come > >>>>>> nicely. > >>>>>>>>> But after I want to do default settings by commented out these > two > >>>>>> lines: > >>>>>>>>> driver = passwd-file > >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>> and uncomment > >>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> # driver = sql > >>>>>>>>> I can't send emails - I use Thunderbird - get error "logging on > >>>> server > >>>>>>>>> mail.example.com not work out". Error in logs: > >>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>>>>>>>> passdbs/userdbs than auth server. > >>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>>>> > >>>>>>>>> Is it possible that hashed password from cram-md5.pwd file was > >>>> written > >>>>>> to > >>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't change > >> any > >>>>>>>> userdb > >>>>>>>>> {} block and this second userdb block has this same lines like > >>>> default > >>>>>>>>> settings in passdb block. > >>>>>>>>> > >>>>>>>> Try > >>>>>>>> > >>>>>>>> auth_debug=yes > >>>>>>>> auth_verbose=yes > >>>>>>>> > >>>>>>>> and see if it gives any more reasonable messages. > >>>>>>>> > >>>>>>>> Aki > >>>>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From aki.tuomi at dovecot.fi Wed Feb 1 08:04:26 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 10:04:26 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: Can you check your logs? Aki On 01.02.2017 10:02, Poliman - Serwis wrote: > When I used backup copy of the dovecot.conf file I have this same error. So > I think that maybe something was written to database? I really would point > out that I only added > passdb { > driver = passwd-file > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > } > > and comment out from above block default lines > #args = /etc/dovecot/dovecot-sql.conf > #driver = sql > > And in auth_mechanisms add line cram-md5. Nothing more in any other file. > > I don't want to use cram-md5. I need move back to default settings. > Cram-md5 was only for testing purposes. :) But I supposed that I can move > back to default by commenting out added lines. But unfortunately it isn't > that simple. > > 2017-02-01 8:59 GMT+01:00 Aki Tuomi : > >> Are you still trying to authenticate using cram-md5? >> >> Aki >> >> >> On 01.02.2017 09:51, Poliman - Serwis wrote: >>> It still use: >>> passdb { >>> driver = passwd-file >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> } >>> >>> When I delete above and delete "cram-md5" in auth_mechanisms it still not >>> working. >>> >>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : >>> >>>> You are probably wanting to do >>>> passdb { >>>> driver = passwd-file >>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>> } >>>> >>>> passdb { >>>> driver = sql >>>> args = /etc/dovecot/dovecot-sql.conf >>>> } >>>> >>>> Why you want to use cram-md5 is beyond me, because using SSL is much >>>> more safer. >>>> >>>> Aki >>>> >>>> On 01.02.2017 09:41, Poliman - Serwis wrote: >>>>> Default it was: "auth_mechanisms = plain login" and I added cram-md5. >>>>> After restart all work perfectly. But after I added: >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> I can't set default lines because I got error. Please tell me which >> lines >>>>> should be changed to resolve this issue. Should I remove "login" from >>>>> auth_mechanism ("login" was default setting and I would like to move >> back >>>>> to default settings)? >>>>> >>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >>>>> >>>>>> Because cram-md5 needs the user's password for calculating responses, >> it >>>>>> cannot work with hashed passwords (one-way encrypted). The only >>>>>> supported password schemes are PLAIN and CRAM-MD5. >>>>>> >>>>>> Aki >>>>>> >>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >>>>>>> I always restart dovecot after change config. ;) Sure, I commented >> out >>>>>>> added two lines by me, restarted dovecot and here it is: >>>>>>> >>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>>>>>> auth_mechanisms = plain login cram-md5 >>>>>>> listen = *,[::] >>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>> mail_max_userip_connections = 100 >>>>>>> mail_plugins = " quota" >>>>>>> mail_privileged_group = vmail >>>>>>> passdb { >>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>> driver = sql >>>>>>> } >>>>>>> plugin { >>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>>>>> sieve = /var/vmail/%d/%n/.sieve >>>>>>> sieve_max_redirects = 25 >>>>>>> } >>>>>>> postmaster_address = postmaster at example.com >>>>>>> protocols = imap pop3 >>>>>>> service auth { >>>>>>> unix_listener /var/spool/postfix/private/auth { >>>>>>> group = postfix >>>>>>> mode = 0660 >>>>>>> user = postfix >>>>>>> } >>>>>>> unix_listener auth-userdb { >>>>>>> group = vmail >>>>>>> mode = 0600 >>>>>>> user = vmail >>>>>>> } >>>>>>> user = root >>>>>>> } >>>>>>> service imap-login { >>>>>>> client_limit = 1000 >>>>>>> process_limit = 512 >>>>>>> } >>>>>>> service lmtp { >>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>>>>> group = postfix >>>>>>> mode = 0600 >>>>>>> user = postfix >>>>>>> } >>>>>>> } >>>>>>> ssl = required >>>>>>> ssl_cert = >>>>>> ssl_cipher_list = >>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>>>>>> ssl_dh_parameters_length = 2048 >>>>>>> ssl_key = >>>>>> ssl_prefer_server_ciphers = yes >>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>> userdb { >>>>>>> driver = prefetch >>>>>>> } >>>>>>> userdb { >>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>> driver = sql >>>>>>> } >>>>>>> protocol imap { >>>>>>> mail_plugins = quota imap_quota >>>>>>> } >>>>>>> protocol pop3 { >>>>>>> mail_plugins = quota >>>>>>> pop3_uidl_format = %08Xu%08Xv >>>>>>> } >>>>>>> protocol lda { >>>>>>> mail_plugins = sieve quota >>>>>>> postmaster_address = webmaster at localhost >>>>>>> } >>>>>>> protocol lmtp { >>>>>>> mail_plugins = quota sieve >>>>>>> postmaster_address = webmaster at localhost >>>>>>> } >>>>>>> >>>>>>> >>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>>>>>> >>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>>>>>>>> This is debug log files in syslog: >>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>>>>>>> m5ldD4= >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >>>> CONT >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: sql( >>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as >>>> user, >>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>> maildir, >>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>> userdb_mail, >>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, >>>> 'B') >>>>>> AS >>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = >>>> '1' >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( >>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 >> scheme, >>>>>>>> but we >>>>>>>>> have only CRYPT >>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>>>> FAIL#0112#011user=do_not_reply at example.com >>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >>>> authentication >>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT >> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >>>>>> dD4= >>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do >>>> echo >>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do >> echo >>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>>>>>>> m5ldD4= >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>> CONT >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: sql( >>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as >>>> user, >>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>> maildir, >>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>> userdb_mail, >>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, >>>> 'B') >>>>>> AS >>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve FROM >>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND server_id = >>>> '1' >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( >>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 >> scheme, >>>>>> but >>>>>>>> we >>>>>>>>> have only CRYPT >>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb out: >>>>>>>>> FAIL#0113#011user=do_not_reply at example.com >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ##################### >>>>>>>>> I added in dovecot.conf lines in passdb block: >>>>>>>>> driver = passwd-file >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>> and commented out default lines >>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>>>>>> #driver = sql >>>>>>>>> When I try set again default lines I got above error >>>>>>>> Can you run doveconf -n with the configuration that causes the above >>>>>>>> error? Also it clearly does SQL lookup, so that error is happening >>>> with >>>>>>>> SQL passdb. You need to remember to restart dovecot between >>>>>>>> configuration changes. >>>>>>>> >>>>>>>> Aki >>>>>>>> >>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>>>>>>>> >>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>>>>>>>> I set up cram-md5 using this tutorial >>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >>>>>> /etc/dovecot/dovecot.conf >>>>>>>> in >>>>>>>>>>> passdb code block: >>>>>>>>>>> listen = *,[::] >>>>>>>>>>> protocols = imap pop3 >>>>>>>>>>> #auth_mechanisms = plain login cram-md5 >>>>>>>>>>> auth_mechanisms = cram-md5 plain login >>>>>>>>>>> #dodana nizej linia >>>>>>>>>>> ssl = required >>>>>>>>>>> disable_plaintext_auth = yes >>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>>>> mail_privileged_group = vmail >>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>>>>>>>> ssl_cert = >>>>>>>>>> ssl_key = >>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>>>> ssl_cipher_list = >>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: >>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>>>> passdb { >>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> # driver = sql >>>>>>>>>>> driver = passwd-file >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>> } >>>>>>>>>>> userdb { >>>>>>>>>>> driver = prefetch >>>>>>>>>>> } >>>>>>>>>>> userdb { >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> driver = sql >>>>>>>>>>> } >>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and come >>>>>>>> nicely. >>>>>>>>>>> But after I want to do default settings by commented out these >> two >>>>>>>> lines: >>>>>>>>>>> driver = passwd-file >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>> and uncomment >>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> # driver = sql >>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging on >>>>>> server >>>>>>>>>>> mail.example.com not work out". Error in logs: >>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different >>>>>>>>>>> passdbs/userdbs than auth server. >>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>>>>>>>> >>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file was >>>>>> written >>>>>>>> to >>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't change >>>> any >>>>>>>>>> userdb >>>>>>>>>>> {} block and this second userdb block has this same lines like >>>>>> default >>>>>>>>>>> settings in passdb block. >>>>>>>>>>> >>>>>>>>>> Try >>>>>>>>>> >>>>>>>>>> auth_debug=yes >>>>>>>>>> auth_verbose=yes >>>>>>>>>> >>>>>>>>>> and see if it gives any more reasonable messages. >>>>>>>>>> >>>>>>>>>> Aki >>>>>>>>>> >>> > > From serwis at poliman.pl Wed Feb 1 08:07:42 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 09:07:42 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: Logs from syslog or mail.err? And with these not working settings with auth_debug and auth_verbose? 2017-02-01 9:04 GMT+01:00 Aki Tuomi : > Can you check your logs? > > Aki > > > On 01.02.2017 10:02, Poliman - Serwis wrote: > > When I used backup copy of the dovecot.conf file I have this same error. > So > > I think that maybe something was written to database? I really would > point > > out that I only added > > passdb { > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > } > > > > and comment out from above block default lines > > #args = /etc/dovecot/dovecot-sql.conf > > #driver = sql > > > > And in auth_mechanisms add line cram-md5. Nothing more in any other file. > > > > I don't want to use cram-md5. I need move back to default settings. > > Cram-md5 was only for testing purposes. :) But I supposed that I can move > > back to default by commenting out added lines. But unfortunately it isn't > > that simple. > > > > 2017-02-01 8:59 GMT+01:00 Aki Tuomi : > > > >> Are you still trying to authenticate using cram-md5? > >> > >> Aki > >> > >> > >> On 01.02.2017 09:51, Poliman - Serwis wrote: > >>> It still use: > >>> passdb { > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> } > >>> > >>> When I delete above and delete "cram-md5" in auth_mechanisms it still > not > >>> working. > >>> > >>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > >>> > >>>> You are probably wanting to do > >>>> passdb { > >>>> driver = passwd-file > >>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>> } > >>>> > >>>> passdb { > >>>> driver = sql > >>>> args = /etc/dovecot/dovecot-sql.conf > >>>> } > >>>> > >>>> Why you want to use cram-md5 is beyond me, because using SSL is much > >>>> more safer. > >>>> > >>>> Aki > >>>> > >>>> On 01.02.2017 09:41, Poliman - Serwis wrote: > >>>>> Default it was: "auth_mechanisms = plain login" and I added > cram-md5. > >>>>> After restart all work perfectly. But after I added: > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> I can't set default lines because I got error. Please tell me which > >> lines > >>>>> should be changed to resolve this issue. Should I remove "login" from > >>>>> auth_mechanism ("login" was default setting and I would like to move > >> back > >>>>> to default settings)? > >>>>> > >>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> Because cram-md5 needs the user's password for calculating > responses, > >> it > >>>>>> cannot work with hashed passwords (one-way encrypted). The only > >>>>>> supported password schemes are PLAIN and CRAM-MD5. > >>>>>> > >>>>>> Aki > >>>>>> > >>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>>>>>> I always restart dovecot after change config. ;) Sure, I commented > >> out > >>>>>>> added two lines by me, restarted dovecot and here it is: > >>>>>>> > >>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf > >>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>>>>>> auth_mechanisms = plain login cram-md5 > >>>>>>> listen = *,[::] > >>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>> mail_max_userip_connections = 100 > >>>>>>> mail_plugins = " quota" > >>>>>>> mail_privileged_group = vmail > >>>>>>> passdb { > >>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>> driver = sql > >>>>>>> } > >>>>>>> plugin { > >>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>>>>>> sieve = /var/vmail/%d/%n/.sieve > >>>>>>> sieve_max_redirects = 25 > >>>>>>> } > >>>>>>> postmaster_address = postmaster at example.com > >>>>>>> protocols = imap pop3 > >>>>>>> service auth { > >>>>>>> unix_listener /var/spool/postfix/private/auth { > >>>>>>> group = postfix > >>>>>>> mode = 0660 > >>>>>>> user = postfix > >>>>>>> } > >>>>>>> unix_listener auth-userdb { > >>>>>>> group = vmail > >>>>>>> mode = 0600 > >>>>>>> user = vmail > >>>>>>> } > >>>>>>> user = root > >>>>>>> } > >>>>>>> service imap-login { > >>>>>>> client_limit = 1000 > >>>>>>> process_limit = 512 > >>>>>>> } > >>>>>>> service lmtp { > >>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>>>>>> group = postfix > >>>>>>> mode = 0600 > >>>>>>> user = postfix > >>>>>>> } > >>>>>>> } > >>>>>>> ssl = required > >>>>>>> ssl_cert = >>>>>>> ssl_cipher_list = > >>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>> ssl_key = >>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>> userdb { > >>>>>>> driver = prefetch > >>>>>>> } > >>>>>>> userdb { > >>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>> driver = sql > >>>>>>> } > >>>>>>> protocol imap { > >>>>>>> mail_plugins = quota imap_quota > >>>>>>> } > >>>>>>> protocol pop3 { > >>>>>>> mail_plugins = quota > >>>>>>> pop3_uidl_format = %08Xu%08Xv > >>>>>>> } > >>>>>>> protocol lda { > >>>>>>> mail_plugins = sieve quota > >>>>>>> postmaster_address = webmaster at localhost > >>>>>>> } > >>>>>>> protocol lmtp { > >>>>>>> mail_plugins = quota sieve > >>>>>>> postmaster_address = webmaster at localhost > >>>>>>> } > >>>>>>> > >>>>>>> > >>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>>>>>> > >>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>>>>>> This is debug log files in syslog: > >>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > >>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>>>>>> m5ldD4= > >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > >>>> CONT > >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: > sql( > >>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > >>>> user, > >>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>> maildir, > >>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>> userdb_mail, > >>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > >>>> 'B') > >>>>>> AS > >>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > FROM > >>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > server_id = > >>>> '1' > >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > >>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 > >> scheme, > >>>>>>>> but we > >>>>>>>>> have only CRYPT > >>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > >>>> authentication > >>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT > >> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >>>>>> dD4= > >>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; > do > >>>> echo > >>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do > >> echo > >>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > >>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>>>>>> m5ldD4= > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>> CONT > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: > sql( > >>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > >>>> user, > >>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>> maildir, > >>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>> userdb_mail, > >>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > >>>> 'B') > >>>>>> AS > >>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > FROM > >>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > server_id = > >>>> '1' > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > >>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 > >> scheme, > >>>>>> but > >>>>>>>> we > >>>>>>>>> have only CRYPT > >>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ##################### > >>>>>>>>> I added in dovecot.conf lines in passdb block: > >>>>>>>>> driver = passwd-file > >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>> and commented out default lines > >>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> #driver = sql > >>>>>>>>> When I try set again default lines I got above error > >>>>>>>> Can you run doveconf -n with the configuration that causes the > above > >>>>>>>> error? Also it clearly does SQL lookup, so that error is happening > >>>> with > >>>>>>>> SQL passdb. You need to remember to restart dovecot between > >>>>>>>> configuration changes. > >>>>>>>> > >>>>>>>> Aki > >>>>>>>> > >>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>>>>>> > >>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>>>>>> I set up cram-md5 using this tutorial > >>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >>>>>> /etc/dovecot/dovecot.conf > >>>>>>>> in > >>>>>>>>>>> passdb code block: > >>>>>>>>>>> listen = *,[::] > >>>>>>>>>>> protocols = imap pop3 > >>>>>>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>>>>>> #dodana nizej linia > >>>>>>>>>>> ssl = required > >>>>>>>>>>> disable_plaintext_auth = yes > >>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>>>> mail_privileged_group = vmail > >>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>>>>>> ssl_cert = >>>>>>>>>>> ssl_key = >>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>>>> ssl_cipher_list = > >>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ > image: > >>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>>>> passdb { > >>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> # driver = sql > >>>>>>>>>>> driver = passwd-file > >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>> } > >>>>>>>>>>> userdb { > >>>>>>>>>>> driver = prefetch > >>>>>>>>>>> } > >>>>>>>>>>> userdb { > >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> driver = sql > >>>>>>>>>>> } > >>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and > come > >>>>>>>> nicely. > >>>>>>>>>>> But after I want to do default settings by commented out these > >> two > >>>>>>>> lines: > >>>>>>>>>>> driver = passwd-file > >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>> and uncomment > >>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> # driver = sql > >>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging on > >>>>>> server > >>>>>>>>>>> mail.example.com not work out". Error in logs: > >>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>>>>>>>>>> passdbs/userdbs than auth server. > >>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>>>>>> > >>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file was > >>>>>> written > >>>>>>>> to > >>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't > change > >>>> any > >>>>>>>>>> userdb > >>>>>>>>>>> {} block and this second userdb block has this same lines like > >>>>>> default > >>>>>>>>>>> settings in passdb block. > >>>>>>>>>>> > >>>>>>>>>> Try > >>>>>>>>>> > >>>>>>>>>> auth_debug=yes > >>>>>>>>>> auth_verbose=yes > >>>>>>>>>> > >>>>>>>>>> and see if it gives any more reasonable messages. > >>>>>>>>>> > >>>>>>>>>> Aki > >>>>>>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From serwis at poliman.pl Wed Feb 1 08:25:39 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 09:25:39 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: I can check each logs, I have root privileges. 2017-02-01 9:04 GMT+01:00 Aki Tuomi : > Can you check your logs? > > Aki > > > On 01.02.2017 10:02, Poliman - Serwis wrote: > > When I used backup copy of the dovecot.conf file I have this same error. > So > > I think that maybe something was written to database? I really would > point > > out that I only added > > passdb { > > driver = passwd-file > > args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > > } > > > > and comment out from above block default lines > > #args = /etc/dovecot/dovecot-sql.conf > > #driver = sql > > > > And in auth_mechanisms add line cram-md5. Nothing more in any other file. > > > > I don't want to use cram-md5. I need move back to default settings. > > Cram-md5 was only for testing purposes. :) But I supposed that I can move > > back to default by commenting out added lines. But unfortunately it isn't > > that simple. > > > > 2017-02-01 8:59 GMT+01:00 Aki Tuomi : > > > >> Are you still trying to authenticate using cram-md5? > >> > >> Aki > >> > >> > >> On 01.02.2017 09:51, Poliman - Serwis wrote: > >>> It still use: > >>> passdb { > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> } > >>> > >>> When I delete above and delete "cram-md5" in auth_mechanisms it still > not > >>> working. > >>> > >>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > >>> > >>>> You are probably wanting to do > >>>> passdb { > >>>> driver = passwd-file > >>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>> } > >>>> > >>>> passdb { > >>>> driver = sql > >>>> args = /etc/dovecot/dovecot-sql.conf > >>>> } > >>>> > >>>> Why you want to use cram-md5 is beyond me, because using SSL is much > >>>> more safer. > >>>> > >>>> Aki > >>>> > >>>> On 01.02.2017 09:41, Poliman - Serwis wrote: > >>>>> Default it was: "auth_mechanisms = plain login" and I added > cram-md5. > >>>>> After restart all work perfectly. But after I added: > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> I can't set default lines because I got error. Please tell me which > >> lines > >>>>> should be changed to resolve this issue. Should I remove "login" from > >>>>> auth_mechanism ("login" was default setting and I would like to move > >> back > >>>>> to default settings)? > >>>>> > >>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> Because cram-md5 needs the user's password for calculating > responses, > >> it > >>>>>> cannot work with hashed passwords (one-way encrypted). The only > >>>>>> supported password schemes are PLAIN and CRAM-MD5. > >>>>>> > >>>>>> Aki > >>>>>> > >>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>>>>>> I always restart dovecot after change config. ;) Sure, I commented > >> out > >>>>>>> added two lines by me, restarted dovecot and here it is: > >>>>>>> > >>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf > >>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>>>>>> auth_mechanisms = plain login cram-md5 > >>>>>>> listen = *,[::] > >>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>> mail_max_userip_connections = 100 > >>>>>>> mail_plugins = " quota" > >>>>>>> mail_privileged_group = vmail > >>>>>>> passdb { > >>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>> driver = sql > >>>>>>> } > >>>>>>> plugin { > >>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>>>>>> sieve = /var/vmail/%d/%n/.sieve > >>>>>>> sieve_max_redirects = 25 > >>>>>>> } > >>>>>>> postmaster_address = postmaster at example.com > >>>>>>> protocols = imap pop3 > >>>>>>> service auth { > >>>>>>> unix_listener /var/spool/postfix/private/auth { > >>>>>>> group = postfix > >>>>>>> mode = 0660 > >>>>>>> user = postfix > >>>>>>> } > >>>>>>> unix_listener auth-userdb { > >>>>>>> group = vmail > >>>>>>> mode = 0600 > >>>>>>> user = vmail > >>>>>>> } > >>>>>>> user = root > >>>>>>> } > >>>>>>> service imap-login { > >>>>>>> client_limit = 1000 > >>>>>>> process_limit = 512 > >>>>>>> } > >>>>>>> service lmtp { > >>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>>>>>> group = postfix > >>>>>>> mode = 0600 > >>>>>>> user = postfix > >>>>>>> } > >>>>>>> } > >>>>>>> ssl = required > >>>>>>> ssl_cert = >>>>>>> ssl_cipher_list = > >>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>> ssl_key = >>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>> userdb { > >>>>>>> driver = prefetch > >>>>>>> } > >>>>>>> userdb { > >>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>> driver = sql > >>>>>>> } > >>>>>>> protocol imap { > >>>>>>> mail_plugins = quota imap_quota > >>>>>>> } > >>>>>>> protocol pop3 { > >>>>>>> mail_plugins = quota > >>>>>>> pop3_uidl_format = %08Xu%08Xv > >>>>>>> } > >>>>>>> protocol lda { > >>>>>>> mail_plugins = sieve quota > >>>>>>> postmaster_address = webmaster at localhost > >>>>>>> } > >>>>>>> protocol lmtp { > >>>>>>> mail_plugins = quota sieve > >>>>>>> postmaster_address = webmaster at localhost > >>>>>>> } > >>>>>>> > >>>>>>> > >>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>>>>>> > >>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>>>>>> This is debug log files in syslog: > >>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > >>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>>>>>> m5ldD4= > >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > >>>> CONT > >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: > sql( > >>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > >>>> user, > >>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>> maildir, > >>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>> userdb_mail, > >>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > >>>> 'B') > >>>>>> AS > >>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > FROM > >>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > server_id = > >>>> '1' > >>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( > >>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 > >> scheme, > >>>>>>>> but we > >>>>>>>>> have only CRYPT > >>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > >>>> authentication > >>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT > >> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >>>>>> dD4= > >>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; > do > >>>> echo > >>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do > >> echo > >>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > >>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>>>>>> m5ldD4= > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>> CONT > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: > sql( > >>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as > >>>> user, > >>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>> maildir, > >>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>> userdb_mail, > >>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, > >>>> 'B') > >>>>>> AS > >>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > FROM > >>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' > >>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > server_id = > >>>> '1' > >>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( > >>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 > >> scheme, > >>>>>> but > >>>>>>>> we > >>>>>>>>> have only CRYPT > >>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb > out: > >>>>>>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ##################### > >>>>>>>>> I added in dovecot.conf lines in passdb block: > >>>>>>>>> driver = passwd-file > >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>> and commented out default lines > >>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> #driver = sql > >>>>>>>>> When I try set again default lines I got above error > >>>>>>>> Can you run doveconf -n with the configuration that causes the > above > >>>>>>>> error? Also it clearly does SQL lookup, so that error is happening > >>>> with > >>>>>>>> SQL passdb. You need to remember to restart dovecot between > >>>>>>>> configuration changes. > >>>>>>>> > >>>>>>>> Aki > >>>>>>>> > >>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>>>>>> > >>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>>>>>> I set up cram-md5 using this tutorial > >>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >>>>>> /etc/dovecot/dovecot.conf > >>>>>>>> in > >>>>>>>>>>> passdb code block: > >>>>>>>>>>> listen = *,[::] > >>>>>>>>>>> protocols = imap pop3 > >>>>>>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>>>>>> #dodana nizej linia > >>>>>>>>>>> ssl = required > >>>>>>>>>>> disable_plaintext_auth = yes > >>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>>>> mail_privileged_group = vmail > >>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>>>>>> ssl_cert = >>>>>>>>>>> ssl_key = >>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>>>> ssl_cipher_list = > >>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ > image: > >>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>>>> passdb { > >>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> # driver = sql > >>>>>>>>>>> driver = passwd-file > >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>> } > >>>>>>>>>>> userdb { > >>>>>>>>>>> driver = prefetch > >>>>>>>>>>> } > >>>>>>>>>>> userdb { > >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> driver = sql > >>>>>>>>>>> } > >>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and > come > >>>>>>>> nicely. > >>>>>>>>>>> But after I want to do default settings by commented out these > >> two > >>>>>>>> lines: > >>>>>>>>>>> driver = passwd-file > >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>> and uncomment > >>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> # driver = sql > >>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging on > >>>>>> server > >>>>>>>>>>> mail.example.com not work out". Error in logs: > >>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different > >>>>>>>>>>> passdbs/userdbs than auth server. > >>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>>>>>> > >>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file was > >>>>>> written > >>>>>>>> to > >>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't > change > >>>> any > >>>>>>>>>> userdb > >>>>>>>>>>> {} block and this second userdb block has this same lines like > >>>>>> default > >>>>>>>>>>> settings in passdb block. > >>>>>>>>>>> > >>>>>>>>>> Try > >>>>>>>>>> > >>>>>>>>>> auth_debug=yes > >>>>>>>>>> auth_verbose=yes > >>>>>>>>>> > >>>>>>>>>> and see if it gives any more reasonable messages. > >>>>>>>>>> > >>>>>>>>>> Aki > >>>>>>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From aki.tuomi at dovecot.fi Wed Feb 1 08:40:38 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 1 Feb 2017 10:40:38 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> Message-ID: <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> doveadm log errors can be helpful too On 01.02.2017 10:25, Poliman - Serwis wrote: > I can check each logs, I have root privileges. > > 2017-02-01 9:04 GMT+01:00 Aki Tuomi : > >> Can you check your logs? >> >> Aki >> >> >> On 01.02.2017 10:02, Poliman - Serwis wrote: >>> When I used backup copy of the dovecot.conf file I have this same error. >> So >>> I think that maybe something was written to database? I really would >> point >>> out that I only added >>> passdb { >>> driver = passwd-file >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> } >>> >>> and comment out from above block default lines >>> #args = /etc/dovecot/dovecot-sql.conf >>> #driver = sql >>> >>> And in auth_mechanisms add line cram-md5. Nothing more in any other file. >>> >>> I don't want to use cram-md5. I need move back to default settings. >>> Cram-md5 was only for testing purposes. :) But I supposed that I can move >>> back to default by commenting out added lines. But unfortunately it isn't >>> that simple. >>> >>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : >>> >>>> Are you still trying to authenticate using cram-md5? >>>> >>>> Aki >>>> >>>> >>>> On 01.02.2017 09:51, Poliman - Serwis wrote: >>>>> It still use: >>>>> passdb { >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> } >>>>> >>>>> When I delete above and delete "cram-md5" in auth_mechanisms it still >> not >>>>> working. >>>>> >>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : >>>>> >>>>>> You are probably wanting to do >>>>>> passdb { >>>>>> driver = passwd-file >>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>> } >>>>>> >>>>>> passdb { >>>>>> driver = sql >>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>> } >>>>>> >>>>>> Why you want to use cram-md5 is beyond me, because using SSL is much >>>>>> more safer. >>>>>> >>>>>> Aki >>>>>> >>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: >>>>>>> Default it was: "auth_mechanisms = plain login" and I added >> cram-md5. >>>>>>> After restart all work perfectly. But after I added: >>>>>>> driver = passwd-file >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>> I can't set default lines because I got error. Please tell me which >>>> lines >>>>>>> should be changed to resolve this issue. Should I remove "login" from >>>>>>> auth_mechanism ("login" was default setting and I would like to move >>>> back >>>>>>> to default settings)? >>>>>>> >>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >>>>>>> >>>>>>>> Because cram-md5 needs the user's password for calculating >> responses, >>>> it >>>>>>>> cannot work with hashed passwords (one-way encrypted). The only >>>>>>>> supported password schemes are PLAIN and CRAM-MD5. >>>>>>>> >>>>>>>> Aki >>>>>>>> >>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >>>>>>>>> I always restart dovecot after change config. ;) Sure, I commented >>>> out >>>>>>>>> added two lines by me, restarted dovecot and here it is: >>>>>>>>> >>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>>>>>>>> auth_mechanisms = plain login cram-md5 >>>>>>>>> listen = *,[::] >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>> mail_plugins = " quota" >>>>>>>>> mail_privileged_group = vmail >>>>>>>>> passdb { >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>> driver = sql >>>>>>>>> } >>>>>>>>> plugin { >>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>>>>>>> sieve = /var/vmail/%d/%n/.sieve >>>>>>>>> sieve_max_redirects = 25 >>>>>>>>> } >>>>>>>>> postmaster_address = postmaster at example.com >>>>>>>>> protocols = imap pop3 >>>>>>>>> service auth { >>>>>>>>> unix_listener /var/spool/postfix/private/auth { >>>>>>>>> group = postfix >>>>>>>>> mode = 0660 >>>>>>>>> user = postfix >>>>>>>>> } >>>>>>>>> unix_listener auth-userdb { >>>>>>>>> group = vmail >>>>>>>>> mode = 0600 >>>>>>>>> user = vmail >>>>>>>>> } >>>>>>>>> user = root >>>>>>>>> } >>>>>>>>> service imap-login { >>>>>>>>> client_limit = 1000 >>>>>>>>> process_limit = 512 >>>>>>>>> } >>>>>>>>> service lmtp { >>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>>>>>>> group = postfix >>>>>>>>> mode = 0600 >>>>>>>>> user = postfix >>>>>>>>> } >>>>>>>>> } >>>>>>>>> ssl = required >>>>>>>>> ssl_cert = >>>>>>>> ssl_cipher_list = >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>> ssl_key = >>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>> userdb { >>>>>>>>> driver = prefetch >>>>>>>>> } >>>>>>>>> userdb { >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>> driver = sql >>>>>>>>> } >>>>>>>>> protocol imap { >>>>>>>>> mail_plugins = quota imap_quota >>>>>>>>> } >>>>>>>>> protocol pop3 { >>>>>>>>> mail_plugins = quota >>>>>>>>> pop3_uidl_format = %08Xu%08Xv >>>>>>>>> } >>>>>>>>> protocol lda { >>>>>>>>> mail_plugins = sieve quota >>>>>>>>> postmaster_address = webmaster at localhost >>>>>>>>> } >>>>>>>>> protocol lmtp { >>>>>>>>> mail_plugins = quota sieve >>>>>>>>> postmaster_address = webmaster at localhost >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>>>>>>>> >>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>>>>>>>>>> This is debug log files in syslog: >>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb >> out: >>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>>>>>>>>> m5ldD4= >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >>>>>> CONT >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: >> sql( >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as >>>>>> user, >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>>>> maildir, >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>>>> userdb_mail, >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, >>>>>> 'B') >>>>>>>> AS >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >> FROM >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >> server_id = >>>>>> '1' >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): password( >>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 >>>> scheme, >>>>>>>>>> but we >>>>>>>>>>> have only CRYPT >>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb >> out: >>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com >>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >>>>>> authentication >>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT >>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >>>>>>>> dD4= >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; >> do >>>>>> echo >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do >>>> echo >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb >> out: >>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>>>>>>>>> m5ldD4= >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>> CONT >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: >> sql( >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email as >>>>>> user, >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>>>> maildir, >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>>>> userdb_mail, >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', quota, >>>>>> 'B') >>>>>>>> AS >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >> FROM >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email = ' >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >> server_id = >>>>>> '1' >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): password( >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 >>>> scheme, >>>>>>>> but >>>>>>>>>> we >>>>>>>>>>> have only CRYPT >>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb >> out: >>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ##################### >>>>>>>>>>> I added in dovecot.conf lines in passdb block: >>>>>>>>>>> driver = passwd-file >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>> and commented out default lines >>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> #driver = sql >>>>>>>>>>> When I try set again default lines I got above error >>>>>>>>>> Can you run doveconf -n with the configuration that causes the >> above >>>>>>>>>> error? Also it clearly does SQL lookup, so that error is happening >>>>>> with >>>>>>>>>> SQL passdb. You need to remember to restart dovecot between >>>>>>>>>> configuration changes. >>>>>>>>>> >>>>>>>>>> Aki >>>>>>>>>> >>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>>>>>>>>>> >>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>>>>>>>>>> I set up cram-md5 using this tutorial >>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >>>>>>>> /etc/dovecot/dovecot.conf >>>>>>>>>> in >>>>>>>>>>>>> passdb code block: >>>>>>>>>>>>> listen = *,[::] >>>>>>>>>>>>> protocols = imap pop3 >>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 >>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login >>>>>>>>>>>>> #dodana nizej linia >>>>>>>>>>>>> ssl = required >>>>>>>>>>>>> disable_plaintext_auth = yes >>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>>>>>> mail_privileged_group = vmail >>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>>>>>>>>>> ssl_cert = >>>>>>>>>>>> ssl_key = >>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>>>>>> ssl_cipher_list = >>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ >> image: >>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>>>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>>>>>> passdb { >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>> # driver = sql >>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>> } >>>>>>>>>>>>> userdb { >>>>>>>>>>>>> driver = prefetch >>>>>>>>>>>>> } >>>>>>>>>>>>> userdb { >>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>> driver = sql >>>>>>>>>>>>> } >>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and >> come >>>>>>>>>> nicely. >>>>>>>>>>>>> But after I want to do default settings by commented out these >>>> two >>>>>>>>>> lines: >>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>> and uncomment >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>> # driver = sql >>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging on >>>>>>>> server >>>>>>>>>>>>> mail.example.com not work out". Error in logs: >>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees different >>>>>>>>>>>>> passdbs/userdbs than auth server. >>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>>>>>>>>>> >>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file was >>>>>>>> written >>>>>>>>>> to >>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't >> change >>>>>> any >>>>>>>>>>>> userdb >>>>>>>>>>>>> {} block and this second userdb block has this same lines like >>>>>>>> default >>>>>>>>>>>>> settings in passdb block. >>>>>>>>>>>>> >>>>>>>>>>>> Try >>>>>>>>>>>> >>>>>>>>>>>> auth_debug=yes >>>>>>>>>>>> auth_verbose=yes >>>>>>>>>>>> >>>>>>>>>>>> and see if it gives any more reasonable messages. >>>>>>>>>>>> >>>>>>>>>>>> Aki >>>>>>>>>>>> >>> > > From kepa at oceanvoyages.com Thu Feb 2 00:28:23 2017 From: kepa at oceanvoyages.com (Kepa Lyman) Date: Wed, 1 Feb 2017 16:28:23 -0800 Subject: Restoring a deleted imap folder from backup to user Maildir Message-ID: <40b020fd-ec51-4ca1-f65e-733902b2105f@oceanvoyages.com> Hi, A user accidentally delete a folder from their Maildir (probably while using Thunderbird). The folder was at Archives/Sent/2015 (/.Archives.Sent.2015 in the Maildir format). I attempted to copy the folder from a tar backup into the Maildir folder but the new folder does not show up in Thunderbird or in the Subscribe option. I also copied the .Archives.Sent. folder as well. A "Sent" folder shows up after this in Thunderbird's Subscribe menu but not the 2015 subfolder. Is there some other additional step(s) to undertake when moving Maildir folders around so they show up in Dovecot (or Thunderbird- though I did try deleting Thunderbird's local mail tmp folder). Re: this message about SELinux on CentOS, SELinux is set to permissive: https://dovecot.org/list/dovecot/2009-January/036257.html Thanks! Kepa From bunkertor at tiscali.it Thu Feb 2 01:48:25 2017 From: bunkertor at tiscali.it (dovecot) Date: Thu, 2 Feb 2017 01:48:25 +0000 Subject: =?utf-8?B?d2hhdCBkbyB5b3UgdGhpbmsgYWJvdXQgdGhhdD8=?= Message-ID: <1122193668.20170202044825@tiscali.it> Dear! I just wanted to show you my last article and to ask your opinion about it, please read it here http://gianluigi.bigshowonline.net/0607 dovecot From skdovecot at smail.inf.fh-brs.de Thu Feb 2 07:31:05 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 2 Feb 2017 08:31:05 +0100 (CET) Subject: Restoring a deleted imap folder from backup to user Maildir In-Reply-To: <40b020fd-ec51-4ca1-f65e-733902b2105f@oceanvoyages.com> References: <40b020fd-ec51-4ca1-f65e-733902b2105f@oceanvoyages.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 1 Feb 2017, Kepa Lyman wrote: > I attempted to copy the folder from a tar backup into the Maildir folder but > the new folder does not show up in Thunderbird or in the Subscribe option. Check out: doveadm mailbox list -u <> If the folder does not show up, check out the server logs, access permissions, ... If it's there, force Thunderbird to reload the folder structure by clicking on the triangle left of the folders or top most mailbox name. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWJLgOXz1H7kL/d9rAQLzQAgAuVeONRzd7qkX5SJqLzJ/j8VvBbMDWh0s ZwRUY23oVSUVxgFMsmL257bVRq1PG+7gGZ/7a1cTxoAT2yVP+k3NXZortr2r7d8a yLuMY8Up8fDaLuTkggq+zVMpoTTXMxT9W5N46iH/bGxh8wb2eH8qQ2BEE+eEF1tb /XktQWPUbOeARAZCijIujXxJTr3lUsbOxoa2F8Qm/gKypZnvSMKtbe2FG0ysMQO1 Gny9EB0qtVqef/8MH4dy7dZ82EftmuZX65FP0rYe5YcZglFhDQiMlvVbZmuRKj/g DPR+UoglTYF/zz3vfyef3BAOsO4yXdAvsXYBq3eRGwy42jB/daDy+w== =aotv -----END PGP SIGNATURE----- From famzah at icdsoft.com Thu Feb 2 10:49:46 2017 From: famzah at icdsoft.com (Ivan Zahariev) Date: Thu, 2 Feb 2017 12:49:46 +0200 Subject: Doveadm option for a non-wildcard single-user with userdb Message-ID: Hello, Most "doveadm" commands accept "[-A|-u user|-F file]" for user selection, or the environment "USER". I'm testing with "doveadm quota recalc". The problem is that there is no way to ask "doveadm" to work in (1) single-user mode with (2) no wildcard support, and at the same time to (3) make a lookup in "userdb", in order to get the user's specific configuration. We have mailboxes which contain "?" and "*" symbols, and we can't work with them using "doveadm" now. * If we use "-A", this works with all users. Not our case at all. * If we use "-F" and provide just one user in the file, this works for a single user + lookup in "userdb", does not interpret wildcard, but "doveadm" works in a "users list" mode and the output is different. What's more problematic is that errors for an mbox do not end up in "doveadm" exiting with a non-zero exit code. * If we use "-u", this works for a single user + lookup in "userdb", but interprets wildcards. Does not work for mailboxes which contain "?" and "*". * If we use the USER environment, this works for a single users and does not interpret wildcards but does not do a lookup in "userdb". Should we add another user-selection argument, for example "-U", which (1) selects a single-user like "-u", does a "userdb" lookup like "-u" does, but does not interpret wildcards unlike "-u" ? Best regards. --Ivan From ruga at protonmail.com Thu Feb 2 12:53:38 2017 From: ruga at protonmail.com (Ruga) Date: Thu, 02 Feb 2017 07:53:38 -0500 Subject: SNI with mixed certs Message-ID: <1iLqCjd2c-3hBVKfnqCP_p-8ykytBgTKmrG4ipWU6_hlHQ7tsXnrotn-US3Yzm0xbYqRFkMrSvM4WI70-23LaQh4Ka0ERMf0lmViSPrUbSs=@protonmail.com> Dovecot SNI is failing hard today. Server with n domains, each with a startssl certificate of its own, all certificates expired this morning. Decision: move to Letsencrypt. Firsr certificate issued and installed. Other domains in the pipeline. Dovecot server rebooted. Expected result: one domain returning the new cert, and the n-1 domains returning the expiration notification. Actual result: the domain with LE is returning startssl expired notifications. Manual check of the key and pem files is OK... From dbetz at df.eu Thu Feb 2 13:07:38 2017 From: dbetz at df.eu (Daniel Betz) Date: Thu, 2 Feb 2017 13:07:38 +0000 Subject: Dovecot performance and proxy loops with IPv6 Message-ID: <2ff9c5333ca94814912c8fe7a481e6ed@EXDAG08-1.EXCHANGE.INT> Hello list, i run here an large mailsetup with some million mailboxes and got strange performance problems, cause i think i have overseen or forgotten an simple setting. Here are some details: 21 CentOS 7 Servers with dovecot 2.2.25 and ldap userdb/passdb via socket behind an hardware loadbalancer. The storage behind is an ISCSI Storage with 4 10Gbit/s multipath paths, splitted up to 10 TB volumes for each server with LVM and xfs filesystem. No Cluster FS Each server has about 60.000 to 75.000 mailboxes on it. mailboxes can have up to 10Gbyte space. The Log says this sometimes and complete random: Feb 1 10:42:49 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: imap-login: Error: net_connect_unix(imap) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Feb 1 10:42:50 server1 dovecot: pop3-login: Error: net_connect_unix(pop3) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable Sure i have read the SocketUnavailabe wiki page and changed some settings, but the errors are not gone. Could you please look over my dovecot config and give me some tips or hints what to change. The next this is, when adding IPv6 via DNS to the hosts and login with IPv6 i will become an proxy loop. Settings in nameserver: server1.domain.com IN A 123.123.123.123 server1.domain.com IN AAAA 2001:123::1 The host entry comes from the ldap and says: mailHost: server1.domain.com Imap Login with IPv6 to server1.domain.com tries to proxy from server1.domain.com ( IPv6 ) to server1.domain.com ( IPv6 ) and loops then. I have removed the IPv6 AAAA entries in the dns to stop this loops. Sorry, but i have no logs for this anymore. Thanks in advise, Daniel And here system configs and dovecot configs: sysctl: fs.inotify.max_user_instances = 65535 fs.inotify.max_user_watches = 16384 systemd startup with ulimit settings: [Unit] Description=Dovecot Mailservice IMAP/POP [Service] Type=simple LimitCORE=0 LimitNPROC=5000000 LimitNOFILE=65535 LimitSTACK=81920 LimitDATA=infinity LimitMEMLOCK=infinity LimitRSS=infinity LimitAS=infinity ExecStart=/usr/local/dovecot2/sbin/dovecot -F -c /usr/local/dovecot2/etc/dovecot/dovecot.conf [Install] WantedBy=multi-user.target dovecot-ldap.conf: uris = ldapi://%2Fvar%2Frun%2Fldapi dn = cn=xxxxxxx,o=domain,c=com dnpass = xxxxxxxxxxxxx auth_bind = no ldap_version = 3 base = o=domain,c=com user_attrs = mail=user,mailMessageStore=home,\ mailQuota=quota_rule=*:storage=%$ iterate_filter= (|(mailHost=server1.domain.com)(mailHost=popserver1.domain.com)) user_filter = (&(accountstatus=active)(|(uid=%u)(mail=%u))) pass_attrs = mail=user,userPassword=password,=proxy_maybe=y,mailHost=host,=destuser=%u[%r] pass_filter = (&(accountstatus=active)(|(uid=%u)(mail=%u))) dovecot.conf: # 2.2.25 (7be1766): /usr/local/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.36.3.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_cache_negative_ttl = 1 mins auth_cache_size = 64 M auth_cache_ttl = 2 hours auth_mechanisms = plain login auth_username_chars = auth_verbose = yes base_dir = /var/run/dovecot/ debug_log_path = /dev/null default_login_user = dovecot disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 first_valid_gid = 1001 first_valid_uid = 1001 info_log_path = /dev/stderr lda_mailbox_autocreate = yes lda_original_recipient_header = X-Envelope-To log_path = /dev/stderr log_timestamp = login_log_format_elements = user=[%u] method=%m rip=%r lip=%l %c mail_gid = 1001 mail_location = mdbox:~:INDEX=%h/INDEX mail_plugins = "notify replication stats" mail_uid = 1001 mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_warning = storage=85%% quota-warning 85 %u stats_refresh = 30 secs stats_track_cmds = yes } replication_max_conns = 30 sendmail_path = /usr/local/exim/bin/exim service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = popuser } unix_listener replication-notify { mode = 0666 user = popuser } } service anvil { client_limit = 60000 } service auth { client_limit = 60000 unix_listener auth-userdb { mode = 0666 user = popuser } unix_listener auth { mode = 0666 user = popuser } } service config { unix_listener config { user = popuser } } service dict { unix_listener dict { mode = 0666 user = popuser } } service doveadm { inet_listener { port = 12345 } user = popuser } service imap-login { chroot = login process_limit = 60000 process_min_avail = 16 } service imap { executable = /usr/local/dovecot2/libexec/dovecot/imap process_limit = 250000 } service ipc { client_limit = 60000 unix_listener ipc { mode = 0650 user = dovecot } unix_listener login/ipc-proxy { mode = 0650 user = dovecot } } service lmtp { unix_listener lmtp { mode = 0666 user = popuser } } service pop3-login { chroot = login process_limit = 60000 process_min_avail = 16 } service pop3 { executable = /usr/local/dovecot2/libexec/dovecot/pop3 process_limit = 250000 } service replicator { unix_listener replicator-doveadm { mode = 0600 user = popuser } } service stats { fifo_listener stats-mail { mode = 0600 user = popuser } } ssl_cert = References: <58870F64.2030906@asom-net.dk> <5890AF03.30308@asom-net.dk> <26E5C771-BFF5-4982-BB79-69998F19FC80@valo.at> Message-ID: <589332E4.6050109@asom-net.dk> Hi Christian, On 2017-01-31 23:20, Christian Kivalo wrote: >> dovecot -n: >> # 2.2.13: /etc/dovecot/dovecot.conf >> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 ext4 >> auth_default_realm = vejen-net.dk >> auth_mechanisms = plain login >> auth_verbose = yes >> disable_plaintext_auth = no >> first_valid_uid = 110 >> log_timestamp = "%Y-%m-%d %H:%M:%S " >> mail_debug = yes >> mail_location = maildir:/data/vmail/%d/%n/ > The one line i'm missing here from your doveconf -n output is mail_plugins = " quota" set in conf.d/10-mail.conf > > Have you added quota to the global mail plugins setting? http://wiki2.dovecot.org/Quota That did it, now it seems to work! I thought the global mail_plugins was only a variable not a config option. But it seems it must be set. So this works: mail_plugins = $mail_plugins quota protocol imap { mail_plugins = $mail_plugins imap_quota } protocol pop3 { mail_plugins = $mail_plugins } But this does not: protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota } Thank you very much for helping me along. Also thanks to Aki Tuomi who pointed out the same issue. Regards, -- Kristian Pedersen ASOM-Net Systemadministrator www.asom-net.dk Telefon: 44 400 970 From kremels at kreme.com Thu Feb 2 16:28:17 2017 From: kremels at kreme.com (@lbutlr) Date: Thu, 2 Feb 2017 09:28:17 -0700 Subject: Moving to new password scheme In-Reply-To: References: <275F8426-C222-4D56-9650-58DF87028FDA@kreme.com> <45232a6d-9766-bf07-3c42-8383c1f9b174@skye.it> <6717F2B9-8297-49A0-B178-684F9FC37F68@kreme.com> Message-ID: <866C9E1C-0191-4314-A267-430822AE35E8@kreme.com> On Jan 25, 2017, at 4:57 AM, Steffen Kaiser wrote: > yes, userdb's are checked in the same order as they appear in the config file(s). Thanks for all the help, got everyone migrated over to SHA256-CRYPT now. -- Apple broke AppleScripting signatures in Mail.app, so no random signatures. From mantas.geguzis at ittc.vu.lt Thu Feb 2 17:38:40 2017 From: mantas.geguzis at ittc.vu.lt (Mantas =?utf-8?b?R2VndcW+aXM=?=) Date: Thu, 02 Feb 2017 19:38:40 +0200 Subject: Compiling Dovecot on Solaris 10 Message-ID: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> Hello, I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: test-ioloop.c: In function `test_ioloop_pending_io': test-ioloop.c:188: error: size of array `type name' is negative My configuration is like this: Install prefix . : /usr/local File offsets ... : 64bit I/O polling .... : poll I/O notifys .... : none SSL ............ : yes (OpenSSL) GSSAPI ......... : no passdbs ........ : static passwd passwd-file shadow pam checkpassword dcrypt ..........: yes : -bsdauth -sia -ldap -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword : -ldap -sql -vpopmail -nss SQL drivers .... : : -pgsql -mysql -sqlite -cassandra Full text search : squat : -lucene -solr Last version that I have compiled was 2.2.24, version 2.2.25 failed with error: In file included from guid.c:6: sha1.h:80: error: static or type qualifiers in abstract declarator Is there anyone who can help me? -- Pagarbiai Mantas Gegu?is VU Informacini? technologij? taikymo centras tel. 8 5 236 6208 From matt at matthoran.com Fri Feb 3 04:29:48 2017 From: matt at matthoran.com (Matt Horan) Date: Thu, 2 Feb 2017 23:29:48 -0500 Subject: Dovecot, Postfix, and SASL AUTH EXTERNAL Message-ID: <20170203042948.GA28889@friction.matthoran.com> Hey folks, I've been using the ever popular Dovecot and Postfix combo for years. A while back I also introduced mutual TLS for mail clients to Dovecot and Postfix. I achieved this by a custom checkpassword script and SASL AUTH EXTERNAL for IMAP. This all worked great with clients like Thunderbird, which can be configured to use mutual TLS and SASL EXTERNAL for IMAP, and mutual TLS with no additional authentication for SMTP. However, I found that other mail clients, in particular K-9 mail on Android, [1] are not compatible with this configuration. I've been patching K-9 mail to work around this issue for some time now. If I configure K-9 to behave like Thunderbird when sending messages via SMTP, all is well. However, there's been some activity on an issue [2] which suggests some changes may be upcoming which will be incompatible with my patch. Without my patch, K-9 tries to auth with Postfix via AUTH EXTERNAL after presenting its client certificate. Despite configuring Postfix to prefer certificates before SASL, Postfix forwards the authentication request to Dovecot, which rejects it without even trying my checkpassword script. With my patch, K-9 simply initiates an SMTP connection without any additional authentication when mutual TLS is used. This behavior is similar to Thunderbird. The K-9 maintainers do not seem interested in merging this behavior into mainline. I can't seem to get Postfix to ignore the SASL failures in the case of successful mutual TLS. I want to use SASL authentication as a fallback from untrusted clients, where I use a combination of password and one time code. Even if Dovecot did not reject the AUTH EXTERNAL request from Postfix, I'm not sure how it could determine whether a valid client certificate were presented to Postfix, unless some additional information were passed along in the SASL request. I'd love to hear any thoughts from the community on how to move forward here. Should I pressure the K-9 maintainers to behave more like other clients? Would it make sense to extend the SASL interface in some way such that Dovecot could handle an EXTERNAL request from Postfix? Or should Postfix simply ignore SASL EXTERNAL based on the configured authentication mechanism order? Thanks, Matt [1] https://github.com/k9mail/k-9/ [2] https://github.com/k9mail/k-9/issues/793 -- Matt Horan matt at matthoran.com http://matthoran.com/ From serwis at poliman.pl Fri Feb 3 06:14:55 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Fri, 3 Feb 2017 07:14:55 +0100 Subject: postfix/smtpd[725]: fatal: no SASL authentication mechanisms Message-ID: I haven't doveadm logs in /var/log/. Are they default in another place or maybe should I turn on something? My config (default passdb block and auth_mechanisms, nothing more changed): root at vps342401:/etc/dovecot# doveconf -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS auth_mechanisms = plain login listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster at vps342401.ovh.net protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = &1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; do ne) Feb 1 09:53:01 vps342401 CRON[778]: (root) CMD (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done ) Is there any strange thing in these config lines? -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From lenaigst at maelenn.org Fri Feb 3 07:51:11 2017 From: lenaigst at maelenn.org (Thierry) Date: Fri, 3 Feb 2017 09:51:11 +0200 Subject: Dovecot dsync 'ssl_client_ca' Message-ID: <1547742423.20170203095111@maelenn.org> Hello, Still working with my dsync pb. I have done a clone (vmware) of my email server. Today I have two strictly identical emails servers (server1 (main) and server2 (bck) (except IP, hostname and mail_replica). The ssl config on my both server: ssl_protocols = !SSLv2 !SSLv3 ssl = required verbose_ssl = no ssl_key = Ok, got it. change imap-login and pop-login to these like showed in dovocot wiki for high-performance login mode. service imap-login { chroot = login service_count = 0 client_limit = 600 process_limit = 100 process_min_avail = 16 } service pop3-login { chroot = login service_count = 0 client_limit = 600 process_limit = 100 process_min_avail = 16 } From serwis at poliman.pl Wed Feb 1 08:55:20 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 09:55:20 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> Message-ID: I haven't doveadm logs in /var/log/. Are they default in another place or maybe should I turn on something? My config (default passdb block and auth_mechanisms, nothing more changed): root at vps342401:/etc/dovecot# doveconf -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS auth_mechanisms = plain login listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster at vps342401.ovh.net protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = &1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; do ne) Feb 1 09:53:01 vps342401 CRON[778]: (root) CMD (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done ) 2017-02-01 9:40 GMT+01:00 Aki Tuomi : > doveadm log errors can be helpful too > > > On 01.02.2017 10:25, Poliman - Serwis wrote: > > I can check each logs, I have root privileges. > > > > 2017-02-01 9:04 GMT+01:00 Aki Tuomi : > > > >> Can you check your logs? > >> > >> Aki > >> > >> > >> On 01.02.2017 10:02, Poliman - Serwis wrote: > >>> When I used backup copy of the dovecot.conf file I have this same > error. > >> So > >>> I think that maybe something was written to database? I really would > >> point > >>> out that I only added > >>> passdb { > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> } > >>> > >>> and comment out from above block default lines > >>> #args = /etc/dovecot/dovecot-sql.conf > >>> #driver = sql > >>> > >>> And in auth_mechanisms add line cram-md5. Nothing more in any other > file. > >>> > >>> I don't want to use cram-md5. I need move back to default settings. > >>> Cram-md5 was only for testing purposes. :) But I supposed that I can > move > >>> back to default by commenting out added lines. But unfortunately it > isn't > >>> that simple. > >>> > >>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : > >>> > >>>> Are you still trying to authenticate using cram-md5? > >>>> > >>>> Aki > >>>> > >>>> > >>>> On 01.02.2017 09:51, Poliman - Serwis wrote: > >>>>> It still use: > >>>>> passdb { > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> } > >>>>> > >>>>> When I delete above and delete "cram-md5" in auth_mechanisms it still > >> not > >>>>> working. > >>>>> > >>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> You are probably wanting to do > >>>>>> passdb { > >>>>>> driver = passwd-file > >>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>> } > >>>>>> > >>>>>> passdb { > >>>>>> driver = sql > >>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>> } > >>>>>> > >>>>>> Why you want to use cram-md5 is beyond me, because using SSL is much > >>>>>> more safer. > >>>>>> > >>>>>> Aki > >>>>>> > >>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: > >>>>>>> Default it was: "auth_mechanisms = plain login" and I added > >> cram-md5. > >>>>>>> After restart all work perfectly. But after I added: > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> I can't set default lines because I got error. Please tell me which > >>>> lines > >>>>>>> should be changed to resolve this issue. Should I remove "login" > from > >>>>>>> auth_mechanism ("login" was default setting and I would like to > move > >>>> back > >>>>>>> to default settings)? > >>>>>>> > >>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > >>>>>>> > >>>>>>>> Because cram-md5 needs the user's password for calculating > >> responses, > >>>> it > >>>>>>>> cannot work with hashed passwords (one-way encrypted). The only > >>>>>>>> supported password schemes are PLAIN and CRAM-MD5. > >>>>>>>> > >>>>>>>> Aki > >>>>>>>> > >>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>>>>>>>> I always restart dovecot after change config. ;) Sure, I > commented > >>>> out > >>>>>>>>> added two lines by me, restarted dovecot and here it is: > >>>>>>>>> > >>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf > >>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>>>>>>>> auth_mechanisms = plain login cram-md5 > >>>>>>>>> listen = *,[::] > >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>> mail_plugins = " quota" > >>>>>>>>> mail_privileged_group = vmail > >>>>>>>>> passdb { > >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> driver = sql > >>>>>>>>> } > >>>>>>>>> plugin { > >>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>>>>>>>> sieve = /var/vmail/%d/%n/.sieve > >>>>>>>>> sieve_max_redirects = 25 > >>>>>>>>> } > >>>>>>>>> postmaster_address = postmaster at example.com > >>>>>>>>> protocols = imap pop3 > >>>>>>>>> service auth { > >>>>>>>>> unix_listener /var/spool/postfix/private/auth { > >>>>>>>>> group = postfix > >>>>>>>>> mode = 0660 > >>>>>>>>> user = postfix > >>>>>>>>> } > >>>>>>>>> unix_listener auth-userdb { > >>>>>>>>> group = vmail > >>>>>>>>> mode = 0600 > >>>>>>>>> user = vmail > >>>>>>>>> } > >>>>>>>>> user = root > >>>>>>>>> } > >>>>>>>>> service imap-login { > >>>>>>>>> client_limit = 1000 > >>>>>>>>> process_limit = 512 > >>>>>>>>> } > >>>>>>>>> service lmtp { > >>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>>>>>>>> group = postfix > >>>>>>>>> mode = 0600 > >>>>>>>>> user = postfix > >>>>>>>>> } > >>>>>>>>> } > >>>>>>>>> ssl = required > >>>>>>>>> ssl_cert = >>>>>>>>> ssl_cipher_list = > >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>> ssl_key = >>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>> userdb { > >>>>>>>>> driver = prefetch > >>>>>>>>> } > >>>>>>>>> userdb { > >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> driver = sql > >>>>>>>>> } > >>>>>>>>> protocol imap { > >>>>>>>>> mail_plugins = quota imap_quota > >>>>>>>>> } > >>>>>>>>> protocol pop3 { > >>>>>>>>> mail_plugins = quota > >>>>>>>>> pop3_uidl_format = %08Xu%08Xv > >>>>>>>>> } > >>>>>>>>> protocol lda { > >>>>>>>>> mail_plugins = sieve quota > >>>>>>>>> postmaster_address = webmaster at localhost > >>>>>>>>> } > >>>>>>>>> protocol lmtp { > >>>>>>>>> mail_plugins = quota sieve > >>>>>>>>> postmaster_address = webmaster at localhost > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>>>>>>>> > >>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>>>>>>>> This is debug log files in syslog: > >>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > >>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>>>>>>>> m5ldD4= > >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > >>>>>> CONT > >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: > >> sql( > >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email > as > >>>>>> user, > >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>>>> maildir, > >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>>>> userdb_mail, > >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', > quota, > >>>>>> 'B') > >>>>>>>> AS > >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > >> FROM > >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email > = ' > >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > >> server_id = > >>>>>> '1' > >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): > password( > >>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 > >>>> scheme, > >>>>>>>>>> but we > >>>>>>>>>>> have only CRYPT > >>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > >>>>>> authentication > >>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT > >>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >>>>>>>> dD4= > >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; > >> do > >>>>>> echo > >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; > do > >>>> echo > >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > >>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>>>>>>>> m5ldD4= > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>> CONT > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: > >> sql( > >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email > as > >>>>>> user, > >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>>>> maildir, > >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>>>> userdb_mail, > >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', > quota, > >>>>>> 'B') > >>>>>>>> AS > >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > >> FROM > >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email > = ' > >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > >> server_id = > >>>>>> '1' > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): > password( > >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 > >>>> scheme, > >>>>>>>> but > >>>>>>>>>> we > >>>>>>>>>>> have only CRYPT > >>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ##################### > >>>>>>>>>>> I added in dovecot.conf lines in passdb block: > >>>>>>>>>>> driver = passwd-file > >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>> and commented out default lines > >>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> #driver = sql > >>>>>>>>>>> When I try set again default lines I got above error > >>>>>>>>>> Can you run doveconf -n with the configuration that causes the > >> above > >>>>>>>>>> error? Also it clearly does SQL lookup, so that error is > happening > >>>>>> with > >>>>>>>>>> SQL passdb. You need to remember to restart dovecot between > >>>>>>>>>> configuration changes. > >>>>>>>>>> > >>>>>>>>>> Aki > >>>>>>>>>> > >>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>>>>>>>> > >>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>>>>>>>> I set up cram-md5 using this tutorial > >>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >>>>>>>> /etc/dovecot/dovecot.conf > >>>>>>>>>> in > >>>>>>>>>>>>> passdb code block: > >>>>>>>>>>>>> listen = *,[::] > >>>>>>>>>>>>> protocols = imap pop3 > >>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>>>>>>>> #dodana nizej linia > >>>>>>>>>>>>> ssl = required > >>>>>>>>>>>>> disable_plaintext_auth = yes > >>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>>>>>> mail_privileged_group = vmail > >>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>>>>>>>> ssl_cert = >>>>>>>>>>>>> ssl_key = >>>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>>>>>> ssl_cipher_list = > >>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ > >> image: > >>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>>>>>> passdb { > >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>>>> # driver = sql > >>>>>>>>>>>>> driver = passwd-file > >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>>>> } > >>>>>>>>>>>>> userdb { > >>>>>>>>>>>>> driver = prefetch > >>>>>>>>>>>>> } > >>>>>>>>>>>>> userdb { > >>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>>>> driver = sql > >>>>>>>>>>>>> } > >>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and > >> come > >>>>>>>>>> nicely. > >>>>>>>>>>>>> But after I want to do default settings by commented out > these > >>>> two > >>>>>>>>>> lines: > >>>>>>>>>>>>> driver = passwd-file > >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>>>> and uncomment > >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>>>> # driver = sql > >>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging > on > >>>>>>>> server > >>>>>>>>>>>>> mail.example.com not work out". Error in logs: > >>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees > different > >>>>>>>>>>>>> passdbs/userdbs than auth server. > >>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>>>>>>>> > >>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file > was > >>>>>>>> written > >>>>>>>>>> to > >>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't > >> change > >>>>>> any > >>>>>>>>>>>> userdb > >>>>>>>>>>>>> {} block and this second userdb block has this same lines > like > >>>>>>>> default > >>>>>>>>>>>>> settings in passdb block. > >>>>>>>>>>>>> > >>>>>>>>>>>> Try > >>>>>>>>>>>> > >>>>>>>>>>>> auth_debug=yes > >>>>>>>>>>>> auth_verbose=yes > >>>>>>>>>>>> > >>>>>>>>>>>> and see if it gives any more reasonable messages. > >>>>>>>>>>>> > >>>>>>>>>>>> Aki > >>>>>>>>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From serwis at poliman.pl Wed Feb 1 11:16:08 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Wed, 1 Feb 2017 12:16:08 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> Message-ID: Is there any strange thing in these config lines? 2017-02-01 9:40 GMT+01:00 Aki Tuomi : > doveadm log errors can be helpful too > > > On 01.02.2017 10:25, Poliman - Serwis wrote: > > I can check each logs, I have root privileges. > > > > 2017-02-01 9:04 GMT+01:00 Aki Tuomi : > > > >> Can you check your logs? > >> > >> Aki > >> > >> > >> On 01.02.2017 10:02, Poliman - Serwis wrote: > >>> When I used backup copy of the dovecot.conf file I have this same > error. > >> So > >>> I think that maybe something was written to database? I really would > >> point > >>> out that I only added > >>> passdb { > >>> driver = passwd-file > >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>> } > >>> > >>> and comment out from above block default lines > >>> #args = /etc/dovecot/dovecot-sql.conf > >>> #driver = sql > >>> > >>> And in auth_mechanisms add line cram-md5. Nothing more in any other > file. > >>> > >>> I don't want to use cram-md5. I need move back to default settings. > >>> Cram-md5 was only for testing purposes. :) But I supposed that I can > move > >>> back to default by commenting out added lines. But unfortunately it > isn't > >>> that simple. > >>> > >>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : > >>> > >>>> Are you still trying to authenticate using cram-md5? > >>>> > >>>> Aki > >>>> > >>>> > >>>> On 01.02.2017 09:51, Poliman - Serwis wrote: > >>>>> It still use: > >>>>> passdb { > >>>>> driver = passwd-file > >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>> } > >>>>> > >>>>> When I delete above and delete "cram-md5" in auth_mechanisms it still > >> not > >>>>> working. > >>>>> > >>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : > >>>>> > >>>>>> You are probably wanting to do > >>>>>> passdb { > >>>>>> driver = passwd-file > >>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>> } > >>>>>> > >>>>>> passdb { > >>>>>> driver = sql > >>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>> } > >>>>>> > >>>>>> Why you want to use cram-md5 is beyond me, because using SSL is much > >>>>>> more safer. > >>>>>> > >>>>>> Aki > >>>>>> > >>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: > >>>>>>> Default it was: "auth_mechanisms = plain login" and I added > >> cram-md5. > >>>>>>> After restart all work perfectly. But after I added: > >>>>>>> driver = passwd-file > >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>> I can't set default lines because I got error. Please tell me which > >>>> lines > >>>>>>> should be changed to resolve this issue. Should I remove "login" > from > >>>>>>> auth_mechanism ("login" was default setting and I would like to > move > >>>> back > >>>>>>> to default settings)? > >>>>>>> > >>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : > >>>>>>> > >>>>>>>> Because cram-md5 needs the user's password for calculating > >> responses, > >>>> it > >>>>>>>> cannot work with hashed passwords (one-way encrypted). The only > >>>>>>>> supported password schemes are PLAIN and CRAM-MD5. > >>>>>>>> > >>>>>>>> Aki > >>>>>>>> > >>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: > >>>>>>>>> I always restart dovecot after change config. ;) Sure, I > commented > >>>> out > >>>>>>>>> added two lines by me, restarted dovecot and here it is: > >>>>>>>>> > >>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf > >>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > >>>>>>>>> auth_mechanisms = plain login cram-md5 > >>>>>>>>> listen = *,[::] > >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>> mail_plugins = " quota" > >>>>>>>>> mail_privileged_group = vmail > >>>>>>>>> passdb { > >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> driver = sql > >>>>>>>>> } > >>>>>>>>> plugin { > >>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage > >>>>>>>>> sieve = /var/vmail/%d/%n/.sieve > >>>>>>>>> sieve_max_redirects = 25 > >>>>>>>>> } > >>>>>>>>> postmaster_address = postmaster at example.com > >>>>>>>>> protocols = imap pop3 > >>>>>>>>> service auth { > >>>>>>>>> unix_listener /var/spool/postfix/private/auth { > >>>>>>>>> group = postfix > >>>>>>>>> mode = 0660 > >>>>>>>>> user = postfix > >>>>>>>>> } > >>>>>>>>> unix_listener auth-userdb { > >>>>>>>>> group = vmail > >>>>>>>>> mode = 0600 > >>>>>>>>> user = vmail > >>>>>>>>> } > >>>>>>>>> user = root > >>>>>>>>> } > >>>>>>>>> service imap-login { > >>>>>>>>> client_limit = 1000 > >>>>>>>>> process_limit = 512 > >>>>>>>>> } > >>>>>>>>> service lmtp { > >>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { > >>>>>>>>> group = postfix > >>>>>>>>> mode = 0600 > >>>>>>>>> user = postfix > >>>>>>>>> } > >>>>>>>>> } > >>>>>>>>> ssl = required > >>>>>>>>> ssl_cert = >>>>>>>>> ssl_cipher_list = > >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: > >>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ > >>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- > >>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- > >>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- > >>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- > >>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: > >>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: > >>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- > >>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! > >>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! > >>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > >>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>> ssl_key = >>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>> userdb { > >>>>>>>>> driver = prefetch > >>>>>>>>> } > >>>>>>>>> userdb { > >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>> driver = sql > >>>>>>>>> } > >>>>>>>>> protocol imap { > >>>>>>>>> mail_plugins = quota imap_quota > >>>>>>>>> } > >>>>>>>>> protocol pop3 { > >>>>>>>>> mail_plugins = quota > >>>>>>>>> pop3_uidl_format = %08Xu%08Xv > >>>>>>>>> } > >>>>>>>>> protocol lda { > >>>>>>>>> mail_plugins = sieve quota > >>>>>>>>> postmaster_address = webmaster at localhost > >>>>>>>>> } > >>>>>>>>> protocol lmtp { > >>>>>>>>> mail_plugins = quota sieve > >>>>>>>>> postmaster_address = webmaster at localhost > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : > >>>>>>>>> > >>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: > >>>>>>>>>>> This is debug log files in syslog: > >>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ > >>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL > >>>>>>>>>> m5ldD4= > >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: > >>>>>> CONT > >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: > >> sql( > >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email > as > >>>>>> user, > >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>>>> maildir, > >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>>>> userdb_mail, > >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', > quota, > >>>>>> 'B') > >>>>>>>> AS > >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > >> FROM > >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email > = ' > >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > >> server_id = > >>>>>> '1' > >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): > password( > >>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 > >>>> scheme, > >>>>>>>>>> but we > >>>>>>>>>>> have only CRYPT > >>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com > >>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: > >>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 > >>>>>> authentication > >>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT > >>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l > >>>>>>>> dD4= > >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD > >>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; > >> do > >>>>>> echo > >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD > >>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; > do > >>>> echo > >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# > >>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ > >>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL > >>>>>>>>>> m5ldD4= > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: > >>>>>> CONT > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: > >> sql( > >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email > as > >>>>>> user, > >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', > >>>>>> maildir, > >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as > >>>>>>>>>> userdb_mail, > >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', > quota, > >>>>>> 'B') > >>>>>>>> AS > >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve > >> FROM > >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email > = ' > >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND > >> server_id = > >>>>>> '1' > >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): > password( > >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 > >>>> scheme, > >>>>>>>> but > >>>>>>>>>> we > >>>>>>>>>>> have only CRYPT > >>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb > >> out: > >>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ##################### > >>>>>>>>>>> I added in dovecot.conf lines in passdb block: > >>>>>>>>>>> driver = passwd-file > >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>> and commented out default lines > >>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>> #driver = sql > >>>>>>>>>>> When I try set again default lines I got above error > >>>>>>>>>> Can you run doveconf -n with the configuration that causes the > >> above > >>>>>>>>>> error? Also it clearly does SQL lookup, so that error is > happening > >>>>>> with > >>>>>>>>>> SQL passdb. You need to remember to restart dovecot between > >>>>>>>>>> configuration changes. > >>>>>>>>>> > >>>>>>>>>> Aki > >>>>>>>>>> > >>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : > >>>>>>>>>>> > >>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: > >>>>>>>>>>>>> I set up cram-md5 using this tutorial > >>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in > >>>>>>>> /etc/dovecot/dovecot.conf > >>>>>>>>>> in > >>>>>>>>>>>>> passdb code block: > >>>>>>>>>>>>> listen = *,[::] > >>>>>>>>>>>>> protocols = imap pop3 > >>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 > >>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login > >>>>>>>>>>>>> #dodana nizej linia > >>>>>>>>>>>>> ssl = required > >>>>>>>>>>>>> disable_plaintext_auth = yes > >>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " > >>>>>>>>>>>>> mail_privileged_group = vmail > >>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net > >>>>>>>>>>>>> ssl_cert = >>>>>>>>>>>>> ssl_key = >>>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > >>>>>>>>>>>>> ssl_cipher_list = > >>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: > >>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ > >> image: > >>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ > >>>>>>>>>>>>> ssl_prefer_server_ciphers = yes > >>>>>>>>>>>>> ssl_dh_parameters_length = 2048 > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> mail_max_userip_connections = 100 > >>>>>>>>>>>>> passdb { > >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>>>> # driver = sql > >>>>>>>>>>>>> driver = passwd-file > >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>>>> } > >>>>>>>>>>>>> userdb { > >>>>>>>>>>>>> driver = prefetch > >>>>>>>>>>>>> } > >>>>>>>>>>>>> userdb { > >>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>>>> driver = sql > >>>>>>>>>>>>> } > >>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and > >> come > >>>>>>>>>> nicely. > >>>>>>>>>>>>> But after I want to do default settings by commented out > these > >>>> two > >>>>>>>>>> lines: > >>>>>>>>>>>>> driver = passwd-file > >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd > >>>>>>>>>>>>> and uncomment > >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf > >>>>>>>>>>>>> # driver = sql > >>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging > on > >>>>>>>> server > >>>>>>>>>>>>> mail.example.com not work out". Error in logs: > >>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees > different > >>>>>>>>>>>>> passdbs/userdbs than auth server. > >>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF > >>>>>>>>>>>>> > >>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file > was > >>>>>>>> written > >>>>>>>>>> to > >>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't > >> change > >>>>>> any > >>>>>>>>>>>> userdb > >>>>>>>>>>>>> {} block and this second userdb block has this same lines > like > >>>>>>>> default > >>>>>>>>>>>>> settings in passdb block. > >>>>>>>>>>>>> > >>>>>>>>>>>> Try > >>>>>>>>>>>> > >>>>>>>>>>>> auth_debug=yes > >>>>>>>>>>>> auth_verbose=yes > >>>>>>>>>>>> > >>>>>>>>>>>> and see if it gives any more reasonable messages. > >>>>>>>>>>>> > >>>>>>>>>>>> Aki > >>>>>>>>>>>> > >>> > > > > > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From serwis at poliman.pl Thu Feb 2 06:30:04 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Thu, 2 Feb 2017 07:30:04 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> Message-ID: I haven't doveadm logs in /var/log/. Are they default in another place or maybe should I turn on something? My config (default passdb block and auth_mechanisms, nothing more changed): root at vps342401:/etc/dovecot# doveconf -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS auth_mechanisms = plain login listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster at vps342401.ovh.net protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = : > Is there any strange thing in these config lines? > > 2017-02-01 9:40 GMT+01:00 Aki Tuomi : > >> doveadm log errors can be helpful too >> >> >> On 01.02.2017 10:25, Poliman - Serwis wrote: >> > I can check each logs, I have root privileges. >> > >> > 2017-02-01 9:04 GMT+01:00 Aki Tuomi : >> > >> >> Can you check your logs? >> >> >> >> Aki >> >> >> >> >> >> On 01.02.2017 10:02, Poliman - Serwis wrote: >> >>> When I used backup copy of the dovecot.conf file I have this same >> error. >> >> So >> >>> I think that maybe something was written to database? I really would >> >> point >> >>> out that I only added >> >>> passdb { >> >>> driver = passwd-file >> >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>> } >> >>> >> >>> and comment out from above block default lines >> >>> #args = /etc/dovecot/dovecot-sql.conf >> >>> #driver = sql >> >>> >> >>> And in auth_mechanisms add line cram-md5. Nothing more in any other >> file. >> >>> >> >>> I don't want to use cram-md5. I need move back to default settings. >> >>> Cram-md5 was only for testing purposes. :) But I supposed that I can >> move >> >>> back to default by commenting out added lines. But unfortunately it >> isn't >> >>> that simple. >> >>> >> >>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : >> >>> >> >>>> Are you still trying to authenticate using cram-md5? >> >>>> >> >>>> Aki >> >>>> >> >>>> >> >>>> On 01.02.2017 09:51, Poliman - Serwis wrote: >> >>>>> It still use: >> >>>>> passdb { >> >>>>> driver = passwd-file >> >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>>>> } >> >>>>> >> >>>>> When I delete above and delete "cram-md5" in auth_mechanisms it >> still >> >> not >> >>>>> working. >> >>>>> >> >>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : >> >>>>> >> >>>>>> You are probably wanting to do >> >>>>>> passdb { >> >>>>>> driver = passwd-file >> >>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>>>>> } >> >>>>>> >> >>>>>> passdb { >> >>>>>> driver = sql >> >>>>>> args = /etc/dovecot/dovecot-sql.conf >> >>>>>> } >> >>>>>> >> >>>>>> Why you want to use cram-md5 is beyond me, because using SSL is >> much >> >>>>>> more safer. >> >>>>>> >> >>>>>> Aki >> >>>>>> >> >>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: >> >>>>>>> Default it was: "auth_mechanisms = plain login" and I added >> >> cram-md5. >> >>>>>>> After restart all work perfectly. But after I added: >> >>>>>>> driver = passwd-file >> >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>>>>>> I can't set default lines because I got error. Please tell me >> which >> >>>> lines >> >>>>>>> should be changed to resolve this issue. Should I remove "login" >> from >> >>>>>>> auth_mechanism ("login" was default setting and I would like to >> move >> >>>> back >> >>>>>>> to default settings)? >> >>>>>>> >> >>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >> >>>>>>> >> >>>>>>>> Because cram-md5 needs the user's password for calculating >> >> responses, >> >>>> it >> >>>>>>>> cannot work with hashed passwords (one-way encrypted). The only >> >>>>>>>> supported password schemes are PLAIN and CRAM-MD5. >> >>>>>>>> >> >>>>>>>> Aki >> >>>>>>>> >> >>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >> >>>>>>>>> I always restart dovecot after change config. ;) Sure, I >> commented >> >>>> out >> >>>>>>>>> added two lines by me, restarted dovecot and here it is: >> >>>>>>>>> >> >>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf >> >>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >> >>>>>>>>> auth_mechanisms = plain login cram-md5 >> >>>>>>>>> listen = *,[::] >> >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >> >>>>>>>>> mail_max_userip_connections = 100 >> >>>>>>>>> mail_plugins = " quota" >> >>>>>>>>> mail_privileged_group = vmail >> >>>>>>>>> passdb { >> >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >> >>>>>>>>> driver = sql >> >>>>>>>>> } >> >>>>>>>>> plugin { >> >>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >> >>>>>>>>> sieve = /var/vmail/%d/%n/.sieve >> >>>>>>>>> sieve_max_redirects = 25 >> >>>>>>>>> } >> >>>>>>>>> postmaster_address = postmaster at example.com >> >>>>>>>>> protocols = imap pop3 >> >>>>>>>>> service auth { >> >>>>>>>>> unix_listener /var/spool/postfix/private/auth { >> >>>>>>>>> group = postfix >> >>>>>>>>> mode = 0660 >> >>>>>>>>> user = postfix >> >>>>>>>>> } >> >>>>>>>>> unix_listener auth-userdb { >> >>>>>>>>> group = vmail >> >>>>>>>>> mode = 0600 >> >>>>>>>>> user = vmail >> >>>>>>>>> } >> >>>>>>>>> user = root >> >>>>>>>>> } >> >>>>>>>>> service imap-login { >> >>>>>>>>> client_limit = 1000 >> >>>>>>>>> process_limit = 512 >> >>>>>>>>> } >> >>>>>>>>> service lmtp { >> >>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> >>>>>>>>> group = postfix >> >>>>>>>>> mode = 0600 >> >>>>>>>>> user = postfix >> >>>>>>>>> } >> >>>>>>>>> } >> >>>>>>>>> ssl = required >> >>>>>>>>> ssl_cert = > >>>>>>>>> ssl_cipher_list = >> >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >> >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >> >>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >> >>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >> >>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >> >>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >> >>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >> >>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >> >>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >> >>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >> >>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >> >>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >> >>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >> >>>>>>>>> ssl_dh_parameters_length = 2048 >> >>>>>>>>> ssl_key = > >>>>>>>>> ssl_prefer_server_ciphers = yes >> >>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >> >>>>>>>>> userdb { >> >>>>>>>>> driver = prefetch >> >>>>>>>>> } >> >>>>>>>>> userdb { >> >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >> >>>>>>>>> driver = sql >> >>>>>>>>> } >> >>>>>>>>> protocol imap { >> >>>>>>>>> mail_plugins = quota imap_quota >> >>>>>>>>> } >> >>>>>>>>> protocol pop3 { >> >>>>>>>>> mail_plugins = quota >> >>>>>>>>> pop3_uidl_format = %08Xu%08Xv >> >>>>>>>>> } >> >>>>>>>>> protocol lda { >> >>>>>>>>> mail_plugins = sieve quota >> >>>>>>>>> postmaster_address = webmaster at localhost >> >>>>>>>>> } >> >>>>>>>>> protocol lmtp { >> >>>>>>>>> mail_plugins = quota sieve >> >>>>>>>>> postmaster_address = webmaster at localhost >> >>>>>>>>> } >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >> >>>>>>>>> >> >>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >> >>>>>>>>>>> This is debug log files in syslog: >> >>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb >> >> out: >> >>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >> >>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >> >>>>>>>>>> m5ldD4= >> >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >> >>>>>> CONT >> >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: >> >> sql( >> >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email >> as >> >>>>>> user, >> >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >> >>>>>> maildir, >> >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) >> as >> >>>>>>>>>> userdb_mail, >> >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >> quota, >> >>>>>> 'B') >> >>>>>>>> AS >> >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >> >> FROM >> >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email >> = ' >> >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >> >> server_id = >> >>>>>> '1' >> >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): >> password( >> >>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 >> >>>> scheme, >> >>>>>>>>>> but we >> >>>>>>>>>>> have only CRYPT >> >>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb >> >> out: >> >>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com >> >>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >> >>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >> >>>>>> authentication >> >>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT >> >>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >> >>>>>>>> dD4= >> >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >> >>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read >> line; >> >> do >> >>>>>> echo >> >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >> >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >> >>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; >> do >> >>>> echo >> >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >> >>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >> >>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb >> >> out: >> >>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >> >>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >> >>>>>>>>>> m5ldD4= >> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >> >>>>>> CONT >> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: >> >> sql( >> >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email >> as >> >>>>>> user, >> >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >> >>>>>> maildir, >> >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) >> as >> >>>>>>>>>> userdb_mail, >> >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >> quota, >> >>>>>> 'B') >> >>>>>>>> AS >> >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >> >> FROM >> >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email >> = ' >> >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >> >> server_id = >> >>>>>> '1' >> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): >> password( >> >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 >> >>>> scheme, >> >>>>>>>> but >> >>>>>>>>>> we >> >>>>>>>>>>> have only CRYPT >> >>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb >> >> out: >> >>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> ##################### >> >>>>>>>>>>> I added in dovecot.conf lines in passdb block: >> >>>>>>>>>>> driver = passwd-file >> >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>>>>>>>>>> and commented out default lines >> >>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf >> >>>>>>>>>>> #driver = sql >> >>>>>>>>>>> When I try set again default lines I got above error >> >>>>>>>>>> Can you run doveconf -n with the configuration that causes the >> >> above >> >>>>>>>>>> error? Also it clearly does SQL lookup, so that error is >> happening >> >>>>>> with >> >>>>>>>>>> SQL passdb. You need to remember to restart dovecot between >> >>>>>>>>>> configuration changes. >> >>>>>>>>>> >> >>>>>>>>>> Aki >> >>>>>>>>>> >> >>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >> >>>>>>>>>>> >> >>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >> >>>>>>>>>>>>> I set up cram-md5 using this tutorial >> >>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >> >>>>>>>> /etc/dovecot/dovecot.conf >> >>>>>>>>>> in >> >>>>>>>>>>>>> passdb code block: >> >>>>>>>>>>>>> listen = *,[::] >> >>>>>>>>>>>>> protocols = imap pop3 >> >>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 >> >>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login >> >>>>>>>>>>>>> #dodana nizej linia >> >>>>>>>>>>>>> ssl = required >> >>>>>>>>>>>>> disable_plaintext_auth = yes >> >>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >> >>>>>>>>>>>>> mail_privileged_group = vmail >> >>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >> >>>>>>>>>>>>> ssl_cert = > >>>>>>>>>>>>> ssl_key = > >>>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >> >>>>>>>>>>>>> ssl_cipher_list = >> >>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >> >>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ >> >> image: >> >>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >> >>>>>>>>>>>>> ssl_prefer_server_ciphers = yes >> >>>>>>>>>>>>> ssl_dh_parameters_length = 2048 >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> mail_max_userip_connections = 100 >> >>>>>>>>>>>>> passdb { >> >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >> >>>>>>>>>>>>> # driver = sql >> >>>>>>>>>>>>> driver = passwd-file >> >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>>>>>>>>>>>> } >> >>>>>>>>>>>>> userdb { >> >>>>>>>>>>>>> driver = prefetch >> >>>>>>>>>>>>> } >> >>>>>>>>>>>>> userdb { >> >>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >> >>>>>>>>>>>>> driver = sql >> >>>>>>>>>>>>> } >> >>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and >> >> come >> >>>>>>>>>> nicely. >> >>>>>>>>>>>>> But after I want to do default settings by commented out >> these >> >>>> two >> >>>>>>>>>> lines: >> >>>>>>>>>>>>> driver = passwd-file >> >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >> >>>>>>>>>>>>> and uncomment >> >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >> >>>>>>>>>>>>> # driver = sql >> >>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error >> "logging on >> >>>>>>>> server >> >>>>>>>>>>>>> mail.example.com not work out". Error in logs: >> >>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees >> different >> >>>>>>>>>>>>> passdbs/userdbs than auth server. >> >>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file >> was >> >>>>>>>> written >> >>>>>>>>>> to >> >>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't >> >> change >> >>>>>> any >> >>>>>>>>>>>> userdb >> >>>>>>>>>>>>> {} block and this second userdb block has this same lines >> like >> >>>>>>>> default >> >>>>>>>>>>>>> settings in passdb block. >> >>>>>>>>>>>>> >> >>>>>>>>>>>> Try >> >>>>>>>>>>>> >> >>>>>>>>>>>> auth_debug=yes >> >>>>>>>>>>>> auth_verbose=yes >> >>>>>>>>>>>> >> >>>>>>>>>>>> and see if it gives any more reasonable messages. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Aki >> >>>>>>>>>>>> >> >>> >> > >> > >> > > > > -- > > *Pozdrawiam / Best Regards* > *Piotr Bracha* > > > > > *tel. 534 555 877* > > *serwis at poliman.pl * > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From serwis at poliman.pl Thu Feb 2 06:30:21 2017 From: serwis at poliman.pl (Poliman - Serwis) Date: Thu, 2 Feb 2017 07:30:21 +0100 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> Message-ID: And my logs: Error from mail.err: Feb 1 09:50:01 vps342401 postfix/smtpd[699]: fatal: no SASL authentication mechanisms Feb 1 09:51:02 vps342401 postfix/smtpd[724]: fatal: no SASL authentication mechanisms Feb 1 09:51:02 vps342401 postfix/smtpd[725]: fatal: no SASL authentication mechanisms Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: fatal: no SASL authentication mechanisms Error from syslog: Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: connect from host9323131.internet.3s.com[12.34.45.56] Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: fatal: no SASL authentication mechanisms Feb 1 09:52:22 vps342401 postfix/master[29133]: warning: process /usr/lib/postfix/smtpd pid 773 exit status 1 Feb 1 09:52:22 vps342401 postfix/master[29133]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling Feb 1 09:53:01 vps342401 CRON[777]: (root) CMD (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; do ne) Feb 1 09:53:01 vps342401 CRON[778]: (root) CMD (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done ) Is there any strange thing in these config lines? 2017-02-02 7:30 GMT+01:00 Poliman - Serwis : > I haven't doveadm logs in /var/log/. Are they default in another place or > maybe should I turn on something? > My config (default passdb block and auth_mechanisms, nothing more changed): > root at vps342401:/etc/dovecot# doveconf -n > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > auth_mechanisms = plain login > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_max_userip_connections = 100 > mail_plugins = " quota" > mail_privileged_group = vmail > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > sieve_max_redirects = 25 > } > postmaster_address = postmaster at vps342401.ovh.net > protocols = imap pop3 > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service imap-login { > client_limit = 1000 > process_limit = 512 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > ssl = required > ssl_cert = ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:EC > DHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDH > E-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS- > AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE- > ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128- > SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384: > ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA- > AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE- > RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA: > AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256- > SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:! > aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS- > DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > ssl_dh_parameters_length = 2048 > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = sieve quota > postmaster_address = webmaster at localhost > } > protocol lmtp { > mail_plugins = quota sieve > postmaster_address = webmaster at localhost > } > > 2017-02-01 12:16 GMT+01:00 Poliman - Serwis : > >> Is there any strange thing in these config lines? >> >> 2017-02-01 9:40 GMT+01:00 Aki Tuomi : >> >>> doveadm log errors can be helpful too >>> >>> >>> On 01.02.2017 10:25, Poliman - Serwis wrote: >>> > I can check each logs, I have root privileges. >>> > >>> > 2017-02-01 9:04 GMT+01:00 Aki Tuomi : >>> > >>> >> Can you check your logs? >>> >> >>> >> Aki >>> >> >>> >> >>> >> On 01.02.2017 10:02, Poliman - Serwis wrote: >>> >>> When I used backup copy of the dovecot.conf file I have this same >>> error. >>> >> So >>> >>> I think that maybe something was written to database? I really would >>> >> point >>> >>> out that I only added >>> >>> passdb { >>> >>> driver = passwd-file >>> >>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>> } >>> >>> >>> >>> and comment out from above block default lines >>> >>> #args = /etc/dovecot/dovecot-sql.conf >>> >>> #driver = sql >>> >>> >>> >>> And in auth_mechanisms add line cram-md5. Nothing more in any other >>> file. >>> >>> >>> >>> I don't want to use cram-md5. I need move back to default settings. >>> >>> Cram-md5 was only for testing purposes. :) But I supposed that I can >>> move >>> >>> back to default by commenting out added lines. But unfortunately it >>> isn't >>> >>> that simple. >>> >>> >>> >>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : >>> >>> >>> >>>> Are you still trying to authenticate using cram-md5? >>> >>>> >>> >>>> Aki >>> >>>> >>> >>>> >>> >>>> On 01.02.2017 09:51, Poliman - Serwis wrote: >>> >>>>> It still use: >>> >>>>> passdb { >>> >>>>> driver = passwd-file >>> >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>>>> } >>> >>>>> >>> >>>>> When I delete above and delete "cram-md5" in auth_mechanisms it >>> still >>> >> not >>> >>>>> working. >>> >>>>> >>> >>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : >>> >>>>> >>> >>>>>> You are probably wanting to do >>> >>>>>> passdb { >>> >>>>>> driver = passwd-file >>> >>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>>>>> } >>> >>>>>> >>> >>>>>> passdb { >>> >>>>>> driver = sql >>> >>>>>> args = /etc/dovecot/dovecot-sql.conf >>> >>>>>> } >>> >>>>>> >>> >>>>>> Why you want to use cram-md5 is beyond me, because using SSL is >>> much >>> >>>>>> more safer. >>> >>>>>> >>> >>>>>> Aki >>> >>>>>> >>> >>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: >>> >>>>>>> Default it was: "auth_mechanisms = plain login" and I added >>> >> cram-md5. >>> >>>>>>> After restart all work perfectly. But after I added: >>> >>>>>>> driver = passwd-file >>> >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>>>>>> I can't set default lines because I got error. Please tell me >>> which >>> >>>> lines >>> >>>>>>> should be changed to resolve this issue. Should I remove "login" >>> from >>> >>>>>>> auth_mechanism ("login" was default setting and I would like to >>> move >>> >>>> back >>> >>>>>>> to default settings)? >>> >>>>>>> >>> >>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >>> >>>>>>> >>> >>>>>>>> Because cram-md5 needs the user's password for calculating >>> >> responses, >>> >>>> it >>> >>>>>>>> cannot work with hashed passwords (one-way encrypted). The only >>> >>>>>>>> supported password schemes are PLAIN and CRAM-MD5. >>> >>>>>>>> >>> >>>>>>>> Aki >>> >>>>>>>> >>> >>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >>> >>>>>>>>> I always restart dovecot after change config. ;) Sure, I >>> commented >>> >>>> out >>> >>>>>>>>> added two lines by me, restarted dovecot and here it is: >>> >>>>>>>>> >>> >>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf >>> >>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>> >>>>>>>>> auth_mechanisms = plain login cram-md5 >>> >>>>>>>>> listen = *,[::] >>> >>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> >>>>>>>>> mail_max_userip_connections = 100 >>> >>>>>>>>> mail_plugins = " quota" >>> >>>>>>>>> mail_privileged_group = vmail >>> >>>>>>>>> passdb { >>> >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>> >>>>>>>>> driver = sql >>> >>>>>>>>> } >>> >>>>>>>>> plugin { >>> >>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>> >>>>>>>>> sieve = /var/vmail/%d/%n/.sieve >>> >>>>>>>>> sieve_max_redirects = 25 >>> >>>>>>>>> } >>> >>>>>>>>> postmaster_address = postmaster at example.com >>> >>>>>>>>> protocols = imap pop3 >>> >>>>>>>>> service auth { >>> >>>>>>>>> unix_listener /var/spool/postfix/private/auth { >>> >>>>>>>>> group = postfix >>> >>>>>>>>> mode = 0660 >>> >>>>>>>>> user = postfix >>> >>>>>>>>> } >>> >>>>>>>>> unix_listener auth-userdb { >>> >>>>>>>>> group = vmail >>> >>>>>>>>> mode = 0600 >>> >>>>>>>>> user = vmail >>> >>>>>>>>> } >>> >>>>>>>>> user = root >>> >>>>>>>>> } >>> >>>>>>>>> service imap-login { >>> >>>>>>>>> client_limit = 1000 >>> >>>>>>>>> process_limit = 512 >>> >>>>>>>>> } >>> >>>>>>>>> service lmtp { >>> >>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>> >>>>>>>>> group = postfix >>> >>>>>>>>> mode = 0600 >>> >>>>>>>>> user = postfix >>> >>>>>>>>> } >>> >>>>>>>>> } >>> >>>>>>>>> ssl = required >>> >>>>>>>>> ssl_cert = >> >>>>>>>>> ssl_cipher_list = >>> >>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>> >>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >>> >>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >>> >>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >>> >>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >>> >>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >>> >>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >>> >>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >>> >>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >>> >>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >>> >>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >>> >>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >>> >>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>> >>>>>>>>> ssl_dh_parameters_length = 2048 >>> >>>>>>>>> ssl_key = >> >>>>>>>>> ssl_prefer_server_ciphers = yes >>> >>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>> >>>>>>>>> userdb { >>> >>>>>>>>> driver = prefetch >>> >>>>>>>>> } >>> >>>>>>>>> userdb { >>> >>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>> >>>>>>>>> driver = sql >>> >>>>>>>>> } >>> >>>>>>>>> protocol imap { >>> >>>>>>>>> mail_plugins = quota imap_quota >>> >>>>>>>>> } >>> >>>>>>>>> protocol pop3 { >>> >>>>>>>>> mail_plugins = quota >>> >>>>>>>>> pop3_uidl_format = %08Xu%08Xv >>> >>>>>>>>> } >>> >>>>>>>>> protocol lda { >>> >>>>>>>>> mail_plugins = sieve quota >>> >>>>>>>>> postmaster_address = webmaster at localhost >>> >>>>>>>>> } >>> >>>>>>>>> protocol lmtp { >>> >>>>>>>>> mail_plugins = quota sieve >>> >>>>>>>>> postmaster_address = webmaster at localhost >>> >>>>>>>>> } >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>> >>>>>>>>> >>> >>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>> >>>>>>>>>>> This is debug log files in syslog: >>> >>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb >>> >> out: >>> >>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >>> >>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>> >>>>>>>>>> m5ldD4= >>> >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >>> >>>>>> CONT >>> >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: >>> >> sql( >>> >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT >>> email as >>> >>>>>> user, >>> >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, >>> ':', >>> >>>>>> maildir, >>> >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) >>> as >>> >>>>>>>>>> userdb_mail, >>> >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >>> quota, >>> >>>>>> 'B') >>> >>>>>>>> AS >>> >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >>> >> FROM >>> >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR >>> email = ' >>> >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >>> >> server_id = >>> >>>>>> '1' >>> >>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): >>> password( >>> >>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 >>> >>>> scheme, >>> >>>>>>>>>> but we >>> >>>>>>>>>>> have only CRYPT >>> >>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb >>> >> out: >>> >>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com >>> >>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: >>> warning: >>> >>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >>> >>>>>> authentication >>> >>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT >>> >>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >>> >>>>>>>> dD4= >>> >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>> >>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read >>> line; >>> >> do >>> >>>>>> echo >>> >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>> >>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>> >>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read >>> line; do >>> >>>> echo >>> >>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>> >>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>> >>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb >>> >> out: >>> >>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >>> >>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>> >>>>>>>>>> m5ldD4= >>> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>> >>>>>> CONT >>> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: >>> >> sql( >>> >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT >>> email as >>> >>>>>> user, >>> >>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, >>> ':', >>> >>>>>> maildir, >>> >>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) >>> as >>> >>>>>>>>>> userdb_mail, >>> >>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >>> quota, >>> >>>>>> 'B') >>> >>>>>>>> AS >>> >>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >>> >> FROM >>> >>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR >>> email = ' >>> >>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >>> >> server_id = >>> >>>>>> '1' >>> >>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): >>> password( >>> >>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 >>> >>>> scheme, >>> >>>>>>>> but >>> >>>>>>>>>> we >>> >>>>>>>>>>> have only CRYPT >>> >>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb >>> >> out: >>> >>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> ##################### >>> >>>>>>>>>>> I added in dovecot.conf lines in passdb block: >>> >>>>>>>>>>> driver = passwd-file >>> >>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>>>>>>>>>> and commented out default lines >>> >>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf >>> >>>>>>>>>>> #driver = sql >>> >>>>>>>>>>> When I try set again default lines I got above error >>> >>>>>>>>>> Can you run doveconf -n with the configuration that causes the >>> >> above >>> >>>>>>>>>> error? Also it clearly does SQL lookup, so that error is >>> happening >>> >>>>>> with >>> >>>>>>>>>> SQL passdb. You need to remember to restart dovecot between >>> >>>>>>>>>> configuration changes. >>> >>>>>>>>>> >>> >>>>>>>>>> Aki >>> >>>>>>>>>> >>> >>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>> >>>>>>>>>>> >>> >>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>> >>>>>>>>>>>>> I set up cram-md5 using this tutorial >>> >>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >>> >>>>>>>> /etc/dovecot/dovecot.conf >>> >>>>>>>>>> in >>> >>>>>>>>>>>>> passdb code block: >>> >>>>>>>>>>>>> listen = *,[::] >>> >>>>>>>>>>>>> protocols = imap pop3 >>> >>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 >>> >>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login >>> >>>>>>>>>>>>> #dodana nizej linia >>> >>>>>>>>>>>>> ssl = required >>> >>>>>>>>>>>>> disable_plaintext_auth = yes >>> >>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> >>>>>>>>>>>>> mail_privileged_group = vmail >>> >>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>> >>>>>>>>>>>>> ssl_cert = >> >>>>>>>>>>>>> ssl_key = >> >>>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>> >>>>>>>>>>>>> ssl_cipher_list = >>> >>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>> >>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ >>> >> image: >>> >>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>> >>>>>>>>>>>>> ssl_prefer_server_ciphers = yes >>> >>>>>>>>>>>>> ssl_dh_parameters_length = 2048 >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> mail_max_userip_connections = 100 >>> >>>>>>>>>>>>> passdb { >>> >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>> >>>>>>>>>>>>> # driver = sql >>> >>>>>>>>>>>>> driver = passwd-file >>> >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>>>>>>>>>>>> } >>> >>>>>>>>>>>>> userdb { >>> >>>>>>>>>>>>> driver = prefetch >>> >>>>>>>>>>>>> } >>> >>>>>>>>>>>>> userdb { >>> >>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>> >>>>>>>>>>>>> driver = sql >>> >>>>>>>>>>>>> } >>> >>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and >>> >> come >>> >>>>>>>>>> nicely. >>> >>>>>>>>>>>>> But after I want to do default settings by commented out >>> these >>> >>>> two >>> >>>>>>>>>> lines: >>> >>>>>>>>>>>>> driver = passwd-file >>> >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>> >>>>>>>>>>>>> and uncomment >>> >>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>> >>>>>>>>>>>>> # driver = sql >>> >>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error >>> "logging on >>> >>>>>>>> server >>> >>>>>>>>>>>>> mail.example.com not work out". Error in logs: >>> >>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees >>> different >>> >>>>>>>>>>>>> passdbs/userdbs than auth server. >>> >>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file >>> was >>> >>>>>>>> written >>> >>>>>>>>>> to >>> >>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't >>> >> change >>> >>>>>> any >>> >>>>>>>>>>>> userdb >>> >>>>>>>>>>>>> {} block and this second userdb block has this same lines >>> like >>> >>>>>>>> default >>> >>>>>>>>>>>>> settings in passdb block. >>> >>>>>>>>>>>>> >>> >>>>>>>>>>>> Try >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> auth_debug=yes >>> >>>>>>>>>>>> auth_verbose=yes >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> and see if it gives any more reasonable messages. >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> Aki >>> >>>>>>>>>>>> >>> >>> >>> > >>> > >>> >> >> >> >> -- >> >> *Pozdrawiam / Best Regards* >> *Piotr Bracha* >> >> >> >> >> *tel. 534 555 877* >> >> *serwis at poliman.pl * >> > > > > -- > > *Pozdrawiam / Best Regards* > *Piotr Bracha* > > > > > *tel. 534 555 877* > > *serwis at poliman.pl * > -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis at poliman.pl * From paul.atreides83 at googlemail.com Thu Feb 2 14:05:58 2017 From: paul.atreides83 at googlemail.com (Paul Atreides) Date: Thu, 2 Feb 2017 15:05:58 +0100 Subject: Revision-proof archiving Message-ID: Hi, I am interested in the "Dovecot Email Archive" solution. Does anyone know if it is suitable revision-proof archiving? Is it available for smaller companies? I tried to contact people at dovecot.fi but I haven?t gotten any response yet. Thanks Regards, From mikefroehner at gmx.de Fri Feb 3 09:34:43 2017 From: mikefroehner at gmx.de (=?UTF-8?Q?Mike_Fr=c3=b6hner?=) Date: Fri, 3 Feb 2017 10:34:43 +0100 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: <1547742423.20170203095111@maelenn.org> References: <1547742423.20170203095111@maelenn.org> Message-ID: Hello, On 02/03/2017 08:51 AM, Thierry wrote: > Hello, > > Still working with my dsync pb. > I have done a clone (vmware) of my email server. > Today I have two strictly identical emails servers (server1 > (main) and server2 (bck) (except IP, hostname and mail_replica). > > The ssl config on my both server: > > ssl_protocols = !SSLv2 !SSLv3 > ssl = required > verbose_ssl = no > ssl_key = ssl_cert = ssl_ca = > This config is working for my email client and my email web > interface ... > > Are they on the right order ? > > mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd > > There is trafic on my iptables rules on my both servers: > > 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 > > > > My error message from server1 (main server): > > Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) > Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) > Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) > Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) > > No logs from server2 > > Any ideas ? > > Thx for your support > > From preen at informatik.uni-freiburg.de Fri Feb 3 10:36:04 2017 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Fri, 03 Feb 2017 11:36:04 +0100 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> Message-ID: <58945D14.3030201@informatik.uni-freiburg.de> Hello, I don't have problems building 2.2.27 on Solaris 10 (using Sun Workshop compiler 5.11). The configuration is the same as your. Maybe a compiler/version problem on your system ? Regards, Martin Mantas Gegu?is wrote: > Hello, > > I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: > test-ioloop.c: In function `test_ioloop_pending_io': > test-ioloop.c:188: error: size of array `type name' is negative > > My configuration is like this: > Install prefix . : /usr/local > File offsets ... : 64bit > I/O polling .... : poll > I/O notifys .... : none > SSL ............ : yes (OpenSSL) > GSSAPI ......... : no > passdbs ........ : static passwd passwd-file shadow pam checkpassword > dcrypt ..........: yes > : -bsdauth -sia -ldap -sql -vpopmail > userdbs ........ : static prefetch passwd passwd-file checkpassword > : -ldap -sql -vpopmail -nss > SQL drivers .... : > : -pgsql -mysql -sqlite -cassandra > Full text search : squat > : -lucene -solr > > Last version that I have compiled was 2.2.24, version 2.2.25 failed with error: > In file included from guid.c:6: > sha1.h:80: error: static or type qualifiers in abstract declarator > > Is there anyone who can help me? ---------------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/staff/preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5047 bytes Desc: S/MIME Cryptographic Signature URL: From skdovecot at smail.inf.fh-brs.de Fri Feb 3 10:42:46 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 3 Feb 2017 11:42:46 +0100 (CET) Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 1 Feb 2017, Poliman - Serwis wrote: > I haven't doveadm logs in /var/log/. Are they default in another place or > maybe should I turn on something? run doveadm log find as root. Maybe: doveadm log errors > My config (default passdb block and auth_mechanisms, nothing more changed): Is this still a question about CRAM ? I don't see it there. > root at vps342401:/etc/dovecot# doveconf -n > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > auth_mechanisms = plain login > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_max_userip_connections = 100 > mail_plugins = " quota" > mail_privileged_group = vmail > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > sieve_max_redirects = 25 > } > postmaster_address = postmaster at vps342401.ovh.net > protocols = imap pop3 > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service imap-login { > client_limit = 1000 > process_limit = 512 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > ssl = required > ssl_cert = ssl_cipher_list = > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > ssl_dh_parameters_length = 2048 > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = sieve quota > postmaster_address = webmaster at localhost > } > protocol lmtp { > mail_plugins = quota sieve > postmaster_address = webmaster at localhost > } > > Error from mail.err: > Feb 1 09:50:01 vps342401 postfix/smtpd[699]: fatal: no SASL authentication > mechanisms > Feb 1 09:51:02 vps342401 postfix/smtpd[724]: fatal: no SASL authentication > mechanisms > Feb 1 09:51:02 vps342401 postfix/smtpd[725]: fatal: no SASL authentication > mechanisms > Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: fatal: no SASL > authentication mechanisms > > Error from syslog: > Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: connect from > host9323131.internet.3s.com[12.34.45.56] > Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: fatal: no SASL > authentication mechanisms > Feb 1 09:52:22 vps342401 postfix/master[29133]: warning: process > /usr/lib/postfix/smtpd pid 773 exit status 1 > Feb 1 09:52:22 vps342401 postfix/master[29133]: warning: > /usr/lib/postfix/smtpd: bad command startup -- throttling > Feb 1 09:53:01 vps342401 CRON[777]: (root) CMD > (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; do ne) > Feb 1 09:53:01 vps342401 CRON[778]: (root) CMD > (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done ) > > > 2017-02-01 9:40 GMT+01:00 Aki Tuomi : > >> doveadm log errors can be helpful too >> >> >> On 01.02.2017 10:25, Poliman - Serwis wrote: >>> I can check each logs, I have root privileges. >>> >>> 2017-02-01 9:04 GMT+01:00 Aki Tuomi : >>> >>>> Can you check your logs? >>>> >>>> Aki >>>> >>>> >>>> On 01.02.2017 10:02, Poliman - Serwis wrote: >>>>> When I used backup copy of the dovecot.conf file I have this same >> error. >>>> So >>>>> I think that maybe something was written to database? I really would >>>> point >>>>> out that I only added >>>>> passdb { >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> } >>>>> >>>>> and comment out from above block default lines >>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>> #driver = sql >>>>> >>>>> And in auth_mechanisms add line cram-md5. Nothing more in any other >> file. >>>>> >>>>> I don't want to use cram-md5. I need move back to default settings. >>>>> Cram-md5 was only for testing purposes. :) But I supposed that I can >> move >>>>> back to default by commenting out added lines. But unfortunately it >> isn't >>>>> that simple. >>>>> >>>>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : >>>>> >>>>>> Are you still trying to authenticate using cram-md5? >>>>>> >>>>>> Aki >>>>>> >>>>>> >>>>>> On 01.02.2017 09:51, Poliman - Serwis wrote: >>>>>>> It still use: >>>>>>> passdb { >>>>>>> driver = passwd-file >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>> } >>>>>>> >>>>>>> When I delete above and delete "cram-md5" in auth_mechanisms it still >>>> not >>>>>>> working. >>>>>>> >>>>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : >>>>>>> >>>>>>>> You are probably wanting to do >>>>>>>> passdb { >>>>>>>> driver = passwd-file >>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>> } >>>>>>>> >>>>>>>> passdb { >>>>>>>> driver = sql >>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>> } >>>>>>>> >>>>>>>> Why you want to use cram-md5 is beyond me, because using SSL is much >>>>>>>> more safer. >>>>>>>> >>>>>>>> Aki >>>>>>>> >>>>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: >>>>>>>>> Default it was: "auth_mechanisms = plain login" and I added >>>> cram-md5. >>>>>>>>> After restart all work perfectly. But after I added: >>>>>>>>> driver = passwd-file >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>> I can't set default lines because I got error. Please tell me which >>>>>> lines >>>>>>>>> should be changed to resolve this issue. Should I remove "login" >> from >>>>>>>>> auth_mechanism ("login" was default setting and I would like to >> move >>>>>> back >>>>>>>>> to default settings)? >>>>>>>>> >>>>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >>>>>>>>> >>>>>>>>>> Because cram-md5 needs the user's password for calculating >>>> responses, >>>>>> it >>>>>>>>>> cannot work with hashed passwords (one-way encrypted). The only >>>>>>>>>> supported password schemes are PLAIN and CRAM-MD5. >>>>>>>>>> >>>>>>>>>> Aki >>>>>>>>>> >>>>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >>>>>>>>>>> I always restart dovecot after change config. ;) Sure, I >> commented >>>>>> out >>>>>>>>>>> added two lines by me, restarted dovecot and here it is: >>>>>>>>>>> >>>>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>>>>>>>>>> auth_mechanisms = plain login cram-md5 >>>>>>>>>>> listen = *,[::] >>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>>>> mail_plugins = " quota" >>>>>>>>>>> mail_privileged_group = vmail >>>>>>>>>>> passdb { >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> driver = sql >>>>>>>>>>> } >>>>>>>>>>> plugin { >>>>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>>>>>>>>> sieve = /var/vmail/%d/%n/.sieve >>>>>>>>>>> sieve_max_redirects = 25 >>>>>>>>>>> } >>>>>>>>>>> postmaster_address = postmaster at example.com >>>>>>>>>>> protocols = imap pop3 >>>>>>>>>>> service auth { >>>>>>>>>>> unix_listener /var/spool/postfix/private/auth { >>>>>>>>>>> group = postfix >>>>>>>>>>> mode = 0660 >>>>>>>>>>> user = postfix >>>>>>>>>>> } >>>>>>>>>>> unix_listener auth-userdb { >>>>>>>>>>> group = vmail >>>>>>>>>>> mode = 0600 >>>>>>>>>>> user = vmail >>>>>>>>>>> } >>>>>>>>>>> user = root >>>>>>>>>>> } >>>>>>>>>>> service imap-login { >>>>>>>>>>> client_limit = 1000 >>>>>>>>>>> process_limit = 512 >>>>>>>>>>> } >>>>>>>>>>> service lmtp { >>>>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>>>>>>>>> group = postfix >>>>>>>>>>> mode = 0600 >>>>>>>>>>> user = postfix >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> ssl = required >>>>>>>>>>> ssl_cert = >>>>>>>>>> ssl_cipher_list = >>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >>>>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >>>>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >>>>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >>>>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >>>>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >>>>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >>>>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >>>>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >>>>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >>>>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >>>>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>>>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>>>> ssl_key = >>>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>>>> userdb { >>>>>>>>>>> driver = prefetch >>>>>>>>>>> } >>>>>>>>>>> userdb { >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> driver = sql >>>>>>>>>>> } >>>>>>>>>>> protocol imap { >>>>>>>>>>> mail_plugins = quota imap_quota >>>>>>>>>>> } >>>>>>>>>>> protocol pop3 { >>>>>>>>>>> mail_plugins = quota >>>>>>>>>>> pop3_uidl_format = %08Xu%08Xv >>>>>>>>>>> } >>>>>>>>>>> protocol lda { >>>>>>>>>>> mail_plugins = sieve quota >>>>>>>>>>> postmaster_address = webmaster at localhost >>>>>>>>>>> } >>>>>>>>>>> protocol lmtp { >>>>>>>>>>> mail_plugins = quota sieve >>>>>>>>>>> postmaster_address = webmaster at localhost >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>>>>>>>>>> >>>>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>>>>>>>>>>>> This is debug log files in syslog: >>>>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >>>>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>>>>>>>>>>> m5ldD4= >>>>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >>>>>>>> CONT >>>>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: >>>> sql( >>>>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email >> as >>>>>>>> user, >>>>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>>>>>> maildir, >>>>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>>>>>> userdb_mail, >>>>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >> quota, >>>>>>>> 'B') >>>>>>>>>> AS >>>>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >>>> FROM >>>>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email >> = ' >>>>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >>>> server_id = >>>>>>>> '1' >>>>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): >> password( >>>>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 >>>>>> scheme, >>>>>>>>>>>> but we >>>>>>>>>>>>> have only CRYPT >>>>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com >>>>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>>>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >>>>>>>> authentication >>>>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT >>>>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >>>>>>>>>> dD4= >>>>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>>>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; >>>> do >>>>>>>> echo >>>>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>>>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; >> do >>>>>> echo >>>>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>>>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >>>>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>>>>>>>>>>> m5ldD4= >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>>> CONT >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: >>>> sql( >>>>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email >> as >>>>>>>> user, >>>>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>>>>>> maildir, >>>>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>>>>>> userdb_mail, >>>>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >> quota, >>>>>>>> 'B') >>>>>>>>>> AS >>>>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >>>> FROM >>>>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email >> = ' >>>>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >>>> server_id = >>>>>>>> '1' >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): >> password( >>>>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 >>>>>> scheme, >>>>>>>>>> but >>>>>>>>>>>> we >>>>>>>>>>>>> have only CRYPT >>>>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> ##################### >>>>>>>>>>>>> I added in dovecot.conf lines in passdb block: >>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>> and commented out default lines >>>>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>> #driver = sql >>>>>>>>>>>>> When I try set again default lines I got above error >>>>>>>>>>>> Can you run doveconf -n with the configuration that causes the >>>> above >>>>>>>>>>>> error? Also it clearly does SQL lookup, so that error is >> happening >>>>>>>> with >>>>>>>>>>>> SQL passdb. You need to remember to restart dovecot between >>>>>>>>>>>> configuration changes. >>>>>>>>>>>> >>>>>>>>>>>> Aki >>>>>>>>>>>> >>>>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>>>>>>>>>>>> >>>>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>>>>>>>>>>>> I set up cram-md5 using this tutorial >>>>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >>>>>>>>>> /etc/dovecot/dovecot.conf >>>>>>>>>>>> in >>>>>>>>>>>>>>> passdb code block: >>>>>>>>>>>>>>> listen = *,[::] >>>>>>>>>>>>>>> protocols = imap pop3 >>>>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 >>>>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login >>>>>>>>>>>>>>> #dodana nizej linia >>>>>>>>>>>>>>> ssl = required >>>>>>>>>>>>>>> disable_plaintext_auth = yes >>>>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>>>>>>>> mail_privileged_group = vmail >>>>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>>>>>>>>>>>> ssl_cert = >>>>>>>>>>>>>> ssl_key = >>>>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>>>>>>>> ssl_cipher_list = >>>>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ >>>> image: >>>>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>>>>>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>>>>>>>> passdb { >>>>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>>>> # driver = sql >>>>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> userdb { >>>>>>>>>>>>>>> driver = prefetch >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> userdb { >>>>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>>>> driver = sql >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and >>>> come >>>>>>>>>>>> nicely. >>>>>>>>>>>>>>> But after I want to do default settings by commented out >> these >>>>>> two >>>>>>>>>>>> lines: >>>>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>>>> and uncomment >>>>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>>>> # driver = sql >>>>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging >> on >>>>>>>>>> server >>>>>>>>>>>>>>> mail.example.com not work out". Error in logs: >>>>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees >> different >>>>>>>>>>>>>>> passdbs/userdbs than auth server. >>>>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file >> was >>>>>>>>>> written >>>>>>>>>>>> to >>>>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't >>>> change >>>>>>>> any >>>>>>>>>>>>>> userdb >>>>>>>>>>>>>>> {} block and this second userdb block has this same lines >> like >>>>>>>>>> default >>>>>>>>>>>>>>> settings in passdb block. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Try >>>>>>>>>>>>>> >>>>>>>>>>>>>> auth_debug=yes >>>>>>>>>>>>>> auth_verbose=yes >>>>>>>>>>>>>> >>>>>>>>>>>>>> and see if it gives any more reasonable messages. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Aki >>>>>>>>>>>>>> >>>>> >>> >>> >> > > > > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWJRepnz1H7kL/d9rAQKj6gf/eKXC6JV/69gmyXaC3iSwNHmOS6qjYlFl L+cUexFQM/t/tk0z/N9olmcIm8tJd1HFruJGrb9/StBirenuJYJ54AOyd3zi8XDg Gu+vbcBE2T97w48SqTsujJKPT/dVFZ9kHtYymNMjLNJANdr/X4r+/QNw710B96US FDNc96xBGKjrn/uE0SToclFXuvOE4Ymu8JGQHDQO7X35r9M9NBLfSP8VXwtIlnDX 9P/UQvisFuLNtXHh4wO77b0Jdw3V2CYgER0l5ctHYAgaS4d8CNGHnINLZvFiJusL s4TG5Yf1OHC3wMiRCikybkO5fNezXuvc7xMbKYV9HDKxjLvP1paAPA== =gHJk -----END PGP SIGNATURE----- From lohm at lynet.de Fri Feb 3 11:18:27 2017 From: lohm at lynet.de (Matthias Lohmann) Date: Fri, 3 Feb 2017 12:18:27 +0100 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> Message-ID: <9875abe5-12bf-d59a-34ff-decac6d5018f@lynet.de> Hi there... I just had the same problem yesterday. I believe you're using the gcc3 compiler from /usr/sfw ? Try with a newer gcc 4. I used/installed the gcc4 packages from opencsw.org to get it going faster. (Installs under /opt/csw, so no problems with existing software.) Change your PATH to "/opt/csw/bin/gcc4:/opt/csw/bin:$PATH" bevor configure/compile and give it a try. Worked for me ... and 2.2.27 runs since a few hours. Am 02.02.2017 um 18:38 schrieb Mantas Gegu?is: > Hello, > > I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: > test-ioloop.c: In function `test_ioloop_pending_io': > test-ioloop.c:188: error: size of array `type name' is negative > > My configuration is like this: > Install prefix . : /usr/local > File offsets ... : 64bit > I/O polling .... : poll > I/O notifys .... : none > SSL ............ : yes (OpenSSL) > GSSAPI ......... : no > passdbs ........ : static passwd passwd-file shadow pam checkpassword > dcrypt ..........: yes > : -bsdauth -sia -ldap -sql -vpopmail > userdbs ........ : static prefetch passwd passwd-file checkpassword > : -ldap -sql -vpopmail -nss > SQL drivers .... : > : -pgsql -mysql -sqlite -cassandra > Full text search : squat > : -lucene -solr > > Last version that I have compiled was 2.2.24, version 2.2.25 failed > with error: > In file included from guid.c:6: > sha1.h:80: error: static or type qualifiers in abstract declarator > > Is there anyone who can help me? > From lenaigst at maelenn.org Fri Feb 3 11:48:21 2017 From: lenaigst at maelenn.org (Thierry) Date: Fri, 3 Feb 2017 13:48:21 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: References: <1547742423.20170203095111@maelenn.org> Message-ID: <1215770952.20170203134821@maelenn.org> Bonjour Mike, I have made the change from 'ssl_ca =' tp 'ssl_client_ca_file =' but now I do have: Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long thx Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : > Hello, > On 02/03/2017 08:51 AM, Thierry wrote: >> Hello, >> >> Still working with my dsync pb. >> I have done a clone (vmware) of my email server. >> Today I have two strictly identical emails servers (server1 >> (main) and server2 (bck) (except IP, hostname and mail_replica). >> >> The ssl config on my both server: >> >> ssl_protocols = !SSLv2 !SSLv3 >> ssl = required >> verbose_ssl = no >> ssl_key = > ssl_cert = > ssl_ca = I think it should be ssl_client_ca_file = > > >> This config is working for my email client and my email web >> interface ... >> >> Are they on the right order ? >> >> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >> >> There is trafic on my iptables rules on my both servers: >> >> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >> >> >> >> My error message from server1 (main server): >> >> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> >> No logs from server2 >> >> Any ideas ? >> >> Thx for your support >> >> -- Cordialement, Thierry e-mail : lenaigst at maelenn.org From lenaigst at maelenn.org Fri Feb 3 13:13:12 2017 From: lenaigst at maelenn.org (Thierry) Date: Fri, 3 Feb 2017 15:13:12 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: References: <1547742423.20170203095111@maelenn.org> Message-ID: <19710310116.20170203151312@maelenn.org> Hi, I have made change: ssl_protocols = !SSLv2 !SSLv3 ssl = required verbose_ssl = no ssl_key = Hello, > On 02/03/2017 08:51 AM, Thierry wrote: >> Hello, >> >> Still working with my dsync pb. >> I have done a clone (vmware) of my email server. >> Today I have two strictly identical emails servers (server1 >> (main) and server2 (bck) (except IP, hostname and mail_replica). >> >> The ssl config on my both server: >> >> ssl_protocols = !SSLv2 !SSLv3 >> ssl = required >> verbose_ssl = no >> ssl_key = > ssl_cert = > ssl_ca = I think it should be ssl_client_ca_file = > > >> This config is working for my email client and my email web >> interface ... >> >> Are they on the right order ? >> >> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >> >> There is trafic on my iptables rules on my both servers: >> >> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >> >> >> >> My error message from server1 (main server): >> >> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >> >> No logs from server2 >> >> Any ideas ? >> >> Thx for your support >> >> -- Cordialement, Thierry e-mail : lenaigst at maelenn.org From aki.tuomi at dovecot.fi Fri Feb 3 13:54:27 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 3 Feb 2017 15:54:27 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: <19710310116.20170203151312@maelenn.org> References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> Message-ID: <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> Yes. The ssl_client_ca_file is not actually expecting <, just file name. Aki On 2017-02-03 15:13, Thierry wrote: > Hi, > > I have made change: > > ssl_protocols = !SSLv2 !SSLv3 > ssl = required > verbose_ssl = no > ssl_key = ssl_cert = ssl_client_ca_file = > > # Create a listener for doveadm-server > service doveadm { > user = vmail > inet_listener { > port = 12345 > ssl= yes > } > } > > and doveadm_port = 12345 // mail_replica = tcps:server2.domain.ltd # use doveadm_port > > And now: > > Feb 03 14:11:16 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long > Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 > Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL > > Thx for your support > > > > > Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : > >> Hello, > >> On 02/03/2017 08:51 AM, Thierry wrote: >>> Hello, >>> >>> Still working with my dsync pb. >>> I have done a clone (vmware) of my email server. >>> Today I have two strictly identical emails servers (server1 >>> (main) and server2 (bck) (except IP, hostname and mail_replica). >>> >>> The ssl config on my both server: >>> >>> ssl_protocols = !SSLv2 !SSLv3 >>> ssl = required >>> verbose_ssl = no >>> ssl_key = >> ssl_cert = >> ssl_ca = > I think it should be ssl_client_ca_file = >> >>> This config is working for my email client and my email web >>> interface ... >>> >>> Are they on the right order ? >>> >>> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >>> >>> There is trafic on my iptables rules on my both servers: >>> >>> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >>> >>> >>> >>> My error message from server1 (main server): >>> >>> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>> >>> No logs from server2 >>> >>> Any ideas ? >>> >>> Thx for your support >>> >>> > > From mantas.geguzis at ittc.vu.lt Fri Feb 3 14:10:20 2017 From: mantas.geguzis at ittc.vu.lt (Mantas =?utf-8?b?R2VndcW+aXM=?=) Date: Fri, 03 Feb 2017 16:10:20 +0200 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <58945D14.3030201@informatik.uni-freiburg.de> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> <58945D14.3030201@informatik.uni-freiburg.de> Message-ID: <20170203161020.Horde.SmuKZ0VptGAHygCqpZfwjA1@webmail.vu.lt> Hi, thank You for a reply, compiling with Solaris Studio 12.5 solved this case. Martin Preen ra??: > Hello, > I don't have problems building 2.2.27 on Solaris 10 > (using Sun Workshop compiler 5.11). > > The configuration is the same as your. > Maybe a compiler/version problem on your system ? > > Regards, > Martin > > Mantas Gegu?is wrote: >> Hello, >> >> I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: >> test-ioloop.c: In function `test_ioloop_pending_io': >> test-ioloop.c:188: error: size of array `type name' is negative >> >> My configuration is like this: >> Install prefix . : /usr/local >> File offsets ... : 64bit >> I/O polling .... : poll >> I/O notifys .... : none >> SSL ............ : yes (OpenSSL) >> GSSAPI ......... : no >> passdbs ........ : static passwd passwd-file shadow pam checkpassword >> dcrypt ..........: yes >> : -bsdauth -sia -ldap -sql -vpopmail >> userdbs ........ : static prefetch passwd passwd-file checkpassword >> : -ldap -sql -vpopmail -nss >> SQL drivers .... : >> : -pgsql -mysql -sqlite -cassandra >> Full text search : squat >> : -lucene -solr >> >> Last version that I have compiled was 2.2.24, version 2.2.25 failed >> with error: >> In file included from guid.c:6: >> sha1.h:80: error: static or type qualifiers in abstract declarator >> >> Is there anyone who can help me? > > ---------------------------------------------------------------------- > Martin Preen, Universit?t Freiburg, Institut f?r Informatik > Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany > > phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de > fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/staff/preen -- Pagarbiai Mantas Gegu?is VU Informacini? technologij? taikymo centras tel. 8 5 236 6208 From aki.tuomi at dovecot.fi Fri Feb 3 15:09:52 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 3 Feb 2017 17:09:52 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: <198484904.20170203170031@maelenn.org> References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> Message-ID: Please keep responses in list. rm -f /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir. On 2017-02-03 17:00, Thierry wrote: > Hi, > > I have removed the '<' : > > ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem > > But now: > > doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 > doveadm: Error: Couldn't initialize SSL parameters, disabling SSL > doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 > doveadm: Error: Couldn't initialize SSL parameters, disabling SSL > > Any idea ? > > Thx > >> Yes. The ssl_client_ca_file is not actually expecting <, just file name. >> Aki > >> On 2017-02-03 15:13, Thierry wrote: >>> Hi, >>> >>> I have made change: >>> >>> ssl_protocols = !SSLv2 !SSLv3 >>> ssl = required >>> verbose_ssl = no >>> ssl_key = >> ssl_cert = >> ssl_client_ca_file = >> >>> >>> # Create a listener for doveadm-server >>> service doveadm { >>> user = vmail >>> inet_listener { >>> port = 12345 >>> ssl= yes >>> } >>> } >>> >>> and doveadm_port = 12345 // mail_replica = tcps:server2.domain.ltd # use doveadm_port >>> >>> And now: >>> >>> Feb 03 14:11:16 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long >>> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>> >>> Thx for your support >>> >>> >>> >>> >>> Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : >>> >>>> Hello, >>>> On 02/03/2017 08:51 AM, Thierry wrote: >>>>> Hello, >>>>> >>>>> Still working with my dsync pb. >>>>> I have done a clone (vmware) of my email server. >>>>> Today I have two strictly identical emails servers (server1 >>>>> (main) and server2 (bck) (except IP, hostname and mail_replica). >>>>> >>>>> The ssl config on my both server: >>>>> >>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>> ssl = required >>>>> verbose_ssl = no >>>>> ssl_key = >>>> ssl_cert = >>>> ssl_ca = >>> I think it should be ssl_client_ca_file = >>>> >>>> This config is working for my email client and my email web >>>>> interface ... >>>>> >>>>> Are they on the right order ? >>>>> >>>>> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >>>>> >>>>> There is trafic on my iptables rules on my both servers: >>>>> >>>>> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >>>>> >>>>> >>>>> >>>>> My error message from server1 (main server): >>>>> >>>>> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>> >>>>> No logs from server2 >>>>> >>>>> Any ideas ? >>>>> >>>>> Thx for your support >>>>> >>>>> >>> From lenaigst at maelenn.org Fri Feb 3 15:39:16 2017 From: lenaigst at maelenn.org (Thierry) Date: Fri, 3 Feb 2017 17:39:16 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> Message-ID: <1004866901.20170203173916@maelenn.org> Hi, I have removed it on both server and on both server I do have: ssl-params: Info: Generating SSL parameters ssl-params: Info: SSL parameters regeneration completed But still: Feb 03 16:36:28 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 Feb 03 16:36:28 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL Thx Le vendredi 3 f?vrier 2017 ? 17:09:52, vous ?criviez : > Please keep responses in list. rm -f > /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir. > On 2017-02-03 17:00, Thierry wrote: >> Hi, >> >> I have removed the '<' : >> >> ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem >> >> But now: >> >> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >> >> Any idea ? >> >> Thx >> >>> Yes. The ssl_client_ca_file is not actually expecting <, just file name. >>> Aki >> >>> On 2017-02-03 15:13, Thierry wrote: >>>> Hi, >>>> >>>> I have made change: >>>> >>>> ssl_protocols = !SSLv2 !SSLv3 >>>> ssl = required >>>> verbose_ssl = no >>>> ssl_key = >>> ssl_cert = >>> ssl_client_ca_file = >>> >>>> >>>> # Create a listener for doveadm-server >>>> service doveadm { >>>> user = vmail >>>> inet_listener { >>>> port = 12345 >>>> ssl= yes >>>> } >>>> } >>>> >>>> and doveadm_port = 12345 // mail_replica = tcps:server2.domain.ltd # use doveadm_port >>>> >>>> And now: >>>> >>>> Feb 03 14:11:16 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long >>>> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>>> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>>> >>>> Thx for your support >>>> >>>> >>>> >>>> >>>> Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : >>>> >>>>> Hello, >>>>> On 02/03/2017 08:51 AM, Thierry wrote: >>>>>> Hello, >>>>>> >>>>>> Still working with my dsync pb. >>>>>> I have done a clone (vmware) of my email server. >>>>>> Today I have two strictly identical emails servers (server1 >>>>>> (main) and server2 (bck) (except IP, hostname and mail_replica). >>>>>> >>>>>> The ssl config on my both server: >>>>>> >>>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>>> ssl = required >>>>>> verbose_ssl = no >>>>>> ssl_key = >>>>> ssl_cert = >>>>> ssl_ca = >>>> I think it should be ssl_client_ca_file = >>>>> >>>>> This config is working for my email client and my email web >>>>>> interface ... >>>>>> >>>>>> Are they on the right order ? >>>>>> >>>>>> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >>>>>> >>>>>> There is trafic on my iptables rules on my both servers: >>>>>> >>>>>> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >>>>>> >>>>>> >>>>>> >>>>>> My error message from server1 (main server): >>>>>> >>>>>> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> >>>>>> No logs from server2 >>>>>> >>>>>> Any ideas ? >>>>>> >>>>>> Thx for your support >>>>>> >>>>>> >>>> -- Cordialement, Thierry e-mail : lenaigst at maelenn.org From aki.tuomi at dovecot.fi Fri Feb 3 16:38:12 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 3 Feb 2017 18:38:12 +0200 Subject: Dovecot auth-worker error after cram-md5 auth In-Reply-To: References: <079ab92e-9492-c1bf-6f1c-839f988fd13a@dovecot.fi> <47d809a7-975b-500c-d846-0c85874ba26d@dovecot.fi> <818268ed-709b-5b83-f138-7edf91ec1dba@dovecot.fi> Message-ID: You could try install libsasl2-modules (on debian/ubuntu) or cyrus-sasl-plain (on rhel/centos) Aki On 2017-02-01 10:55, Poliman - Serwis wrote: > I haven't doveadm logs in /var/log/. Are they default in another place or > maybe should I turn on something? > > My config (default passdb block and auth_mechanisms, nothing more changed): > root at vps342401:/etc/dovecot# doveconf -n > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS > auth_mechanisms = plain login > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_max_userip_connections = 100 > mail_plugins = " quota" > mail_privileged_group = vmail > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > sieve = /var/vmail/%d/%n/.sieve > sieve_max_redirects = 25 > } > postmaster_address = postmaster at vps342401.ovh.net > protocols = imap pop3 > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service imap-login { > client_limit = 1000 > process_limit = 512 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > ssl = required > ssl_cert = ssl_cipher_list = > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA > ssl_dh_parameters_length = 2048 > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = sieve quota > postmaster_address = webmaster at localhost > } > protocol lmtp { > mail_plugins = quota sieve > postmaster_address = webmaster at localhost > } > > Error from mail.err: > Feb 1 09:50:01 vps342401 postfix/smtpd[699]: fatal: no SASL authentication > mechanisms > Feb 1 09:51:02 vps342401 postfix/smtpd[724]: fatal: no SASL authentication > mechanisms > Feb 1 09:51:02 vps342401 postfix/smtpd[725]: fatal: no SASL authentication > mechanisms > Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: fatal: no SASL > authentication mechanisms > > Error from syslog: > Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: connect from > host9323131.internet.3s.com[12.34.45.56] > Feb 1 09:52:21 vps342401 postfix/smtps/smtpd[773]: fatal: no SASL > authentication mechanisms > Feb 1 09:52:22 vps342401 postfix/master[29133]: warning: process > /usr/lib/postfix/smtpd pid 773 exit status 1 > Feb 1 09:52:22 vps342401 postfix/master[29133]: warning: > /usr/lib/postfix/smtpd: bad command startup -- throttling > Feb 1 09:53:01 vps342401 CRON[777]: (root) CMD > (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; do ne) > Feb 1 09:53:01 vps342401 CRON[778]: (root) CMD > (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo > `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done ) > > > 2017-02-01 9:40 GMT+01:00 Aki Tuomi : > >> doveadm log errors can be helpful too >> >> >> On 01.02.2017 10:25, Poliman - Serwis wrote: >>> I can check each logs, I have root privileges. >>> >>> 2017-02-01 9:04 GMT+01:00 Aki Tuomi : >>> >>>> Can you check your logs? >>>> >>>> Aki >>>> >>>> >>>> On 01.02.2017 10:02, Poliman - Serwis wrote: >>>>> When I used backup copy of the dovecot.conf file I have this same >> error. >>>> So >>>>> I think that maybe something was written to database? I really would >>>> point >>>>> out that I only added >>>>> passdb { >>>>> driver = passwd-file >>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>> } >>>>> >>>>> and comment out from above block default lines >>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>> #driver = sql >>>>> >>>>> And in auth_mechanisms add line cram-md5. Nothing more in any other >> file. >>>>> I don't want to use cram-md5. I need move back to default settings. >>>>> Cram-md5 was only for testing purposes. :) But I supposed that I can >> move >>>>> back to default by commenting out added lines. But unfortunately it >> isn't >>>>> that simple. >>>>> >>>>> 2017-02-01 8:59 GMT+01:00 Aki Tuomi : >>>>> >>>>>> Are you still trying to authenticate using cram-md5? >>>>>> >>>>>> Aki >>>>>> >>>>>> >>>>>> On 01.02.2017 09:51, Poliman - Serwis wrote: >>>>>>> It still use: >>>>>>> passdb { >>>>>>> driver = passwd-file >>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>> } >>>>>>> >>>>>>> When I delete above and delete "cram-md5" in auth_mechanisms it still >>>> not >>>>>>> working. >>>>>>> >>>>>>> 2017-02-01 8:45 GMT+01:00 Aki Tuomi : >>>>>>> >>>>>>>> You are probably wanting to do >>>>>>>> passdb { >>>>>>>> driver = passwd-file >>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>> } >>>>>>>> >>>>>>>> passdb { >>>>>>>> driver = sql >>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>> } >>>>>>>> >>>>>>>> Why you want to use cram-md5 is beyond me, because using SSL is much >>>>>>>> more safer. >>>>>>>> >>>>>>>> Aki >>>>>>>> >>>>>>>> On 01.02.2017 09:41, Poliman - Serwis wrote: >>>>>>>>> Default it was: "auth_mechanisms = plain login" and I added >>>> cram-md5. >>>>>>>>> After restart all work perfectly. But after I added: >>>>>>>>> driver = passwd-file >>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>> I can't set default lines because I got error. Please tell me which >>>>>> lines >>>>>>>>> should be changed to resolve this issue. Should I remove "login" >> from >>>>>>>>> auth_mechanism ("login" was default setting and I would like to >> move >>>>>> back >>>>>>>>> to default settings)? >>>>>>>>> >>>>>>>>> 2017-02-01 8:36 GMT+01:00 Aki Tuomi : >>>>>>>>> >>>>>>>>>> Because cram-md5 needs the user's password for calculating >>>> responses, >>>>>> it >>>>>>>>>> cannot work with hashed passwords (one-way encrypted). The only >>>>>>>>>> supported password schemes are PLAIN and CRAM-MD5. >>>>>>>>>> >>>>>>>>>> Aki >>>>>>>>>> >>>>>>>>>> On 01.02.2017 09:33, Poliman - Serwis wrote: >>>>>>>>>>> I always restart dovecot after change config. ;) Sure, I >> commented >>>>>> out >>>>>>>>>>> added two lines by me, restarted dovecot and here it is: >>>>>>>>>>> >>>>>>>>>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>>>>>>>>> # OS: Linux 3.13.0-100-generic x86_64 Ubuntu 14.04.5 LTS >>>>>>>>>>> auth_mechanisms = plain login cram-md5 >>>>>>>>>>> listen = *,[::] >>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>>>> mail_plugins = " quota" >>>>>>>>>>> mail_privileged_group = vmail >>>>>>>>>>> passdb { >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> driver = sql >>>>>>>>>>> } >>>>>>>>>>> plugin { >>>>>>>>>>> quota = dict:user::file:/var/vmail/%d/%n/.quotausage >>>>>>>>>>> sieve = /var/vmail/%d/%n/.sieve >>>>>>>>>>> sieve_max_redirects = 25 >>>>>>>>>>> } >>>>>>>>>>> postmaster_address = postmaster at example.com >>>>>>>>>>> protocols = imap pop3 >>>>>>>>>>> service auth { >>>>>>>>>>> unix_listener /var/spool/postfix/private/auth { >>>>>>>>>>> group = postfix >>>>>>>>>>> mode = 0660 >>>>>>>>>>> user = postfix >>>>>>>>>>> } >>>>>>>>>>> unix_listener auth-userdb { >>>>>>>>>>> group = vmail >>>>>>>>>>> mode = 0600 >>>>>>>>>>> user = vmail >>>>>>>>>>> } >>>>>>>>>>> user = root >>>>>>>>>>> } >>>>>>>>>>> service imap-login { >>>>>>>>>>> client_limit = 1000 >>>>>>>>>>> process_limit = 512 >>>>>>>>>>> } >>>>>>>>>>> service lmtp { >>>>>>>>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>>>>>>>>> group = postfix >>>>>>>>>>> mode = 0600 >>>>>>>>>>> user = postfix >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> ssl = required >>>>>>>>>>> ssl_cert = >>>>>>>>>> ssl_cipher_list = >>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: >>>>>>>>>> DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+ >>>>>>>>>> AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128- >>>>>>>>>> SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE- >>>>>>>>>> RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA- >>>>>>>>>> AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE- >>>>>>>>>> RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256: >>>>>>>>>> DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256: >>>>>>>>>> AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- >>>>>>>>>> SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:! >>>>>>>>>> EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:! >>>>>>>>>> EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA >>>>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>>>> ssl_key = >>>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>>>> userdb { >>>>>>>>>>> driver = prefetch >>>>>>>>>>> } >>>>>>>>>>> userdb { >>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>> driver = sql >>>>>>>>>>> } >>>>>>>>>>> protocol imap { >>>>>>>>>>> mail_plugins = quota imap_quota >>>>>>>>>>> } >>>>>>>>>>> protocol pop3 { >>>>>>>>>>> mail_plugins = quota >>>>>>>>>>> pop3_uidl_format = %08Xu%08Xv >>>>>>>>>>> } >>>>>>>>>>> protocol lda { >>>>>>>>>>> mail_plugins = sieve quota >>>>>>>>>>> postmaster_address = webmaster at localhost >>>>>>>>>>> } >>>>>>>>>>> protocol lmtp { >>>>>>>>>>> mail_plugins = quota sieve >>>>>>>>>>> postmaster_address = webmaster at localhost >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2017-02-01 8:27 GMT+01:00 Aki Tuomi : >>>>>>>>>>> >>>>>>>>>>>> On 01.02.2017 08:18, Poliman - Serwis wrote: >>>>>>>>>>>>> This is debug log files in syslog: >>>>>>>>>>>>> Feb 1 07:10:25 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> CONT#0112#011PDAxODg3ODIzMTUwMzgxNzMuMTQ >>>>>>>> 4NTkyOTQyNUB2cHMzNDI0MDEub3ZoL >>>>>>>>>>>> m5ldD4= >>>>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth: Debug: client in: >>>>>>>> CONT >>>>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): Debug: >>>> sql( >>>>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email >> as >>>>>>>> user, >>>>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>>>>>> maildir, >>>>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>>>>>> userdb_mail, >>>>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >> quota, >>>>>>>> 'B') >>>>>>>>>> AS >>>>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >>>> FROM >>>>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email >> = ' >>>>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >>>> server_id = >>>>>>>> '1' >>>>>>>>>>>>> Feb 1 07:10:26 vps342401 dovecot: auth-worker(27069): >> password( >>>>>>>>>>>>> do_not_reply at example.com, 12.173.211.32): Requested CRAM-MD5 >>>>>> scheme, >>>>>>>>>>>> but we >>>>>>>>>>>>> have only CRYPT >>>>>>>>>>>>> Feb 1 07:10:28 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> FAIL#0112#011user=do_not_reply at example.com >>>>>>>>>>>>> Feb 1 07:10:28 vps342401 postfix/smtps/smtpd[27067]: warning: >>>>>>>>>>>>> host23131.internet.3s.com[12.173.211.32]: SASL CRAM-MD5 >>>>>>>> authentication >>>>>>>>>>>>> failed: PDAxODg3ODIzMTUwMzgxNzMuMTQ4NT >>>>>> kyOTQyNUB2cHMzNDI0MDEub3ZoLm5l >>>>>>>>>> dD4= >>>>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27074]: (root) CMD >>>>>>>>>>>>> (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; >>>> do >>>>>>>> echo >>>>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>>>>>> Feb 1 07:11:02 vps342401 CRON[27075]: (root) CMD >>>>>>>>>>>>> (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; >> do >>>>>> echo >>>>>>>>>>>>> `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>>>>>>>> AUTH#0113#011CRAM-MD5#011service=smtp#011nologin# >>>>>>>>>>>> 011lip=173.72.31.7#011rip=12.173.211.32#011secured >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> CONT#0113#011PDE3NDg1NjE4MTgxNTk2OTAuMTQ >>>>>>>> 4NTkyOTQ3MUB2cHMzNDI0MDEub3ZoL >>>>>>>>>>>> m5ldD4= >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth: Debug: client in: >>>>>>>> CONT >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): Debug: >>>> sql( >>>>>>>>>>>>> do_not_reply at example.com,12.173.211.32): query: SELECT email >> as >>>>>>>> user, >>>>>>>>>>>>> password, maildir as userdb_home, CONCAT( maildir_format, ':', >>>>>>>> maildir, >>>>>>>>>>>>> '/', IF(maildir_format='maildir','Maildir',maildir_format)) as >>>>>>>>>>>> userdb_mail, >>>>>>>>>>>>> uid as userdb_uid, gid as userdb_gid, CONCAT('*:storage=', >> quota, >>>>>>>> 'B') >>>>>>>>>> AS >>>>>>>>>>>>> userdb_quota_rule, CONCAT(maildir, '/.sieve') as userdb_sieve >>>> FROM >>>>>>>>>>>>> mail_user WHERE (login = 'do_not_reply at example.com' OR email >> = ' >>>>>>>>>>>>> do_not_reply at example.com') AND `disablesmtp` = 'n' AND >>>> server_id = >>>>>>>> '1' >>>>>>>>>>>>> Feb 1 07:11:11 vps342401 dovecot: auth-worker(27069): >> password( >>>>>>>>>>>>> do_not_reply at example.com,12.173.211.32): Requested CRAM-MD5 >>>>>> scheme, >>>>>>>>>> but >>>>>>>>>>>> we >>>>>>>>>>>>> have only CRYPT >>>>>>>>>>>>> Feb 1 07:11:13 vps342401 dovecot: auth: Debug: client passdb >>>> out: >>>>>>>>>>>>> FAIL#0113#011user=do_not_reply at example.com >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> ##################### >>>>>>>>>>>>> I added in dovecot.conf lines in passdb block: >>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>> and commented out default lines >>>>>>>>>>>>> #args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>> #driver = sql >>>>>>>>>>>>> When I try set again default lines I got above error >>>>>>>>>>>> Can you run doveconf -n with the configuration that causes the >>>> above >>>>>>>>>>>> error? Also it clearly does SQL lookup, so that error is >> happening >>>>>>>> with >>>>>>>>>>>> SQL passdb. You need to remember to restart dovecot between >>>>>>>>>>>> configuration changes. >>>>>>>>>>>> >>>>>>>>>>>> Aki >>>>>>>>>>>> >>>>>>>>>>>>> 2017-01-31 8:08 GMT+01:00 Aki Tuomi : >>>>>>>>>>>>> >>>>>>>>>>>>>> On 31.01.2017 09:06, Poliman - Serwis wrote: >>>>>>>>>>>>>>> I set up cram-md5 using this tutorial >>>>>>>>>>>>>>> https://wiki2.dovecot.org/HowTo/CRAM-MD5 in >>>>>>>>>> /etc/dovecot/dovecot.conf >>>>>>>>>>>> in >>>>>>>>>>>>>>> passdb code block: >>>>>>>>>>>>>>> listen = *,[::] >>>>>>>>>>>>>>> protocols = imap pop3 >>>>>>>>>>>>>>> #auth_mechanisms = plain login cram-md5 >>>>>>>>>>>>>>> auth_mechanisms = cram-md5 plain login >>>>>>>>>>>>>>> #dodana nizej linia >>>>>>>>>>>>>>> ssl = required >>>>>>>>>>>>>>> disable_plaintext_auth = yes >>>>>>>>>>>>>>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>>>>>>>>>>>>>> mail_privileged_group = vmail >>>>>>>>>>>>>>> postmaster_address = postmaster at vps342401.ovh.net >>>>>>>>>>>>>>> ssl_cert = >>>>>>>>>>>>>> ssl_key = >>>>>>>>>>>>>> ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 >>>>>>>>>>>>>>> ssl_cipher_list = >>>>>>>>>>>>>>> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256: >>>>>>>>>>>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[ >>>> image: >>>>>>>>>>>>>>> :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ >>>>>>>>>>>>>>> ssl_prefer_server_ciphers = yes >>>>>>>>>>>>>>> ssl_dh_parameters_length = 2048 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> mail_max_userip_connections = 100 >>>>>>>>>>>>>>> passdb { >>>>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>>>> # driver = sql >>>>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> userdb { >>>>>>>>>>>>>>> driver = prefetch >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> userdb { >>>>>>>>>>>>>>> args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>>>> driver = sql >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> Of course I created cram-md5.pwd file. All mails go out and >>>> come >>>>>>>>>>>> nicely. >>>>>>>>>>>>>>> But after I want to do default settings by commented out >> these >>>>>> two >>>>>>>>>>>> lines: >>>>>>>>>>>>>>> driver = passwd-file >>>>>>>>>>>>>>> args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd >>>>>>>>>>>>>>> and uncomment >>>>>>>>>>>>>>> # args = /etc/dovecot/dovecot-sql.conf >>>>>>>>>>>>>>> # driver = sql >>>>>>>>>>>>>>> I can't send emails - I use Thunderbird - get error "logging >> on >>>>>>>>>> server >>>>>>>>>>>>>>> mail.example.com not work out". Error in logs: >>>>>>>>>>>>>>> dovecot: auth-worker(22698): Error: Auth worker sees >> different >>>>>>>>>>>>>>> passdbs/userdbs than auth server. >>>>>>>>>>>>>>> dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is it possible that hashed password from cram-md5.pwd file >> was >>>>>>>>>> written >>>>>>>>>>>> to >>>>>>>>>>>>>>> database (if yes then where - I have ISPconfig)? I wasn't >>>> change >>>>>>>> any >>>>>>>>>>>>>> userdb >>>>>>>>>>>>>>> {} block and this second userdb block has this same lines >> like >>>>>>>>>> default >>>>>>>>>>>>>>> settings in passdb block. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Try >>>>>>>>>>>>>> >>>>>>>>>>>>>> auth_debug=yes >>>>>>>>>>>>>> auth_verbose=yes >>>>>>>>>>>>>> >>>>>>>>>>>>>> and see if it gives any more reasonable messages. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Aki >>>>>>>>>>>>>> >>> > > From tobias at kirchhofer.net Fri Feb 3 16:39:05 2017 From: tobias at kirchhofer.net (Tobias Kirchhofer) Date: Fri, 03 Feb 2017 17:39:05 +0100 Subject: dsync backup public namespace - how? Message-ID: Hello, We just celebrated the launch of our new mailserver setup (Dovecot and Postfix). Really nice! Now we want to change our quickfix backup with rsync to dsync. For mailder backup of users this was an easy task: dsync -v -f -u "${user}" backup "maildir:${destination}" (local file path destination) We use many public folder with ACL and now we are wondering how we can backup these folder in a similar manner: a) dsync -u doveadm at domain.com backup -n Namespacename "maildir:/var/vmail-backup/backup/public" or b) dsync -v -u doveadm at domain.com -N backup "maildir:${destination} But this is not working. a) does nothing, no error b) Error message: doveadm(doveadm at domain.com): Fatal: -N parameter requires syncing with remote host Any ideas? # dovecot --version 2.2.13 -- Tobias Kirchhofer tobias at kirchhofer.net -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Fri Feb 3 17:00:18 2017 From: tss at iki.fi (Timo Sirainen) Date: Fri, 3 Feb 2017 19:00:18 +0200 Subject: Revision-proof archiving In-Reply-To: References: Message-ID: <079F5472-B184-49F0-BE77-C16BC3C9B97A@iki.fi> On 2 Feb 2017, at 16.05, Paul Atreides wrote: > > Hi, > > I am interested in the "Dovecot Email Archive" solution. Does anyone know > if it is suitable revision-proof archiving? The mails are stored to archive from incoming & outgoing SMTP mail deliveries. User has read-only access to the archive via IMAP. So it's not tracking any folders or flag changes. > Is it available for smaller companies? I tried to contact people at > dovecot.fi but I haven?t gotten any response yet. Where/how? I didn't find any mails from you, although I found an archive-related mail from someone else in info@ mailbox on Jan 13th. That mail was forwarded to sales.. I don't know what happened afterwards. From citrin at citrin.ru Fri Feb 3 17:33:01 2017 From: citrin at citrin.ru (Anton Yuzhaninov) Date: Fri, 3 Feb 2017 12:33:01 -0500 Subject: Crash while reading compressed mbox: Panic: file istream-zlib.c: line 416 (i_stream_zlib_seek): assertion failed: (ret == -1) Message-ID: <5c46438a-7ca7-0ea5-47f2-335ab1f8af2d@citrin.ru> Hello, I use mbox compressed by gzip as read-only folder (using zlib plugin). This setup worked for a while, but doesn't work with latest dovecot version (2.2.27). I have error in logs: Panic: file istream-zlib.c: line 416 (i_stream_zlib_seek): assertion failed: (ret == -1) I can see list of messages in MUA but can't ready body of any message. I've tried to delete indexes, but it doesn't help. Backtrace looks like: (gdb) bt #0 0x0000000011266a4a in thr_kill () from /lib/libc.so.7 #1 0x0000000011266a1b in raise () from /lib/libc.so.7 #2 0x0000000011266989 in abort () from /lib/libc.so.7 #3 0x0000000010ef3850 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 #4 0x0000000010ef5167 in i_internal_fatal_handler (ctx=0x7fffffffe010, format=0x1222cf93 "file %s: line %d (%s): assertion failed: (%s)", args=0x7fffffffdff0) at failures.c:670 #5 0x0000000010ef3e5f in i_panic (format=0x1222cf93 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 #6 0x0000000012229238 in i_stream_zlib_seek (stream=0x118dd300, v_offset=56248, mark=false) at istream-zlib.c:416 #7 0x0000000010f05e48 in i_stream_skip (stream=0x118dd370, count=32489) at istream.c:278 #8 0x0000000010f057ba in i_stream_seek (stream=0x118dd370, v_offset=56248) at istream.c:300 #9 0x0000000010b17702 in istream_raw_mbox_get_body_size (stream=0x118ec870, expected_body_size=36695, body_size_r=0x7fffffffe2a0) at istream-raw-mbox.c:612 #10 0x0000000010b2b0aa in mbox_sync_read_next_mail (sync_ctx=0x7fffffffe418, mail_ctx=0x7fffffffe270) at mbox-sync.c:162 #11 0x0000000010b2964a in mbox_sync_loop (sync_ctx=0x7fffffffe418, mail_ctx=0x7fffffffe270, partial=false) at mbox-sync.c:1057 #12 0x0000000010b290cb in mbox_sync_do (sync_ctx=0x7fffffffe418, flags=MBOX_SYNC_UNDIRTY) at mbox-sync.c:1642 #13 0x0000000010b28999 in mbox_sync_int (mbox=0x118ee040, flags=MBOX_SYNC_UNDIRTY, lock_id=0x7fffffffe640) at mbox-sync.c:1969 #14 0x0000000010b280ea in mbox_sync (mbox=0x118ee040, flags=MBOX_SYNC_UNDIRTY) at mbox-sync.c:2022 #15 0x0000000010b28c9e in mbox_storage_sync_init (box=0x118ee040, flags=65) at mbox-sync.c:2071 #16 0x0000000010acc280 in mailbox_sync_init (box=0x118ee040, flags=65) at mail-storage.c:1740 #17 0x0000000010acb45d in mailbox_sync (box=0x118ee040, flags=65) at mail-storage.c:1788 #18 0x0000000000418cf5 in select_open (ctx=0x1184a1a8, mailbox=0x11816f50 "old/Example/INBOX.gz", readonly=false) at cmd-select.c:303 #19 0x0000000000418937 in cmd_select_full (cmd=0x1184a040, readonly=false) at cmd-select.c:426 #20 0x00000000004190f7 in cmd_select (cmd=0x1184a040) at cmd-select.c:435 #21 0x0000000000423b00 in command_exec (cmd=0x1184a040) at imap-commands.c:181 #22 0x00000000004220c0 in client_command_input (cmd=0x1184a040) at imap-client.c:986 -- Best Regards, Anton Yuzhaninov From mcguire at neurotica.com Fri Feb 3 17:40:35 2017 From: mcguire at neurotica.com (Dave McGuire) Date: Fri, 3 Feb 2017 12:40:35 -0500 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <58945D14.3030201@informatik.uni-freiburg.de> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> <58945D14.3030201@informatik.uni-freiburg.de> Message-ID: <9d6b6755-9279-bede-9691-0d7d0961ea6a@neurotica.com> Same here Sun compiler v5.12 on SPARC. Built cleanly this morning. I'll be upgrading from 2.2.18 this afternoon. :) -Dave On 02/03/2017 05:36 AM, Martin Preen wrote: > Hello, > I don't have problems building 2.2.27 on Solaris 10 > (using Sun Workshop compiler 5.11). > > The configuration is the same as your. > Maybe a compiler/version problem on your system ? > > Regards, > Martin > > Mantas Gegu?is wrote: >> Hello, >> >> I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: >> test-ioloop.c: In function `test_ioloop_pending_io': >> test-ioloop.c:188: error: size of array `type name' is negative >> >> My configuration is like this: >> Install prefix . : /usr/local >> File offsets ... : 64bit >> I/O polling .... : poll >> I/O notifys .... : none >> SSL ............ : yes (OpenSSL) >> GSSAPI ......... : no >> passdbs ........ : static passwd passwd-file shadow pam checkpassword >> dcrypt ..........: yes >> : -bsdauth -sia -ldap -sql -vpopmail >> userdbs ........ : static prefetch passwd passwd-file checkpassword >> : -ldap -sql -vpopmail -nss >> SQL drivers .... : >> : -pgsql -mysql -sqlite -cassandra >> Full text search : squat >> : -lucene -solr >> >> Last version that I have compiled was 2.2.24, version 2.2.25 failed >> with error: >> In file included from guid.c:6: >> sha1.h:80: error: static or type qualifiers in abstract declarator >> >> Is there anyone who can help me? > > ---------------------------------------------------------------------- > Martin Preen, Universit?t Freiburg, Institut f?r Informatik > Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany > > phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de > fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/staff/preen > -- Dave McGuire, AK4HZ New Kensington, PA From jtam.home at gmail.com Fri Feb 3 20:11:16 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 3 Feb 2017 12:11:16 -0800 (PST) Subject: Compiling Dovecot on Solaris 10 In-Reply-To: References: Message-ID: > I don't have problems building 2.2.27 on Solaris 10 > (using Sun Workshop compiler 5.11). ... and I don't have problems with Solaris10/gcc, but I don't have as many options as you do. Joseph Tam From listeem at ksb.id.lv Fri Feb 3 20:22:46 2017 From: listeem at ksb.id.lv (KSB) Date: Fri, 3 Feb 2017 22:22:46 +0200 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <20170203161020.Horde.SmuKZ0VptGAHygCqpZfwjA1@webmail.vu.lt> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> <58945D14.3030201@informatik.uni-freiburg.de> <20170203161020.Horde.SmuKZ0VptGAHygCqpZfwjA1@webmail.vu.lt> Message-ID: <56699bd5-924c-2915-6d61-52602b3bfc8b@ksb.id.lv> On 2017.02.03. 16:10, Mantas Gegu?is wrote: > Hi, > > thank You for a reply, compiling with Solaris Studio 12.5 solved this case. > > > Martin Preen ra??: > >> Hello, >> I don't have problems building 2.2.27 on Solaris 10 >> (using Sun Workshop compiler 5.11). >> >> The configuration is the same as your. >> Maybe a compiler/version problem on your system ? >> >> Regards, >> Martin >> A bit offtopic, but I'm interested what's the point of using so old OS (support still exists though)? -- Kaspars From drbobllc at yahoo.com Fri Feb 3 20:18:57 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Fri, 3 Feb 2017 20:18:57 +0000 (UTC) Subject: please help this newbie get started References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> Message-ID: <1894124593.523567.1486153137926@mail.yahoo.com> Hi, everyone, I'm trying to get email working on a server. Web servers I have some experience with, but this is new for me. On FreeBSD, with dovecot2 (2.2.27), when I try to check email, Thunderbird says: Sending of password for user xxx did not succeed.Mail server xxxresponded: Authentication failed. And on the server, in the mail log, there's a message: dovecot: pop3-login: Disconnected (user disabled) Any idea what I'm doing wrong? I didn't mean to disable any users. Thanks! Bob From mcguire at neurotica.com Fri Feb 3 20:28:22 2017 From: mcguire at neurotica.com (Dave McGuire) Date: Fri, 3 Feb 2017 15:28:22 -0500 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <56699bd5-924c-2915-6d61-52602b3bfc8b@ksb.id.lv> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> <58945D14.3030201@informatik.uni-freiburg.de> <20170203161020.Horde.SmuKZ0VptGAHygCqpZfwjA1@webmail.vu.lt> <56699bd5-924c-2915-6d61-52602b3bfc8b@ksb.id.lv> Message-ID: On 02/03/2017 03:22 PM, KSB wrote: > A bit offtopic, but I'm interested what's the point of using so old OS > (support still exists though)? Short version: It works. Long version: Solaris 10 is still supported; the production systems here are patched up to current as of last week. So while the base release is quite a few years old, the OS installed on these systems is considered current. When support and a current patch stream are no longer available, we will revisit our configuration. For these production systems, there is currently no need for any capability or feature that exists only in "newer" OS releases. When that changes, we will revisit our configuration. Until then, it's rock solid and does everything required of it. There are no problems to be addressed. At least here, we don't fix things that aren't broken. -Dave -- Dave McGuire, AK4HZ New Kensington, PA From drbobllc at yahoo.com Fri Feb 3 23:16:01 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Fri, 3 Feb 2017 23:16:01 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <1894124593.523567.1486153137926@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> Message-ID: <1687514593.624092.1486163761261@mail.yahoo.com> Hi again, I see now it's possible to restricting IMAP/POP3 access, but that shouldn't be enabled. In conf.d/10-auth.conf that's commented out: #!include auth-deny.conf.ext Thanks, Bob From ml+dovecot at valo.at Sat Feb 4 09:37:02 2017 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sat, 04 Feb 2017 10:37:02 +0100 Subject: please help this newbie get started In-Reply-To: <1687514593.624092.1486163761261@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> Message-ID: <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> Am 4. Februar 2017 00:16:01 MEZ schrieb drbobllc at yahoo.com: > Hi again, >I see now it's possible to restricting IMAP/POP3 access, but that >shouldn't be enabled. In conf.d/10-auth.conf that's commented out: > >#!include auth-deny.conf.ext Please provide doveconf -n output together with a description of your problem. The wiki also has a page about troubleshooting a dovecot installation. http://wiki2.dovecot.org/FrontPage?action=show&redirect=StartSeite#Troubleshooting >Thanks, >Bob > > -- Christian Kivalo From drbobllc at yahoo.com Sat Feb 4 14:37:27 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Sat, 4 Feb 2017 14:37:27 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> Message-ID: <80096989.830779.1486219047023@mail.yahoo.com> Thanks for replying. A. configuration: %??? dovecot -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.3-RELEASE amd64? ufs disable_plaintext_auth = no mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY mail_privileged_group = mail namespace inbox { ? inbox = yes ? location = ? mailbox Drafts { ??? special_use = \Drafts ? } ? mailbox Junk { ??? special_use = \Junk ? } ? mailbox Sent { ??? special_use = \Sent ? } ? mailbox "Sent Messages" { ??? special_use = \Sent ? } ? mailbox Trash { ??? special_use = \Trash ? } ? prefix = } passdb { ? args = blocking=no ? driver = passwd } ssl = no userdb { ? args = blocking=no ? driver = passwd ? override_fields = home=/var/empty } B. description of problem: When I try to check email, Thunderbird says: Sending of password for user www did not succeed.Mail server xxxresponded: Authentication failed. And on the server, in the mail log, there's a message: dovecot: pop3-login: Disconnected (user disabled):user=, method=PLAIN And thanks for the link to that Troubleshooting section. I didn't know that was there and will take a look at it now. Bob On Saturday, February 4, 2017 3:37 AM, Christian Kivalo wrote: Am 4. Februar 2017 00:16:01 MEZ schrieb drbobllc at yahoo.com: > Hi again, >I see now it's possible to restricting IMAP/POP3 access, but that >shouldn't be enabled. In conf.d/10-auth.conf that's commented out: > >#!include auth-deny.conf.ext Please provide doveconf -n output together with a description of your problem. The wiki also has a page about troubleshooting a dovecot installation. http://wiki2.dovecot.org/FrontPage?action=show&redirect=StartSeite#Troubleshooting >Thanks, >Bob > >? -- Christian Kivalo From ruga at protonmail.com Sat Feb 4 18:03:56 2017 From: ruga at protonmail.com (Ruga) Date: Sat, 04 Feb 2017 13:03:56 -0500 Subject: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) Message-ID: https://github.com/libressl-portable/portable/issues/278 From baskaranand_n at rediffmail.com Sun Feb 5 01:33:48 2017 From: baskaranand_n at rediffmail.com (baskar anand) Date: 5 Feb 2017 01:33:48 -0000 Subject: =?utf-8?B?RG92ZWNvdCB2Mi4yLjIyIChmZTc4OWQyKSBzdGFydGluZyB1cCBmb3IgaW1hcCwgcG9wMyAoY29yZSBkdW1wcyBkaXNhYmxlZCk=?= Message-ID: <20170205013348.10377.qmail@f4mail-235-241.rediffmail.com> Always I have the following lines in my mail.log (at startup and service restart) dovecot: master: Warning: Killed with signal 15 (by pid=3156 uid=0 code=kill) dovecot: log: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) dovecot: master: Dovecot v2.2.22 (fe789d2) starting up for imap, pop3 (core dumps disabled) dovecot -n output is below: # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-31-generic x86_64 Ubuntu 16.04.1 LTS auth_mechanisms = plain login mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } protocols = " imap pop3" service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl_cert = From drbobllc at yahoo.com Sun Feb 5 05:55:34 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Sun, 5 Feb 2017 05:55:34 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <80096989.830779.1486219047023@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> Message-ID: <1348327411.1107599.1486274134291@mail.yahoo.com> Hi, everyone, 1. As advised in?Debugging Authentication, I turned on auth_debug and auth_debug_passwords, and now in the mail log I get an additional message: dovecot: auth: passwd(xxx,xxx,<40AjQMFHSLVLGJAC>): invalid password field '*' Of course neither the password I tried nor the actual password was '*'. That's what's in /etc/passwd, but dovecot isn't just using that, is it? 2. In the new debug log, I get: dovecot: auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth dovecot: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat dovecot: auth: Debug: auth client connected (pid=3183) dovecot: auth: Debug: client in: AUTH??? 1??? PLAIN??? service=pop3??? session=RFp0lMFHHotLGJAC??? lip=xxx ?? rip=xxx ?? lport=110??? rport=35614 dovecot: auth: Debug: client passdb out: CONT??? 1??? dovecot: auth: Debug: client in: CONT dovecot: auth: Debug: passwd(xxx,xxx,): lookup dovecot: auth: Debug: client passdb out: FAIL??? 1??? user=xxx ?? user_disabled So it's something with passdb? 3. In TestPop3Installation I can't get past the "Check that it's allowing remote logins" section. telnet gives me an error: -ERR [AUTH] Authentication failed. which I expect, because I have telnet turned off. Does that mean I can't use plaintext authentication? Thanks, Bob On Saturday, February 4, 2017 8:37 AM, "drbobllc at yahoo.com" wrote: And thanks for the link to that Troubleshooting section. I didn't know that was there and will take a look at it now. From ml+dovecot at valo.at Sun Feb 5 08:58:06 2017 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sun, 05 Feb 2017 09:58:06 +0100 Subject: please help this newbie get started In-Reply-To: <1348327411.1107599.1486274134291@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> Message-ID: <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> Am 5. Februar 2017 06:55:34 MEZ schrieb drbobllc at yahoo.com: >Hi, everyone, >1. As advised in?Debugging Authentication, I turned on auth_debug and >auth_debug_passwords, and now in the mail log I get an additional >message: >dovecot: auth: passwd(xxx,xxx,<40AjQMFHSLVLGJAC>): invalid password >field '*' >Of course neither the password I tried nor the actual password was '*'. >That's what's in /etc/passwd, but dovecot isn't just using that, is it? The '*' in passwd password field stands for login disabled. See man 5 passwd or http://www.manpages.info/freebsd/passwd.5.html >2. In the new debug log, I get: >dovecot: auth: Debug: Loading modules from directory: >/usr/local/lib/dovecot/auth >dovecot: auth: Debug: Read auth token secret from >/var/run/dovecot/auth-token-secret.dat >dovecot: auth: Debug: auth client connected (pid=3183) >dovecot: auth: Debug: client in: AUTH??? 1??? PLAIN??? service=pop3??? >session=RFp0lMFHHotLGJAC??? lip=xxx ?? rip=xxx ?? lport=110??? >rport=35614 >dovecot: auth: Debug: client passdb out: CONT??? 1??? >dovecot: auth: Debug: client in: CONT >dovecot: auth: Debug: passwd(xxx,xxx,): lookup >dovecot: auth: Debug: client passdb out: FAIL??? 1??? user=xxx ?? >user_disabled > >So it's something with passdb? > >3. In TestPop3Installation I can't get past the "Check that it's >allowing remote logins" section. telnet gives me an error: > >-ERR [AUTH] Authentication failed. > > >which I expect, because I have telnet turned off. Does that mean I >can't use plaintext authentication? This is probably because the users login is disabled. In one oft your provided log outputs you are trying to login as user 'www'. Why? The webserver user has the login normaly disabled. -- Christian Kivalo >Thanks, >Bob > >On Saturday, February 4, 2017 8:37 AM, "drbobllc at yahoo.com" > wrote: > >And thanks for the link to that Troubleshooting section. I didn't know >that was there and will take a look at it now. > From drbobllc at yahoo.com Sun Feb 5 14:14:51 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Sun, 5 Feb 2017 14:14:51 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> Message-ID: <347428067.1212522.1486304091270@mail.yahoo.com> 1. The man page I get is slightly different: %??? man 5 passwd PASSWD(5)???????????????? FreeBSD File Formats Manual??????????????? PASSWD(5) NAME ???? passwd, master.passwd -- format of the password file DESCRIPTION ???? The passwd files are the local source of password information.? They can ???? be used in conjunction with the Hesiod domains `passwd' and `uid', and ???? the NIS maps `passwd.byname', `passwd.byuid', `master.passwd.byname', and ???? `master.passwd.byuid', as controlled by nsswitch.conf(5). ???? For consistency, none of these files should ever be modified manually. ???? The master.passwd file is readable only by root, and consists of newline ???? separated records, one per user, containing ten colon (`:') separated ???? fields.? These fields are as follows: ???? [...] ???? The passwd file is generated from the master.passwd file by pwd_mkdb(8), ???? has the class, change, and expire fields removed, and the password field ???? replaced by a `*' character. ???? [...] ???? In the master.passwd file, the password field is the encrypted form of ???? the password, see crypt(3).? If the password field is empty, no password ???? will be required to gain access to the machine.? This is almost invari- ???? ably a mistake, so authentication components such as PAM can forcibly ???? disallow remote access to passwordless accounts.? Because this file con- ???? tains the encrypted user passwords, it should not be readable by anyone ???? without appropriate privileges. ???? A password of `*' indicates that password authentication is disabled for ???? that account (logins through other forms of authentication, e.g., using ???? ssh(1) keys, will still work).? The field only contains encrypted pass- ???? words, and `*' can never be the result of encrypting a password. Do I need to tell dovecot to check master.passwd instead of passwd? 2. Is my (simple) passdb OK? passdb { ? args = blocking=no ? driver = passwd } I guess it would be easy to try it without the "args" line. 4. Sometimes I log in as www to do web page stuff, so files are owned by www. www has a shell, and a password, and can ssh fine. Thanks for your help! Bob On Sunday, February 5, 2017 2:58 AM, Christian Kivalo wrote: >dovecot: auth: passwd(xxx,xxx,<40AjQMFHSLVLGJAC>): invalid password >field '*' The '*' in passwd password field stands for login disabled. See man 5 passwd or http://www.manpages.info/freebsd/passwd.5.html >-ERR [AUTH] Authentication failed. This is probably because the users login is disabled. In one oft your provided log outputs you are trying to login as user 'www'. Why? The webserver user has the login normaly disabled. From shawn.pringle at gmail.com Sun Feb 5 15:15:19 2017 From: shawn.pringle at gmail.com (Shawn Pringle) Date: Sun, 5 Feb 2017 12:15:19 -0300 Subject: Dovecot v2.2.22 (fe789d2) starting up for imap, pop3 (core dumps disabled) In-Reply-To: <20170205013348.10377.qmail@f4mail-235-241.rediffmail.com> References: <20170205013348.10377.qmail@f4mail-235-241.rediffmail.com> Message-ID: Hello. This is normal. When processes are shutdown they are sent signal TERM by process 0 when the service is stopped. See "man 7 signal" Sometimes the admin might want to restart after changing config files and shutting down with theTERM signal of the dovecot service happens normally with /etc/init.d/dovecot stop. El 04/02/2017 22:34, "baskar anand" escribi?: > Always I have the following lines in my mail.log (at startup and service > restart) > > dovecot: master: Warning: Killed with signal 15 (by pid=3156 uid=0 > code=kill) > dovecot: log: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) > dovecot: master: Dovecot v2.2.22 (fe789d2) starting up for imap, pop3 > (core dumps disabled) > > dovecot -n output is below: > > # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.13 (7b14904) > # OS: Linux 4.4.0-31-generic x86_64 Ubuntu 16.04.1 LTS > auth_mechanisms = plain login > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_privileged_group = mail > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > driver = pam > } > protocols = " imap pop3" > service auth-worker { > user = root > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > } > ssl_cert = > From shawn.pringle at gmail.com Sun Feb 5 15:18:44 2017 From: shawn.pringle at gmail.com (Shawn Pringle) Date: Sun, 5 Feb 2017 12:18:44 -0300 Subject: please help this newbie get started In-Reply-To: <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> Message-ID: If you want things done as a disabled user use su with the -c switch. For example to simulate a cgi request from outside I do: su www-data -c /cgi-bin/getnewimages.cgi the www-data user is the user that runs scripts. .. El 05/02/2017 05:58, "Christian Kivalo" escribi?: > > > Am 5. Februar 2017 06:55:34 MEZ schrieb drbobllc at yahoo.com: > >Hi, everyone, > >1. As advised in Debugging Authentication, I turned on auth_debug and > >auth_debug_passwords, and now in the mail log I get an additional > >message: > >dovecot: auth: passwd(xxx,xxx,<40AjQMFHSLVLGJAC>): invalid password > >field '*' > >Of course neither the password I tried nor the actual password was '*'. > >That's what's in /etc/passwd, but dovecot isn't just using that, is it? > The '*' in passwd password field stands for login disabled. See man 5 > passwd or http://www.manpages.info/freebsd/passwd.5.html > > > >2. In the new debug log, I get: > >dovecot: auth: Debug: Loading modules from directory: > >/usr/local/lib/dovecot/auth > >dovecot: auth: Debug: Read auth token secret from > >/var/run/dovecot/auth-token-secret.dat > >dovecot: auth: Debug: auth client connected (pid=3183) > >dovecot: auth: Debug: client in: AUTH 1 PLAIN service=pop3 > >session=RFp0lMFHHotLGJAC lip=xxx rip=xxx lport=110 > >rport=35614 > >dovecot: auth: Debug: client passdb out: CONT 1 > >dovecot: auth: Debug: client in: CONT > >dovecot: auth: Debug: passwd(xxx,xxx,): lookup > >dovecot: auth: Debug: client passdb out: FAIL 1 user=xxx > >user_disabled > > > >So it's something with passdb? > > > >3. In TestPop3Installation I can't get past the "Check that it's > >allowing remote logins" section. telnet gives me an error: > > > >-ERR [AUTH] Authentication failed. > > > > > >which I expect, because I have telnet turned off. Does that mean I > >can't use plaintext authentication? > This is probably because the users login is disabled. > > In one oft your provided log outputs you are trying to login as user > 'www'. Why? The webserver user has the login normaly disabled. > > -- > Christian Kivalo > > >Thanks, > >Bob > > > >On Saturday, February 4, 2017 8:37 AM, "drbobllc at yahoo.com" > > wrote: > > > >And thanks for the link to that Troubleshooting section. I didn't know > >that was there and will take a look at it now. > > > From drbobllc at yahoo.com Sun Feb 5 16:12:31 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Sun, 5 Feb 2017 16:12:31 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <347428067.1212522.1486304091270@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> <347428067.1212522.1486304091270@mail.yahoo.com> Message-ID: <320424774.1204094.1486311151591@mail.yahoo.com> Hi, everyone, Got through for the first time! In fact the trick was to switch to: passdb { ? driver = passwd-file ? args = path-to-file-with-encrypted-passwords } Thanks for steering me in the right direction. Next I guess is SSL for more security. Bob On Sunday, February 5, 2017 8:14 AM, "drbobllc at yahoo.com" wrote: Do I need to tell dovecot to check master.passwd instead of passwd? 2. Is my (simple) passdb OK? passdb { ? args = blocking=no ? driver = passwd } From ml+dovecot at valo.at Sun Feb 5 16:14:43 2017 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sun, 05 Feb 2017 17:14:43 +0100 Subject: please help this newbie get started In-Reply-To: <347428067.1212522.1486304091270@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> <347428067.1212522.1486304091270@mail.yahoo.com> Message-ID: Am 5. Februar 2017 15:14:51 MEZ schrieb drbobllc at yahoo.com: >1. The man page I get is slightly different: >%??? man 5 passwd >PASSWD(5)???????????????? FreeBSD File Formats Manual??????????????? >PASSWD(5) > >NAME >???? passwd, master.passwd -- format of the password file > >DESCRIPTION >???? The passwd files are the local source of password information.? >They can >???? be used in conjunction with the Hesiod domains `passwd' and `uid', >and >???? the NIS maps `passwd.byname', `passwd.byuid', >`master.passwd.byname', and >???? `master.passwd.byuid', as controlled by nsswitch.conf(5). > >???? For consistency, none of these files should ever be modified >manually. > >???? The master.passwd file is readable only by root, and consists of >newline >???? separated records, one per user, containing ten colon (`:') >separated >???? fields.? These fields are as follows: > >???? [...] > >???? The passwd file is generated from the master.passwd file by >pwd_mkdb(8), >???? has the class, change, and expire fields removed, and the password >field >???? replaced by a `*' character. > >???? [...] >???? In the master.passwd file, the password field is the encrypted >form of >???? the password, see crypt(3).? If the password field is empty, no >password >???? will be required to gain access to the machine.? This is almost >invari- >???? ably a mistake, so authentication components such as PAM can >forcibly >???? disallow remote access to passwordless accounts.? Because this >file con- >???? tains the encrypted user passwords, it should not be readable by >anyone >???? without appropriate privileges. > >???? A password of `*' indicates that password authentication is >disabled for >???? that account (logins through other forms of authentication, e.g., >using >???? ssh(1) keys, will still work).? The field only contains encrypted >pass- >???? words, and `*' can never be the result of encrypting a password. > Do I need to tell dovecot to check master.passwd instead of passwd? You could try using passwd-file as passdb but i have never used anything else than pam and sql. >2. Is my (simple) passdb OK? > >passdb { >? args = blocking=no >? driver = passwd >} >I guess it would be easy to try it without the "args" line. >4. Sometimes I log in as www to do web page stuff, so files are owned >by www. www has a shell, and a password, and can ssh fine. Whats the uid of 'www'? See http://wiki2.dovecot.org/UserIds the part about uids. It could be that the www user has a uid below 500 and therefore login is disabled with the default settings. Christian >Thanks for your help! >Bob > >On Sunday, February 5, 2017 2:58 AM, Christian Kivalo > wrote: > >>dovecot: auth: passwd(xxx,xxx,<40AjQMFHSLVLGJAC>): invalid password >>field '*' > >The '*' in passwd password field stands for login disabled. See man 5 >passwd or http://www.manpages.info/freebsd/passwd.5.html > >>-ERR [AUTH] Authentication failed. > >This is probably because the users login is disabled. > >In one oft your provided log outputs you are trying to login as user >'www'. Why? The webserver user has the login normaly disabled. > From tss at iki.fi Sun Feb 5 17:01:06 2017 From: tss at iki.fi (Timo Sirainen) Date: Sun, 5 Feb 2017 19:01:06 +0200 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> Message-ID: On 2 Feb 2017, at 19.38, Mantas Gegu?is wrote: > > Hello, > > I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: > test-ioloop.c: In function `test_ioloop_pending_io': > test-ioloop.c:188: error: size of array `type name' is negative Change NULL to (void *)NULL in that line. From tss at iki.fi Sun Feb 5 17:49:20 2017 From: tss at iki.fi (Timo Sirainen) Date: Sun, 5 Feb 2017 19:49:20 +0200 Subject: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) In-Reply-To: References: Message-ID: <612EC288-5486-4AFA-9254-E814AE02474F@iki.fi> On 4 Feb 2017, at 20.03, Ruga wrote: > > https://github.com/libressl-portable/portable/issues/278 I've no idea why that would happen. The only idea I had got rejected by someone. From ygrishin-lists at mail2.ca Sun Feb 5 17:53:59 2017 From: ygrishin-lists at mail2.ca (ygrishin-lists at mail2.ca) Date: Sun, 05 Feb 2017 10:53:59 -0700 Subject: Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22. Message-ID: <9df9b8cb09e1c47a6c5f72f5bfa1b696@mail2.ca> Keywords: dovecot, dict, quota, postgre sql, broken pipe, remote disconnected Having Dovecot 2.2.22 (fe789d2) with Postgre SQL 9.5 (9.5.5-0ubuntu0.16.04) as the backend. I do not understand why quota service is not working, not seeing it as a configuration error at least. My quotas are DICT/SQL based. OS: Ubuntu 16.0.4.1 32-bit (Linux XXX 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:36:54 UTC 2017 i686 i686 i686 GNU/Linux) dovecot --build-options: *********************** Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL driver plugins: mysql postgresql sqlite Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap(plugin) nss passwd prefetch passwd-file sql /etc/dovecot/conf.d/10-master.conf: *********************************** service quota-warning { executable = script /etc/dovecot/some-script.sh unix_listener quota-warning { user = Debian-exim mode = 0660 } } service dict { unix_listener dict { mode = 0660 user = Debian-exim group = Debian-exim } } /etc/dovecot/conf.d/90-quota.conf: ********************************** plugin { quota = dict:user_quota::proxy::sqlquota quota_rule2 = Trash:storage=+10%% quota_rule3 = Junk:storage=+10%% quota_grace = 10%% quota_warning = storage=100%% quota-exceeded 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=75%% quota-warning 75 %u } dict { sqlquota = pgsql:/etc/dovecot/dovecot-dict-sql-user.conf } /etc/dovecot/dovecot-dict-sql-user.conf: **************************************** connect = host=A.B.C.D dbname=db user=DDD password=YYY map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } I will not be able to provide full "doveconf -n" output unfortunately. Logging ALL incoming DB queries: ******************************** ... 2017-02-04 12:03:12 MST [29500-10] DDD at db LOG: statement: SELECT password FROM mailbox WHERE local_part = 'YYY' AND domain = 'XXX' AND active ='t' LIMIT 1; 2017-02-04 12:03:12 MST [29501-10] DDD at db LOG: statement: SELECT 111 AS uid, 222 AS gid, '/var/mail/AAA/' || 'BBB' || '/' || 'YYY' AS home, '*:bytes=' || mailbox.quota AS quota_rule FROM mailbox WHERE local_part = 'YYY' AND active ='t' LIMIT 1; And seeing that BOTH dict statements are missing: SELECT and UPDATE. dovecot-lda-erros.log: ********************** Feb 04 14:23:33 lda(testuser at XXX): Error: read(/var/run/dovecot/dict) failed: Remote disconnected Feb 04 14:23:33 lda(testuser at XXX): Error: Internal quota calculation error Feb 04 14:23:33 lda(testuser at XXX): Error: Internal quota calculation error dovecot.log: ************ Feb 04 13:57:06 imap(YYY at XXX): Error: write(/var/run/dovecot/dict) failed: Broken pipe Feb 04 13:57:06 imap(YYY at XXX): Error: write(/var/run/dovecot/dict) failed: Broken pipe ... Feb 04 13:57:07 imap(YYY at XXX): Error: write(/var/run/dovecot/dict) failed: Broken pipe ... Feb 04 13:57:10 imap(YYY at XXX): Error: write(/var/run/dovecot/dict) failed: Broken pipe I have tried stopping dovecot, removing /var/run/dovecot/dict manually and starting dovecot. This has not changed the behavior. dovecot-debug.log: ****************** Feb 04 13:18:12 lda(YYY at XXX): Error: read(/var/run/dovecot/dict) failed: Remote disconnected Feb 04 13:18:12 lda(YYY at XXX): Error: dict quota: Quota update failed, it's now desynced Feb 04 13:57:07 lda(testuser at XXX): Error: write(/var/run/dovecot/dict) failed: Broken pipe Feb 04 13:57:07 lda(testuser at XXX): Error: Internal quota calculation error Feb 04 13:57:07 lda(testuser at XXX): Error: Internal quota calculation error Running 'user_query' manually on the server: ******************************************** db=> SELECT 111 AS uid, 222 AS gid, '/var/mail/AAA/' || 'XXX' || '/' || 'testuser' AS home, '*:bytes=' || mailbox.quota AS quota_rule FROM mailbox WHERE local_part = 'testuser' AND active ='t' LIMIT 1; uid | gid | home | quota_rule -----+-----+-----------------------------------+------------------ 111 | 222 | /var/mail/AAA/BBB/testuser | *:bytes=10485760 (1 row) The identity had been granted 'all' privilege for 'quota2' table: ***************************************************************** db=> SELECT table_catalog, table_schema, table_name, privilege_type db-> FROM information_schema.table_privileges db-> WHERE grantee='DDD'; table_catalog | table_schema | table_name | privilege_type ---------------+--------------+------------+---------------- mail | public | quota2 | INSERT mail | public | quota2 | SELECT mail | public | quota2 | UPDATE mail | public | quota2 | DELETE mail | public | quota2 | TRUNCATE mail | public | quota2 | REFERENCES mail | public | quota2 | TRIGGER mail | public | mailbox | SELECT mail | public | quota | INSERT mail | public | quota | SELECT mail | public | quota | UPDATE mail | public | quota | DELETE mail | public | quota | TRUNCATE mail | public | quota | REFERENCES mail | public | quota | TRIGGER (15 rows) Rows are obviously not getting inserted: *************************************** db=> select * from quota2; username | bytes | messages ----------+-------+---------- (0 rows) I have checked logs for any crash-related messages, there's none. Also, I have tried to deploy 2.27-release version on the server to see whether the behavior is gone. I set-up a testing environment which consists of ubuntu-16.0.4-i386 server, installed build-essential package and unpacked the dovecot-2.2.27.tar.gz there. Getting exactly same issue as Ricardo Machini was having in his post to "v2.2.27 released" thread on Sat Dec 3 21:20:05 UTC 2016: **************************************** checking that generated files are newer than configure... done configure: error: conditional "SSL_VERSION_GE_102" was never defined. Usually this means the macro was only invoked conditionally. All I ran was ./configure 'openssl' package is installed as well. From tss at iki.fi Sun Feb 5 17:58:44 2017 From: tss at iki.fi (Timo Sirainen) Date: Sun, 5 Feb 2017 19:58:44 +0200 Subject: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) In-Reply-To: <612EC288-5486-4AFA-9254-E814AE02474F@iki.fi> References: <612EC288-5486-4AFA-9254-E814AE02474F@iki.fi> Message-ID: <07017C75-6348-48CC-BE01-1D99061997A5@iki.fi> On 5 Feb 2017, at 19.49, Timo Sirainen wrote: > > On 4 Feb 2017, at 20.03, Ruga wrote: >> >> https://github.com/libressl-portable/portable/issues/278 > > I've no idea why that would happen. The only idea I had got rejected by someone. Oh, that's with OSX. I think that's the reason. Nothing to do with libressl. I'll see if I can reproduce it, as soon as I can figure out how to get a newer OpenSSL installed to my OSX.. From tss at iki.fi Sun Feb 5 18:19:32 2017 From: tss at iki.fi (Timo Sirainen) Date: Sun, 5 Feb 2017 20:19:32 +0200 Subject: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) In-Reply-To: <07017C75-6348-48CC-BE01-1D99061997A5@iki.fi> References: <612EC288-5486-4AFA-9254-E814AE02474F@iki.fi> <07017C75-6348-48CC-BE01-1D99061997A5@iki.fi> Message-ID: <8AFA5F80-B117-4F9F-A95A-94EC13BA6617@iki.fi> On 5 Feb 2017, at 19.58, Timo Sirainen wrote: > > On 5 Feb 2017, at 19.49, Timo Sirainen wrote: >> >> On 4 Feb 2017, at 20.03, Ruga wrote: >>> >>> https://github.com/libressl-portable/portable/issues/278 >> >> I've no idea why that would happen. The only idea I had got rejected by someone. > > Oh, that's with OSX. I think that's the reason. Nothing to do with libressl. I'll see if I can reproduce it, as soon as I can figure out how to get a newer OpenSSL installed to my OSX.. Attached patch seems to work. -------------- next part -------------- A non-text attachment was scrubbed... Name: osx-dcrypt-fix.diff Type: application/octet-stream Size: 2636 bytes Desc: not available URL: From aki.tuomi at dovecot.fi Sun Feb 5 18:21:54 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sun, 05 Feb 2017 20:21:54 +0200 Subject: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) Message-ID: jos ldflags laittaa -static vivun niin se preferoi .a filedn linkkausta. ---Aki TuomiDovecot oy -------- Original message --------From: Timo Sirainen Date: 05/02/2017 20:19 (GMT+02:00) To: Dovecot Mailing List Subject: Re: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) On 5 Feb 2017, at 19.58, Timo Sirainen wrote: > > On 5 Feb 2017, at 19.49, Timo Sirainen wrote: >> >> On 4 Feb 2017, at 20.03, Ruga wrote: >>> >>> https://github.com/libressl-portable/portable/issues/278 >> >> I've no idea why that would happen. The only idea I had got rejected by someone. > > Oh, that's with OSX. I think that's the reason. Nothing to do with libressl. I'll see if I can reproduce it, as soon as I can figure out how to get a newer OpenSSL installed to my OSX.. Attached patch seems to work. From aki.tuomi at dovecot.fi Sun Feb 5 18:24:13 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sun, 05 Feb 2017 20:24:13 +0200 Subject: =?US-ASCII?Q?Re:_Panic_error_from_dovecot_2.2.27_using?= =?US-ASCII?Q?_libressl_2.4.5=0D_=09(cross-posting_at_GitHub)?= Message-ID: Sorry for finnish. I think it might also work with libtool flag -static in Makefile.am LDADD flags.---Aki TuomiDovecot oy -------- Original message --------From: Aki Tuomi Date: 05/02/2017 20:21 (GMT+02:00) To: Dovecot Mailing List Subject: Re: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) jos ldflags laittaa -static vivun niin se preferoi .a filedn linkkausta. ---Aki TuomiDovecot oy -------- Original message --------From: Timo Sirainen Date: 05/02/2017? 20:19? (GMT+02:00) To: Dovecot Mailing List Subject: Re: Panic error from dovecot 2.2.27 using libressl 2.4.5 (cross-posting at GitHub) On 5 Feb 2017, at 19.58, Timo Sirainen wrote: > > On 5 Feb 2017, at 19.49, Timo Sirainen wrote: >> >> On 4 Feb 2017, at 20.03, Ruga wrote: >>> >>> https://github.com/libressl-portable/portable/issues/278 >> >> I've no idea why that would happen. The only idea I had got rejected by someone. > > Oh, that's with OSX. I think that's the reason. Nothing to do with libressl. I'll see if I can reproduce it, as soon as I can figure out how to get a newer OpenSSL installed to my OSX.. Attached patch seems to work. From nicolas at andrillon.net Sun Feb 5 18:50:31 2017 From: nicolas at andrillon.net (nicolas at andrillon.net) Date: Sun, 05 Feb 2017 18:50:31 +0000 Subject: dovecot/auth CPU spikes Message-ID: Hi All, I have recently moved by webmail server from a VPS to a hosted dedicated server running Ubuntu 16.04. Everything is fine except that login is particularly and consistently long (around 4-5 seconds). I have noticed that the process dovecot/auth seems to eat all of the resources of one of the cores available on the host during login. The authentication backend is a postgres database which is running absolutely fine. I have been scavenging on the dovecot mailing list for some time but I have not been able to find a solution to my problem so decided to send this bottle to the sea. Here is my config: $ sudo dovecot -n # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 3.14.32-xxxx-grs-ipv6-64 x86_64 Ubuntu 16.04.1 LTS ext4 auth_cache_size = 10 M auth_mechanisms = plain login default_internal_user = vmail first_valid_uid = 0 mail_location = maildir:/home/data/vmail/%d/%n mail_plugins = " fts fts_solr" mail_privileged_group = vmail maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = Archive } mailbox Drafts { auto = subscribe special_use = Drafts } mailbox Junk { auto = subscribe special_use = Junk } mailbox Sent { auto = subscribe special_use = Sent } mailbox "Sent Messages" { special_use = Sent } mailbox Trash { auto = subscribe special_use = Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = pipe antispam_mail_notspam = learn_ham antispam_mail_sendmail = /usr/bin/rspamc antispam_mail_sendmail_args = -h;localhost:11334;-P;q1 antispam_mail_spam = learn_spam antispam_spam = Junk antispam_trash = Trash fts = solr fts_solr = break-imap-search url=http://localhost:8080/solr/ sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve.d/ } postmaster_address = postmaster at domain.net protocols = imap lmtp sieve pop3 service auth-worker { unix_listener auth-worker { user = vmail } user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } user = vmail } service imap-login { inet_listener imap { port = 0 } service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } user = vmail } service pop3-login { inet_listener pop3 { port = 0 } } ssl = required ssl_cert = From nicolas at andrillon.net Sun Feb 5 19:06:40 2017 From: nicolas at andrillon.net (nicolas at andrillon.net) Date: Sun, 05 Feb 2017 19:06:40 +0000 Subject: dovecot/auth CPU spikes In-Reply-To: References: Message-ID: <54fafdd7d1c5b0aa926382c8c6266f6a@andrillon.net> Full dovecot -n output ================= # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 3.14.32-xxxx-grs-ipv6-64 x86_64 Ubuntu 16.04.1 LTS ext4 auth_cache_size = 10 M auth_mechanisms = plain login default_internal_user = vmail first_valid_uid = 0 mail_location = maildir:/home/data/vmail/%d/%n mail_plugins = " fts fts_solr" mail_privileged_group = vmail maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = Archive } mailbox Drafts { auto = subscribe special_use = Drafts } mailbox Junk { auto = subscribe special_use = Junk } mailbox Sent { auto = subscribe special_use = Sent } mailbox "Sent Messages" { special_use = Sent } mailbox Trash { auto = subscribe special_use = Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = pipe antispam_mail_notspam = learn_ham antispam_mail_sendmail = /usr/bin/rspamc antispam_mail_sendmail_args = -h;localhost:11334;-P;q1 antispam_mail_spam = learn_spam antispam_spam = Junk antispam_trash = Trash fts = solr fts_solr = break-imap-search url=http://localhost:8080/solr/ sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve.d/ } postmaster_address = postmaster at domain.net protocols = imap lmtp sieve pop3 service auth-worker { unix_listener auth-worker { user = vmail } user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } user = vmail } service imap-login { inet_listener imap { port = 0 } service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } user = vmail } service pop3-login { inet_listener pop3 { port = 0 } } ssl = required ssl_cert = From nicolas at andrillon.net Sun Feb 5 19:09:58 2017 From: nicolas at andrillon.net (nicolas at andrillon.net) Date: Sun, 05 Feb 2017 19:09:58 +0000 Subject: dovecot/auth CPU spikes In-Reply-To: References: Message-ID: And output from strace, nothing i can make sense of really... 10:38:46.859514 epoll_wait(16, [{EPOLLIN, {u32=1696469520, u64=15038376972816}}], 17, -1) = 1 10:38:47.768364 accept(7, {sa_family=AF_LOCAL, NULL}, [2]) = 23 10:38:47.768687 getsockname(23, {sa_family=AF_LOCAL, sun_path="/var/run/dovecot/login/log255r"}, [31]) = 0 10:38:47.768945 fcntl(23, F_GETFL) = 0x2 (flags O_RDWR) 10:38:47.769132 fcntl(23, F_SETFL, O_RDWR|O_NONBLOCK) = 0 10:38:47.769316 write(5, "372f53453", 12) = 12 10:38:47.769529 read(4, "nBW211316333t371341203251206317b367220", 16) = 16 10:38:47.769747 fstat(23, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0 10:38:47.769979 lseek(23, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 10:38:47.770129 getsockname(23, {sa_family=AF_LOCAL, sun_path="/var/run/dovecot/login/log"e"}, [31]) = 0 10:38:47.770320 epoll_ctl(16, EPOLL_CTL_ADD, 23, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=1696840896, u64=15038377344192}}) = 0 10:38:47.770533 write(23, "VERSIONt1t1nMECHtPLAINtplaintext"..., 118) = 118 10:38:47.770735 epoll_wait(16, [{EPOLLIN, {u32=1696840896, u64=15038377344192}}], 17, -1) = 1 10:38:47.770927 read(23, "VERSIONt1t1nCPIDt10995n", 8192) = 23 10:38:47.771109 epoll_wait(16, [{EPOLLIN, {u32=1696840896, u64=15038377344192}}], 17, -1) = 1 10:38:47.916004 read(23, "AUTHt1tPLAINtservice=imaptsecure"..., 8169) = 145 10:38:47.916428 writev(15, [{"PENALTY-GETt2001:41d0:a::", 25}, {"n", 1}], 2) = 26 10:38:47.916851 epoll_wait(16, [{EPOLLIN, {u32=1696458048, u64=15038376961344}}], 17, 5000) = 1 10:38:47.917177 read(15, "0 0n", 332) = 4 10:38:47.917478 writev(23, [{"CONTt1t", 7}, {"n", 1}], 2) = 8 10:38:47.917835 read(15, 0xdad65237f68, 328) = -1 EAGAIN (Resource temporarily unavailable) 10:38:47.918218 epoll_wait(16, [{EPOLLIN, {u32=1696840896, u64=15038377344192}}], 17, 149998) = 1 10:38:47.919198 read(23, "CONTt1tAG5pY29sYXNAYW5kcmlsbG9uL"..., 8024) = 52 10:38:49.558718 writev(23, [{"OKt1tuser=address at domain.nett", 32}, {"n", 1}], 2) = 33 10:38:49.558978 epoll_wait(16, [{EPOLLIN, {u32=1696470560, u64=15038376973856}}], 17, 150000) = 1 From dovelist at tesla.demon.nl Sun Feb 5 19:53:54 2017 From: dovelist at tesla.demon.nl (dovelist) Date: Sun, 05 Feb 2017 20:53:54 +0100 Subject: Managesieve cannot access script store Message-ID: Hi, I am trying to get sieve working on a new OpenSuse leap 42.2 install. On my 'old' OpenSuse 13.2 machine it worked fine. The problem is that Managesieve can't access the script store and won't let me create any script. It says permission denied on ~/sieve directory. See log below. I 've activated debug logging, but that doesn't give any clues to me. Also, I've set the directory accessible to all, but Managesieve still complains. > cd ~ > ls -l drwx------ 1 rogier users 8340 5 feb 16:54 Maildir drwxrwxrwx 1 rogier users 24 5 feb 18:38 sieve To rule out client issues (kmail) I tested also with Manual TLS Login as described in: http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Troubleshooting Same result. I am puzzled. I can't find anything wrong in the dovecot configuration. The output of dovecot -n is shown below. Hope someone has a solution. A lot of mail is waiting to get sorted... Best Regards, Rogier The log: feb 05 20:22:18 p150 dovecot[12120]: managesieve-login: Login: user=, method=PLAIN, rip=192.168.0.18, lip=192.168.0.20, mpid=12135, TLS, session= feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Effective uid=1000, gid=100, home=/home/rogier feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: maildir++: root=/home/rogier/Maildir, index=, indexpvt=, control=, inbox=/home/rogier/Maildir, alt= feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: Pigeonhole version 0.4.15 (97b3da0) initializing feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using active Sieve script path: /home/rogier/.dovecot.sieve feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using script storage path: /home/rogier/sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using permissions from /home/rogier/sieve/: mode=0777 gid=-1 feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: sync: Synchronization active feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Error: sieve: file storage: Failed to list scripts: opendir(/home/rogier/sieve) failed: Permission denied Output of dovecot -n: # 2.2.25 (7be1766): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 4.4.36-8-default x86_64 openSUSE 42.2 (x86_64) auth_username_format = %Ln base_dir = /var/run/dovecot/ mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = file:~/sieve/;active=~/.dovecot.sieve sieve_trace_debug = yes } protocols = imap lmtp sieve service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> Message-ID: <03798325.20170206083620@maelenn.org> Hi Aki, I do not have any error message but (on both server): doveadm replicator status '*' doveadm(root): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Connection refused Thx Le vendredi 3 f?vrier 2017 ? 17:09:52, vous ?criviez : > Please keep responses in list. rm -f > /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir. > On 2017-02-03 17:00, Thierry wrote: >> Hi, >> >> I have removed the '<' : >> >> ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem >> >> But now: >> >> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >> >> Any idea ? >> >> Thx >> >>> Yes. The ssl_client_ca_file is not actually expecting <, just file name. >>> Aki >> >>> On 2017-02-03 15:13, Thierry wrote: >>>> Hi, >>>> >>>> I have made change: >>>> >>>> ssl_protocols = !SSLv2 !SSLv3 >>>> ssl = required >>>> verbose_ssl = no >>>> ssl_key = >>> ssl_cert = >>> ssl_client_ca_file = >>> >>>> >>>> # Create a listener for doveadm-server >>>> service doveadm { >>>> user = vmail >>>> inet_listener { >>>> port = 12345 >>>> ssl= yes >>>> } >>>> } >>>> >>>> and doveadm_port = 12345 // mail_replica = tcps:server2.domain.ltd # use doveadm_port >>>> >>>> And now: >>>> >>>> Feb 03 14:11:16 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long >>>> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>>> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>>> >>>> Thx for your support >>>> >>>> >>>> >>>> >>>> Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : >>>> >>>>> Hello, >>>>> On 02/03/2017 08:51 AM, Thierry wrote: >>>>>> Hello, >>>>>> >>>>>> Still working with my dsync pb. >>>>>> I have done a clone (vmware) of my email server. >>>>>> Today I have two strictly identical emails servers (server1 >>>>>> (main) and server2 (bck) (except IP, hostname and mail_replica). >>>>>> >>>>>> The ssl config on my both server: >>>>>> >>>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>>> ssl = required >>>>>> verbose_ssl = no >>>>>> ssl_key = >>>>> ssl_cert = >>>>> ssl_ca = >>>> I think it should be ssl_client_ca_file = >>>>> >>>>> This config is working for my email client and my email web >>>>>> interface ... >>>>>> >>>>>> Are they on the right order ? >>>>>> >>>>>> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >>>>>> >>>>>> There is trafic on my iptables rules on my both servers: >>>>>> >>>>>> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >>>>>> >>>>>> >>>>>> >>>>>> My error message from server1 (main server): >>>>>> >>>>>> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>> >>>>>> No logs from server2 >>>>>> >>>>>> Any ideas ? >>>>>> >>>>>> Thx for your support >>>>>> >>>>>> >>>> -- Cordialement, Thierry e-mail : lenaigst at maelenn.org From drbobllc at yahoo.com Mon Feb 6 10:37:23 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Mon, 6 Feb 2017 10:37:23 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <320424774.1204094.1486311151591@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> <347428067.1212522.1486304091270@mail.yahoo.com> <320424774.1204094.1486311151591@mail.yahoo.com> Message-ID: <1783607102.1579943.1486377443606@mail.yahoo.com> Hi again, everyone, Adding SSL seemed to go smoothly, I can check my email now with Thunderbird with "connection security" set to STARTTLS. My next issue is receiving emails. Can you help me with that, too? It works to use "mail" on the command line to send email from one account to another. But email from this yahoo account never appears. How should I start to try to figure this out? Thanks! Bob On Sunday, February 5, 2017 10:12 AM, "drbobllc at yahoo.com" wrote: Next I guess is SSL for more security. From mail at tomsommer.dk Mon Feb 6 12:59:23 2017 From: mail at tomsommer.dk (Tom Sommer) Date: Mon, 06 Feb 2017 13:59:23 +0100 Subject: Changes to userdb not picked up Message-ID: <5f233d2bfff9cd222192ea56d680e2d7@tomsommer.dk> I have my quota limits stored in userdb and auth_cache enabled (default settings). When I change the quota limit the old limit is still cached for the user for another hour. Is there any way to prevent this from happening? Thanks -- Tom From bind at enas.net Mon Feb 6 13:29:08 2017 From: bind at enas.net (Urban Loesch) Date: Mon, 6 Feb 2017 14:29:08 +0100 Subject: Changes to userdb not picked up In-Reply-To: <5f233d2bfff9cd222192ea56d680e2d7@tomsommer.dk> References: <5f233d2bfff9cd222192ea56d680e2d7@tomsommer.dk> Message-ID: You can flush the cache with: "doveadm auth cache flush $USER" Regards Urban Am 06.02.2017 um 13:59 schrieb Tom Sommer: > I have my quota limits stored in userdb and auth_cache enabled (default settings). > > When I change the quota limit the old limit is still cached for the user for another hour. Is there any way to prevent this from happening? > > Thanks > From ueberall at projektzentrisch.de Mon Feb 6 14:05:24 2017 From: ueberall at projektzentrisch.de (Markus Ueberall) Date: Mon, 6 Feb 2017 15:05:24 +0100 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: <03798325.20170206083620@maelenn.org> References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> <03798325.20170206083620@maelenn.org> Message-ID: Dear Thierry, - Have you checked that port 12345 as specified below is open/forwarded and actually /used/ by dovecot (e.g., use "netstat -tulpn|grep dovecot")? - Did you retrace your steps and have you verified that synchronisation works with ssl disabled? - Did you verify your certificate files (e.g., "openssl verify -verbose -CAfile /etc/ssl/certs/GandiCA2.pem /etc/ssl/certs/key.crt")? Personally, I prefer to use a single, specialised tool to manage certificates/encryption (which in my case is stunnel); all other programs are set up using (link-)local ip addresses only. If everything but encryption works with your setup, this might be a possible "workaround". (Apart from that, stunnel debug mode is very detailed and can help you to rule out problems with the certificates/connections between two nodes.) And once the latter works but the dovecot setup below still does not, it would also point to a problem with certificate handling by dovecot (could be library related). KR, Markus Am 06.02.2017 um 07:36 schrieb Thierry: > Hi Aki, > > I do not have any error message but (on both server): > > doveadm replicator status '*' > doveadm(root): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Connection refused > > Thx > > > Le vendredi 3 f?vrier 2017 ? 17:09:52, vous ?criviez : > >> Please keep responses in list. rm -f >> /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir. > >> On 2017-02-03 17:00, Thierry wrote: >>> Hi, >>> >>> I have removed the '<' : >>> >>> ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem >>> >>> But now: >>> >>> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>> >>> Any idea ? >>> >>> Thx >>> >>>> Yes. The ssl_client_ca_file is not actually expecting <, just file name. >>>> Aki >>>> On 2017-02-03 15:13, Thierry wrote: >>>>> Hi, >>>>> >>>>> I have made change: >>>>> >>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>> ssl = required >>>>> verbose_ssl = no >>>>> ssl_key = >>>> ssl_cert = >>>> ssl_client_ca_file = >>>> >>>>> >>>>> # Create a listener for doveadm-server >>>>> service doveadm { >>>>> user = vmail >>>>> inet_listener { >>>>> port = 12345 >>>>> ssl= yes >>>>> } >>>>> } >>>>> >>>>> and doveadm_port = 12345 // mail_replica = tcps:server2.domain.ltd # use doveadm_port >>>>> >>>>> And now: >>>>> >>>>> Feb 03 14:11:16 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long >>>>> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>>>> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>>>> >>>>> Thx for your support >>>>> >>>>> >>>>> >>>>> >>>>> Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : >>>>> >>>>>> Hello, >>>>>> On 02/03/2017 08:51 AM, Thierry wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Still working with my dsync pb. >>>>>>> I have done a clone (vmware) of my email server. >>>>>>> Today I have two strictly identical emails servers (server1 >>>>>>> (main) and server2 (bck) (except IP, hostname and mail_replica). >>>>>>> >>>>>>> The ssl config on my both server: >>>>>>> >>>>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>>>> ssl = required >>>>>>> verbose_ssl = no >>>>>>> ssl_key = >>>>>> ssl_cert = >>>>>> ssl_ca = >>>>> I think it should be ssl_client_ca_file = >>>>>> >>>>>> This config is working for my email client and my email web >>>>>>> interface ... >>>>>>> >>>>>>> Are they on the right order ? >>>>>>> >>>>>>> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >>>>>>> >>>>>>> There is trafic on my iptables rules on my both servers: >>>>>>> >>>>>>> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >>>>>>> >>>>>>> >>>>>>> >>>>>>> My error message from server1 (main server): >>>>>>> >>>>>>> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>> >>>>>>> No logs from server2 >>>>>>> >>>>>>> Any ideas ? >>>>>>> >>>>>>> Thx for your support >>>>>>> >>>>>>> > > From mantas.geguzis at ittc.vu.lt Mon Feb 6 15:06:07 2017 From: mantas.geguzis at ittc.vu.lt (Mantas =?utf-8?b?R2VndcW+aXM=?=) Date: Mon, 06 Feb 2017 17:06:07 +0200 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> Message-ID: <20170206170607.Horde.jMqNd4VPiFheMaLP5MnpTw1@webmail.vu.lt> Hello, thank You, this solution worked too. But had to do same thing for those files: test-http-client-errors.c:388 test-http-client-errors.c:484 test-http-client-errors.c:556 test-http-client-errors.c:636 test-http-server-errors.c:594 main.c:63 director.c:1445 director.c:1448 imap-client.c:253 director.c:1445 director.c:1448 mail-stats.c:56 Is this an old compiler issue or something else? Timo Sirainen ra??: > On 2 Feb 2017, at 19.38, Mantas Gegu?is wrote: >> >> Hello, >> >> I am tying to compile Dovecot 2.2.27 on Solaris 10, and I get this error: >> test-ioloop.c: In function `test_ioloop_pending_io': >> test-ioloop.c:188: error: size of array `type name' is negative > > Change NULL to (void *)NULL in that line. -- Pagarbiai Mantas Gegu?is VU Informacini? technologij? taikymo centras tel. 8 5 236 6208 From tss at iki.fi Mon Feb 6 15:24:57 2017 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2017 17:24:57 +0200 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <20170206170607.Horde.jMqNd4VPiFheMaLP5MnpTw1@webmail.vu.lt> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> <20170206170607.Horde.jMqNd4VPiFheMaLP5MnpTw1@webmail.vu.lt> Message-ID: <00C6AC67-BFC7-4B22-AD1F-8E74E5DA8E9A@iki.fi> On 6 Feb 2017, at 17.06, Mantas Gegu?is wrote: > > Hello, > > thank You, this solution worked too. But had to do same thing for those files: > > test-http-client-errors.c:388 > test-http-client-errors.c:484 > test-http-client-errors.c:556 > test-http-client-errors.c:636 > test-http-server-errors.c:594 > main.c:63 > director.c:1445 > director.c:1448 > imap-client.c:253 > director.c:1445 > director.c:1448 > mail-stats.c:56 > > Is this an old compiler issue or something else? Try this: https://github.com/dovecot/core/commit/dd6043c05e32a8e8db1233ed711a2c74d1477a89 From g.dimakopoulos at sophimail.com Mon Feb 6 16:08:25 2017 From: g.dimakopoulos at sophimail.com (George Dimakopoulos) Date: Mon, 6 Feb 2017 18:08:25 +0200 Subject: Dsync migration with shared folders Message-ID: <4478BEF3-14E8-42E6-8BC2-30EB9B647431@sophimail.com> Hi everyone, We discovered that the migration path from a VPS to our 2XHP replication server environment with shared folders should be: 1) Rsync all domains from the VPS to serverA. Then rsync all domains from ServerA to ServerB. ServerA & ServerB are in a identical state. 2) Create an empty directory for the shared indexes 3) Start dovecot on both machines. If we did not sync both machines then dsync could not replicate the domains that contained shared folders. What is the right migration process for domains with shared folders ? Regards, George dovecot -n ------------------------- # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 11.0-RELEASE-p2 amd64 zfs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 2560 default_process_limit = 512 dict { acl = mysql:/usr/local/etc/dovecot/dovecot-shared-sql.conf.ext quota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_password = # hidden, use -P to show it doveadm_port = 12345 log_path = /var/log/dovecot.log mail_debug = yes mail_home = /usr/local/vhosts/mail/%d/%n mail_location = maildir:/usr/local/vhosts/mail/%d/%n:LAYOUT=fs mail_max_userip_connections = 70 mail_plugins = quota acl notify replication mail_privileged_group = vmail mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace { inbox = no list = children location = maildir:/usr/local/vhosts/mail/%%d/%%n:LAYOUT=fs:INDEX=/usr/local/vhosts/indexes/%d/%n/shared/%%u:INDEXPVT=/usr/local/vhosts/indexes/%d/%n/shared/%%u prefix = shared/%%d/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:mx1 quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = /usr/local/vhosts/mail/%d/%n/.dovecot.sieve sieve_before = /usr/local/vhosts/sieve/before.d/ sieve_dir = /usr/local/vhosts/mail/%d/%n sieve_global_dir = /usr/local/vhosts/sieve/%d sieve_global_path = /usr/local/vhosts/sieve/%d/default.sieve } protocols = imap lmtp sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service config { unix_listener config { user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service replicator { unix_listener replicator-doveadm { mode = 0666 } } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota acl notify replication sieve } protocol lda { mail_plugins = quota acl notify replication sieve acl postmaster_address = root } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = quota acl notify replication imap_quota imap_acl } ssl_cert = ssl_key = From mantas.geguzis at ittc.vu.lt Mon Feb 6 16:12:33 2017 From: mantas.geguzis at ittc.vu.lt (Mantas =?utf-8?b?R2VndcW+aXM=?=) Date: Mon, 06 Feb 2017 18:12:33 +0200 Subject: Compiling Dovecot on Solaris 10 In-Reply-To: <00C6AC67-BFC7-4B22-AD1F-8E74E5DA8E9A@iki.fi> References: <20170202193840.Horde.agibFlWIEHQTkiJcu7aj2w1@webmail.vu.lt> <20170206170607.Horde.jMqNd4VPiFheMaLP5MnpTw1@webmail.vu.lt> <00C6AC67-BFC7-4B22-AD1F-8E74E5DA8E9A@iki.fi> Message-ID: <20170206181233.Horde.EUVqOkufGrVasE-oKXN3hQ1@webmail.vu.lt> Timo Sirainen ra??: > On 6 Feb 2017, at 17.06, Mantas Gegu?is wrote: > >> Hello, >> >> thank You, this solution worked too. But had to do same thing for those >> files: >> >> test-http-client-errors.c:388 >> test-http-client-errors.c:484 >> test-http-client-errors.c:556 >> test-http-client-errors.c:636 >> test-http-server-errors.c:594 >> main.c:63 >> director.c:1445 >> director.c:1448 >> imap-client.c:253 >> director.c:1445 >> director.c:1448 >> mail-stats.c:56 >> >> Is this an old compiler issue or something else? > > ? > Try > this:?https://github.com/dovecot/core/commit/dd6043c05e32a8e8db1233ed711a2c74d1477a89 This one fixes issue with old Solaris 10 compiler. Pagarbiai Mantas Gegu?is VU Informacini? technologij? taikymo centras tel. 8 5 236 6208 From g.dimakopoulos at sophimail.com Mon Feb 6 18:16:14 2017 From: g.dimakopoulos at sophimail.com (George Dimakopoulos) Date: Mon, 6 Feb 2017 20:16:14 +0200 Subject: Dsync migration with shared folders Message-ID: Hi everyone, We discovered that the migration path from a VPS to our 2XHP replication server environment with shared folders should be: 1) Rsync all domains from the VPS to serverA. Then rsync all domains from ServerA to ServerB. ServerA & ServerB are in a identical state. 2) Create an empty directory for the shared indexes 3) Start dovecot on both machines. If we did not sync both machines then dsync could not replicate the domains that contained shared folders. What is the right migration process for domains with shared folders ? Regards, George dovecot -n ------------------------- # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 11.0-RELEASE-p2 amd64 zfs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 2560 default_process_limit = 512 dict { acl = mysql:/usr/local/etc/dovecot/dovecot-shared-sql.conf.ext quota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_password = # hidden, use -P to show it doveadm_port = 12345 log_path = /var/log/dovecot.log mail_debug = yes mail_home = /usr/local/vhosts/mail/%d/%n mail_location = maildir:/usr/local/vhosts/mail/%d/%n:LAYOUT=fs mail_max_userip_connections = 70 mail_plugins = quota acl notify replication mail_privileged_group = vmail mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace { inbox = no list = children location = maildir:/usr/local/vhosts/mail/%%d/%%n:LAYOUT=fs:INDEX=/usr/local/vhosts/indexes/%d/%n/shared/%%u:INDEXPVT=/usr/local/vhosts/indexes/%d/%n/shared/%%u prefix = shared/%%d/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:mx1 quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = /usr/local/vhosts/mail/%d/%n/.dovecot.sieve sieve_before = /usr/local/vhosts/sieve/before.d/ sieve_dir = /usr/local/vhosts/mail/%d/%n sieve_global_dir = /usr/local/vhosts/sieve/%d sieve_global_path = /usr/local/vhosts/sieve/%d/default.sieve } protocols = imap lmtp sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service config { unix_listener config { user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service replicator { unix_listener replicator-doveadm { mode = 0666 } } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota acl notify replication sieve } protocol lda { mail_plugins = quota acl notify replication sieve acl postmaster_address = root } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = quota acl notify replication imap_quota imap_acl } ssl_cert = ssl_key = From steve at degga.net Mon Feb 6 22:39:07 2017 From: steve at degga.net (Steven Mainor) Date: Mon, 06 Feb 2017 17:39:07 -0500 Subject: Messages on this list are often marked as spam. Message-ID: <1486420747.1665.4.camel@degga.net> Hello, It seems that I get several emails a week from this list in my spam folder. Usually because the DKIM signature fails. Has anyone else noticed this problem or is it just me? -- Steven mainor steve at degga.net From nlekkas at gmail.com Mon Feb 6 23:34:44 2017 From: nlekkas at gmail.com (Nick Lekkas) Date: Tue, 7 Feb 2017 01:34:44 +0200 Subject: Dovecot with dsync Message-ID: Hello to all !!! I am trying to setup dsync on CentOS latest with dovecot 2.2.10 ...but does not seem to work !! I have tried to follow many guides i have found but none seem to work ! Before i mention my issues please clear me something: 1. dsynch to work needs for sure mysql ..? or can work with the default linux user scheme ..? 2. if mysql is required please where can i a working working user schema ? thanks in advance for your time ... __________ Information from ESET Endpoint Antivirus, version of virus signature database 14893 (20170206) __________ The message was checked by ESET Endpoint Antivirus. Email message - is OK http://www.eset.com From rlaager at wiktel.com Tue Feb 7 04:04:12 2017 From: rlaager at wiktel.com (Richard Laager) Date: Mon, 6 Feb 2017 22:04:12 -0600 Subject: Sieve LDA Errors (Improper Saving?) Message-ID: <4a673d8e-6ea5-4ffe-eb7a-4af9f1eaaf03@wiktel.com> I'm getting lots of errors like this (possibly on every message delivery): imap2 dovecot: lmtp(rlaager at wiktel.com): Error: OU02K+gQmFhUAwAAVtfydQ : sieve: binary save: failed to create temporary file: open(/var/lib/dovecot/sieve/junk-mail.svbin.ima p2.852.) failed: Permission denied (euid=500(vmail) egid=500(vmail) missing +w perm: /var/lib/dovecot/ sieve, dir owned by 0:0 mode=0755) imap2 dovecot: lmtp(rlaager at wiktel.com): Error: OU02K+gQmFhUAwAAVtfydQ: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/var/lib/dovecot/sieve/junk-mail.sieve' need to be pre-compiled using the sievec tool It's intentional in my setup that the vmail user can't write to the global sieve script directory. But it shouldn't need to, as those are already pre-compiled: rlaager at imap2:/var/lib/dovecot/sieve$ ls -la total 12 drwxr-xr-x 2 root root 4096 Nov 29 22:27 . drwxr-xr-x 3 root root 4096 Feb 6 20:39 .. lrwxrwxrwx 1 root root 53 Sep 12 01:35 junk-mail.sieve -> /usr/share/wiktel-server-mail-backend/junk-mail.sieve -rw-r--r-- 1 root root 254 Nov 29 22:27 junk-mail.svbin rlaager at imap2:/var/lib/dovecot/sieve$ ls -la /usr/share/wiktel-server-mail-backend/junk-mail.sieve -rw-r--r-- 1 root root 124 Oct 31 09:34 /usr/share/wiktel-server-mail-backend/junk-mail.sieve Note that the .svbin is from November, while the text version is from October. Even if something is looking at the date of the symlink, that's from September. So the first question is... why is Dovecot trying to write the binary file? I dug into the Pigeonhole code... I think, but am certainly not sure, that lda_sieve_open() in pigeonhole/src/plugins/lda-sieve/lda-sieve-plugin.c is the relevant function calling lda_sieve_binary_save(). At the end of the function, it has: if (!recompile) lda_sieve_binary_save(srctx, sbin, script); This seems odd to me. Why is it trying to save in the "!recompile" case? It seems like it should be saving in the "recompile" case. If I'm reading this code right, recompile is set when it loads a corrupt sieve binary script and needs to recompile from text. I could be completely off, though. Any thoughts? -- Richard From lenaigst at maelenn.org Tue Feb 7 05:21:49 2017 From: lenaigst at maelenn.org (Thierry) Date: Tue, 7 Feb 2017 07:21:49 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> <03798325.20170206083620@maelenn.org> Message-ID: <1876413577.20170207072149@maelenn.org> Bonjour Markus, > - Have you checked that port 12345 as specified below is open/forwarded > and actually /used/ by dovecot (e.g., use "netstat -tulpn|grep dovecot")? Yes of course: tcp 0 0 0.0.0.0:12345 0.0.0.0:* LISTEN 22025/dovecot tcp6 0 0 :::12345 :::* LISTEN 22025/dovecot > - Did you retrace your steps and have you verified that synchronisation > works with ssl disabled? This dovecot is working well with my email client and web mail interface, I would prefer not to start playing with this config file ... > - Did you verify your certificate files (e.g., "openssl verify -verbose > -CAfile /etc/ssl/certs/GandiCA2.pem /etc/ssl/certs/key.crt")? yes: openssl verify -verbose -CAfile /etc/ssl/certs/GandiCA2.pem /etc/ssl/certs/key.crt /etc/ssl/certs/key.crt: OK > Personally, I prefer to use a single, specialised tool to manage > certificates/encryption (which in my case is stunnel); all other > programs are set up using (link-)local ip addresses only. If everything > but encryption works with your setup, this might be a possible > "workaround". (Apart from that, stunnel debug mode is very detailed and > can help you to rule out problems with the certificates/connections > between two nodes.) > And once the latter works but the dovecot setup below still does not, it > would also point to a problem with certificate handling by dovecot > (could be library related). This morning logs: Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL > KR, Markus Thx > Am 06.02.2017 um 07:36 schrieb Thierry: >> Hi Aki, >> >> I do not have any error message but (on both server): >> >> doveadm replicator status '*' >> doveadm(root): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Connection refused >> >> Thx >> >> >> Le vendredi 3 f?vrier 2017 ? 17:09:52, vous ?criviez : >> >>> Please keep responses in list. rm -f >>> /var/lib/dovecot/ssl-parameters.dat, i think it was in that dir. >> >>> On 2017-02-03 17:00, Thierry wrote: >>>> Hi, >>>> >>>> I have removed the '<' : >>>> >>>> ssl_client_ca_file = /etc/ssl/certs/GandiCA2.pem >>>> >>>> But now: >>>> >>>> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>>> doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>>> doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>>> >>>> Any idea ? >>>> >>>> Thx >>>> >>>>> Yes. The ssl_client_ca_file is not actually expecting <, just file name. >>>>> Aki >>>>> On 2017-02-03 15:13, Thierry wrote: >>>>>> Hi, >>>>>> >>>>>> I have made change: >>>>>> >>>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>>> ssl = required >>>>>> verbose_ssl = no >>>>>> ssl_key = >>>>> ssl_cert = >>>>> ssl_client_ca_file = >>>>> >>>>>> >>>>>> # Create a listener for doveadm-server >>>>>> service doveadm { >>>>>> user = vmail >>>>>> inet_listener { >>>>>> port = 12345 >>>>>> ssl= yes >>>>>> } >>>>>> } >>>>>> >>>>>> and doveadm_port = 12345 // mail_replica = tcps:server2.domain.ltd # use doveadm_port >>>>>> >>>>>> And now: >>>>>> >>>>>> Feb 03 14:11:16 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't load CA certs from directory : error:02001024:system library:fopen:File name too long >>>>>> Feb 03 14:11:17 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 >>>>>> Feb 03 14:11:17 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >>>>>> >>>>>> Thx for your support >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Le vendredi 3 f?vrier 2017 ? 11:34:43, vous ?criviez : >>>>>> >>>>>>> Hello, >>>>>>> On 02/03/2017 08:51 AM, Thierry wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Still working with my dsync pb. >>>>>>>> I have done a clone (vmware) of my email server. >>>>>>>> Today I have two strictly identical emails servers (server1 >>>>>>>> (main) and server2 (bck) (except IP, hostname and mail_replica). >>>>>>>> >>>>>>>> The ssl config on my both server: >>>>>>>> >>>>>>>> ssl_protocols = !SSLv2 !SSLv3 >>>>>>>> ssl = required >>>>>>>> verbose_ssl = no >>>>>>>> ssl_key = >>>>>>> ssl_cert = >>>>>>> ssl_ca = >>>>>> I think it should be ssl_client_ca_file = >>>>>>> >>>>>>> This config is working for my email client and my email web >>>>>>>> interface ... >>>>>>>> >>>>>>>> Are they on the right order ? >>>>>>>> >>>>>>>> mail_replica = tcps:server1 at domain.ltd and tcps:server2 at domain.ltd >>>>>>>> >>>>>>>> There is trafic on my iptables rules on my both servers: >>>>>>>> >>>>>>>> 60 3600 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:4711 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> My error message from server1 (main server): >>>>>>>> >>>>>>>> Feb 03 08:38:08 doveadm(user1 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>>> Feb 03 08:42:35 doveadm(user2 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>>> Feb 03 08:42:35 doveadm(user3 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>>> Feb 03 08:42:35 doveadm(user4 at domain.ltd): Error: sync: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) >>>>>>>> >>>>>>>> No logs from server2 >>>>>>>> >>>>>>>> Any ideas ? >>>>>>>> >>>>>>>> Thx for your support >>>>>>>> >>>>>>>> >> >> -- Cordialement, Thierry e-mail : lenaigst at maelenn.org From juri at koschikode.com Tue Feb 7 07:21:24 2017 From: juri at koschikode.com (Juri Haberland) Date: Tue, 7 Feb 2017 08:21:24 +0100 Subject: Messages on this list are often marked as spam. In-Reply-To: <1486420747.1665.4.camel@degga.net> References: <1486420747.1665.4.camel@degga.net> Message-ID: <3cb91977-1602-870d-c13b-bfc01b8f04c5@koschikode.com> On 06.02.2017 23:39, Steven Mainor wrote: > Hello, > > It seems that I get several emails a week from this list in my spam > folder. Usually because the DKIM signature fails. Has anyone else > noticed this problem or is it just me? No, it's not just you. There are some people that have a DMARC policy but fail to add a DKIM signature or people that use a gmail.com address but do not relay their outgoing mail through GMail, hence missing the GMail DKIM signature... There is a third category that has a DKIM signature but this fails to verify for whatever reason... Juri From btj at havleik.no Tue Feb 7 07:22:59 2017 From: btj at havleik.no (=?UTF-8?B?QmrDuHJu?= T Johansen) Date: Tue, 7 Feb 2017 08:22:59 +0100 Subject: Messages on this list are often marked as spam. In-Reply-To: <1486420747.1665.4.camel@degga.net> References: <1486420747.1665.4.camel@degga.net> Message-ID: <20170207082259.406481aa@pennywise.havleik.net> On Mon, 06 Feb 2017 17:39:07 -0500 Steven Mainor wrote: > Hello, > > It seems that I get several emails a week from this list in my spam > folder. Usually because the DKIM signature fails. Has anyone else > noticed this problem or is it just me? > > -- > Steven mainor > steve at degga.net Yes, same here... Having the same problem on other maillists as well, not sure why. BTJ From aki.tuomi at dovecot.fi Tue Feb 7 07:55:08 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 7 Feb 2017 09:55:08 +0200 Subject: Plugin "mail_crypt" - using folder keys In-Reply-To: <16d8a273-457e-5a15-266e-4cf9bfe7eed2@dovecot.fi> References: <406817759.137767.1485529114389.JavaMail.zimbra@remotesystems.ru> <16d8a273-457e-5a15-266e-4cf9bfe7eed2@dovecot.fi> Message-ID: <77fc5da6-4c20-dc37-1f68-9477c0d8c957@dovecot.fi> On 30.01.2017 09:15, Aki Tuomi wrote: > Hi Evgeniy! > > > On 27.01.2017 16:58, Evgeniy Korneechev wrote: >> Hi, i have two questions about using "folder keys" for encryption: >> >> 1. If i use this method how can i decrypt files manually? > You have to export the private key from mailbox attributes, see doveadm > mailbox cryptokey export. > I noticed that this is probably bit too short answer, but for longer answer I'll write up information under https://wiki.dovecot.org/Design/Dcrypt soon. Aki From stephan at rename-it.nl Tue Feb 7 11:29:22 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 7 Feb 2017 12:29:22 +0100 Subject: fts_solr and connection via https:// In-Reply-To: <4c177121-f649-11e5-7f34-f91ea20ceb79@jan-von.de> References: <3e405043-9e5b-39a5-6ff1-0c462a7bb8cb@jan-von.de> <836bf373-79be-a2cf-3012-45b238014b1d@rename-it.nl> <4c177121-f649-11e5-7f34-f91ea20ceb79@jan-von.de> Message-ID: <69269fb5-65a9-8cfa-ea06-4e6a3cf0232d@rename-it.nl> Op 31-1-2017 om 6:33 schreef Jan Vonde: > Am 31.01.2017 um 00:04 schrieb Stephan Bosch: >> Op 1/22/2017 om 12:01 PM schreef Stephan Bosch: >>> Op 1/22/2017 om 10:01 AM schreef Jan Vonde: >>>> I tried adding the following settings but that didn't help: >>>> ssl_ca = < /etc/ssl/certs/ca-certificates.crt >>>> ssl_client_ca_dir = /etc/ssl/certs >>>> >>>> Can you give me a hint how I can get the ssl certificate accepted? >>> That should normally have done the trick. However, the sources tell me >>> that no ssl_client settings are propagated to the http_client used by >>> fts-solr, so SSL is not currently supported it seems. >>> >>> I'll check how easy it is to add that. >> >> Just to keep you informed: I created a patch, but it is still being >> tested. >> > > Thanks for the update Stephan! Awesome! Looking forward to test it > myself :-) https://github.com/dovecot/core/commit/526631052ca3175357302af8fa7dcbf763b40c53 Regards, Stephan. From velicrongr at gmail.com Mon Feb 6 17:28:54 2017 From: velicrongr at gmail.com (=?UTF-8?B?zpPOuc+Oz4HOs86/z4IgzpTOt868zrHOus+Mz4DOv8+FzrvOv8+C?=) Date: Mon, 6 Feb 2017 19:28:54 +0200 Subject: Dsync migration with shared folders Message-ID: Hi everyone, We discovered that the migration path from a VPS to our 2XHP replication server environment with shared folders should be: 1) Rsync all domains from the VPS to serverA. Then rsync all domains from ServerA to ServerB. ServerA & ServerB are in a identical state. 2) Create an empty directory for the shared indexes 3) Start dovecot on both machines. If we did not sync both machines then dsync could not replicate the domains that contained shared folders. What is the right migration process for domains with shared folders ? Regards, George g.dimakopoulos at sophimail.com dovecot -n ------------------------- # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 11.0-RELEASE-p2 amd64 zfs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 2560 default_process_limit = 512 dict { acl = mysql:/usr/local/etc/dovecot/dovecot-shared-sql.conf.ext quota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } doveadm_password = # hidden, use -P to show it doveadm_port = 12345 log_path = /var/log/dovecot.log mail_debug = yes mail_home = /usr/local/vhosts/mail/%d/%n mail_location = maildir:/usr/local/vhosts/mail/%d/%n:LAYOUT=fs mail_max_userip_connections = 70 mail_plugins = quota acl notify replication mail_privileged_group = vmail mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace { inbox = no list = children location = maildir:/usr/local/vhosts/mail/%%d/%%n:LAYOUT=fs:INDEX=/ usr/local/vhosts/indexes/%d/%n/shared/%%u:INDEXPVT=/usr/ local/vhosts/indexes/%d/%n/shared/%%u prefix = shared/%%d/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:mx1 quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = /usr/local/vhosts/mail/%d/%n/.dovecot.sieve sieve_before = /usr/local/vhosts/sieve/before.d/ sieve_dir = /usr/local/vhosts/mail/%d/%n sieve_global_dir = /usr/local/vhosts/sieve/%d sieve_global_path = /usr/local/vhosts/sieve/%d/default.sieve } protocols = imap lmtp sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service config { unix_listener config { user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service replicator { unix_listener replicator-doveadm { mode = 0666 } } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota acl notify replication sieve } protocol lda { mail_plugins = quota acl notify replication sieve acl postmaster_address = root } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = quota acl notify replication imap_quota imap_acl } ssl_cert = ssl_key = From ueberall at projektzentrisch.de Tue Feb 7 22:31:08 2017 From: ueberall at projektzentrisch.de (Markus Ueberall) Date: Tue, 7 Feb 2017 23:31:08 +0100 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: <1876413577.20170207072149@maelenn.org> References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> <03798325.20170206083620@maelenn.org> <1876413577.20170207072149@maelenn.org> Message-ID: Dear Thierry, (I'm omitting the remainder of your post because the below has a separate root cause from what has been assumed.) >[...] > This morning logs: > > Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in state_dir: ssl-parameters.dat - disabling SSL 360 > Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, disabling SSL >[...] Did I miss these lines before or did the messages change? In either case, have a look at http://wiki.dovecot.org/SSL/DovecotConfiguration#SSL_security_settings which explains how to fix this in detail--if you're lucky, your problems might be gone afterwards. KR, Markus From adam at shostack.org Wed Feb 8 00:27:46 2017 From: adam at shostack.org (Adam Shostack) Date: Tue, 7 Feb 2017 19:27:46 -0500 Subject: Migrating to sieve Message-ID: <20170208002746.GA4092@calypso.stonekeep.com> Hi, As I migrate to sieve, I would like to carry over the logic I had in procmail rules, which let me take all mail where I'd never sent email to that address into a final spam catchall, which I called "neversent." I see that sieve can't run external programs, and I wonder if that means I'm SOL, or is there an alternate approach that does roughly the same thing? (I understand the general security logic for that decision. I have root access on the server in question if that helps.) Thank you for any help you can provide! Adam The procmail code was: # a bunch of complex shell script to extract From: into a variable of that # name. :0: * ^TO_$ME * !? fgrep -q --ignore-case -e "$FROM" $HOME/.lbdb/m_inmail.list neversent along with a procmail-outbound setup of: :0hc | lbdb-fetchaddr -a :0 ! "$@" From stephan at rename-it.nl Wed Feb 8 00:41:01 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 8 Feb 2017 01:41:01 +0100 Subject: Migrating to sieve In-Reply-To: <20170208002746.GA4092@calypso.stonekeep.com> References: <20170208002746.GA4092@calypso.stonekeep.com> Message-ID: <49f0a17f-fdf9-852f-a885-74dd5be7d822@rename-it.nl> Op 2/8/2017 om 1:27 AM schreef Adam Shostack: > Hi, > > As I migrate to sieve, I would like to carry over the logic I had in > procmail rules, which let me take all mail where I'd never sent email > to that address into a final spam catchall, which I called > "neversent." > > I see that sieve can't run external programs, and I wonder if that > means I'm SOL, or is there an alternate approach that does roughly the > same thing? > > (I understand the general security logic for that decision. I have > root access on the server in question if that helps.) > > Thank you for any help you can provide! http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms Regards, Stephan. From adam at shostack.org Wed Feb 8 00:45:26 2017 From: adam at shostack.org (Adam Shostack) Date: Tue, 7 Feb 2017 19:45:26 -0500 Subject: Migrating to sieve In-Reply-To: <49f0a17f-fdf9-852f-a885-74dd5be7d822@rename-it.nl> References: <20170208002746.GA4092@calypso.stonekeep.com> <49f0a17f-fdf9-852f-a885-74dd5be7d822@rename-it.nl> Message-ID: <20170208004526.GA7866@calypso.stonekeep.com> On Wed, Feb 08, 2017 at 01:41:01AM +0100, Stephan Bosch wrote: | Op 2/8/2017 om 1:27 AM schreef Adam Shostack: | > Hi, | > | > As I migrate to sieve, I would like to carry over the logic I had in | > procmail rules, which let me take all mail where I'd never sent email | > to that address into a final spam catchall, which I called | > "neversent." | > | > I see that sieve can't run external programs, and I wonder if that | > means I'm SOL, or is there an alternate approach that does roughly the | > same thing? | > | > (I understand the general security logic for that decision. I have | > root access on the server in question if that helps.) | > | > Thank you for any help you can provide! | | http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms | | Regards, | | Stephan. Danke! Adam From stephan at rename-it.nl Wed Feb 8 00:51:58 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 8 Feb 2017 01:51:58 +0100 Subject: Sieve LDA Errors (Improper Saving?) In-Reply-To: <4a673d8e-6ea5-4ffe-eb7a-4af9f1eaaf03@wiktel.com> References: <4a673d8e-6ea5-4ffe-eb7a-4af9f1eaaf03@wiktel.com> Message-ID: <8d0e0a23-eef4-b0ee-b342-373a083702cd@rename-it.nl> Op 2/7/2017 om 5:04 AM schreef Richard Laager: > I'm getting lots of errors like this (possibly on every message delivery): > > imap2 dovecot: lmtp(rlaager at wiktel.com): Error: OU02K+gQmFhUAwAAVtfydQ > : sieve: binary save: failed to create temporary file: > open(/var/lib/dovecot/sieve/junk-mail.svbin.ima > p2.852.) failed: Permission denied (euid=500(vmail) egid=500(vmail) > missing +w perm: /var/lib/dovecot/ > sieve, dir owned by 0:0 mode=0755) > > imap2 dovecot: lmtp(rlaager at wiktel.com): Error: OU02K+gQmFhUAwAAVtfydQ: > sieve: The LDA Sieve plugin does not have permission to save global > Sieve script binaries; global Sieve scripts like > `/var/lib/dovecot/sieve/junk-mail.sieve' need to be pre-compiled using > the sievec tool > > It's intentional in my setup that the vmail user can't write to the > global sieve script directory. But it shouldn't need to, as those are > already pre-compiled: > > rlaager at imap2:/var/lib/dovecot/sieve$ ls -la > total 12 > drwxr-xr-x 2 root root 4096 Nov 29 22:27 . > drwxr-xr-x 3 root root 4096 Feb 6 20:39 .. > lrwxrwxrwx 1 root root 53 Sep 12 01:35 junk-mail.sieve -> > /usr/share/wiktel-server-mail-backend/junk-mail.sieve > -rw-r--r-- 1 root root 254 Nov 29 22:27 junk-mail.svbin > > rlaager at imap2:/var/lib/dovecot/sieve$ ls -la > /usr/share/wiktel-server-mail-backend/junk-mail.sieve > -rw-r--r-- 1 root root 124 Oct 31 09:34 > /usr/share/wiktel-server-mail-backend/junk-mail.sieve > > Note that the .svbin is from November, while the text version is from > October. Even if something is looking at the date of the symlink, that's > from September. > > So the first question is... why is Dovecot trying to write the binary file? Newer versions of Pigeonhole may use a different version of the compiled binary format. So, for some upgrades it may be necessary to recompile. > I dug into the Pigeonhole code... I think, but am certainly not sure, > that lda_sieve_open() in > pigeonhole/src/plugins/lda-sieve/lda-sieve-plugin.c is the relevant > function calling lda_sieve_binary_save(). At the end of the function, it > has: > > if (!recompile) > lda_sieve_binary_save(srctx, sbin, script); > > This seems odd to me. Why is it trying to save in the "!recompile" case? > It seems like it should be saving in the "recompile" case. If I'm > reading this code right, recompile is set when it loads a corrupt sieve > binary script and needs to recompile from text. That part of the code is only relevant for binaries that are seen to be corrupt at runtime, not when they are just perceived to be out-of-date. I'll be reviewing this code soon anyway, because there are some known problems with the up-to-date check itself (file time stamp race condition). I'll be reviewing this as well. Anyway, for now you should be helped by just manually recompiling. Regards, Stephan. From stephan at rename-it.nl Wed Feb 8 01:22:52 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 8 Feb 2017 02:22:52 +0100 Subject: Managesieve cannot access script store In-Reply-To: References: Message-ID: Op 2/5/2017 om 8:53 PM schreef dovelist: > Hi, > > I am trying to get sieve working on a new OpenSuse leap 42.2 install. > On my 'old' OpenSuse 13.2 machine it worked fine. > > The problem is that Managesieve can't access the script store and > won't let me create any script. It says permission denied on ~/sieve > directory. See log below. I 've activated debug logging, but that > doesn't give any clues to me. Also, I've set the directory accessible > to all, but Managesieve still complains. > >> cd ~ >> ls -l > drwx------ 1 rogier users 8340 5 feb 16:54 Maildir > drwxrwxrwx 1 rogier users 24 5 feb 18:38 sieve > > To rule out client issues (kmail) I tested also with Manual TLS Login > as described in: > http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Troubleshooting > > Same result. > > I am puzzled. I can't find anything wrong in the dovecot > configuration. The output of dovecot -n is shown below. > Hope someone has a solution. A lot of mail is waiting to get sorted... > > Best Regards, > Rogier > > > The log: > > feb 05 20:22:18 p150 dovecot[12120]: managesieve-login: Login: > user=, method=PLAIN, rip=192.168.0.18, lip=192.168.0.20, > mpid=12135, TLS, session= > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > Effective uid=1000, gid=100, home=/home/rogier > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, > list=yes, subscriptions=yes location=maildir:~/Maildir > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > maildir++: root=/home/rogier/Maildir, index=, indexpvt=, control=, > inbox=/home/rogier/Maildir, alt= > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: Pigeonhole version 0.4.15 (97b3da0) initializing > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: include: sieve_global is not set; it is currently not possible > to include `:global' scripts. > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: file storage: Using active Sieve script path: > /home/rogier/.dovecot.sieve > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: file storage: Using script storage path: /home/rogier/sieve/ > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: file storage: Using permissions from /home/rogier/sieve/: > mode=0777 gid=-1 > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: file storage: Relative path to sieve storage in active link: > sieve/ > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: > sieve: file storage: sync: Synchronization active > feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Error: > sieve: file storage: Failed to list scripts: > opendir(/home/rogier/sieve) failed: Permission denied Normally, Dovecot permission errors are more helpful than that. So, this error message in itself is a bit of a bug: https://github.com/dovecot/pigeonhole/commit/51e4ff296987781e1ce93cb1c0ccc14e863bf8d6 About the cause of this error: keep in mind that the whole directory path needs read/execute permission, not only the leaf directory. You could try a command other than LISTSCRIPTS in your manual debugging efforts. That should take a different code path that provides a more detailed error. Regards, Stephan. From rlaager at wiktel.com Wed Feb 8 01:44:03 2017 From: rlaager at wiktel.com (Richard Laager) Date: Tue, 7 Feb 2017 19:44:03 -0600 Subject: Sieve LDA Errors (Improper Saving?) In-Reply-To: <8d0e0a23-eef4-b0ee-b342-373a083702cd@rename-it.nl> References: <4a673d8e-6ea5-4ffe-eb7a-4af9f1eaaf03@wiktel.com> <8d0e0a23-eef4-b0ee-b342-373a083702cd@rename-it.nl> Message-ID: <81172497-0ed0-7382-5768-4533ce0d6609@wiktel.com> On 02/07/2017 06:51 PM, Stephan Bosch wrote: > Newer versions of Pigeonhole may use a different version of the compiled > binary format. So, for some upgrades it may be necessary to recompile. > Anyway, for now you should be helped by just manually recompiling. Manually recompiling fixed it. We had upgraded a while back, so the version difference could very well be the issue. Thanks! -- Richard From lenaigst at maelenn.org Wed Feb 8 05:20:01 2017 From: lenaigst at maelenn.org (Thierry) Date: Wed, 8 Feb 2017 07:20:01 +0200 Subject: Dovecot dsync 'ssl_client_ca' In-Reply-To: References: <1547742423.20170203095111@maelenn.org> <19710310116.20170203151312@maelenn.org> <61bc30ed-2966-ba68-ec84-63d12cd34e72@dovecot.fi> <198484904.20170203170031@maelenn.org> <03798325.20170206083620@maelenn.org> <1876413577.20170207072149@maelenn.org> Message-ID: <366918343.20170208072001@maelenn.org> Bonjour Markus, Things are working but without SSL. I will have a look and come back to you. Thx Le mercredi 8 f?vrier 2017 ? 00:31:08, vous ?criviez : > Dear Thierry, > (I'm omitting the remainder of your post because the below has a > separate root cause from what has been assumed.) >>[...] >> This morning logs: >> >> Feb 07 05:50:13 doveadm: Error: Corrupted SSL parameters file in > state_dir: ssl-parameters.dat - disabling SSL 360 >> Feb 07 05:50:13 doveadm: Error: Couldn't initialize SSL parameters, > disabling SSL >>[...] > Did I miss these lines before or did the messages change? > In either case, have a look at > http://wiki.dovecot.org/SSL/DovecotConfiguration#SSL_security_settings > which explains how to fix this in detail--if you're lucky, your problems > might be gone afterwards. > KR, Markus -- Cordialement, Thierry e-mail : lenaigst at maelenn.org From carsten at czichos.net Wed Feb 8 06:39:16 2017 From: carsten at czichos.net (Carsten Czichos) Date: Wed, 8 Feb 2017 07:39:16 +0100 Subject: Filenames and flags Message-ID: <3116A510-27E4-4E1B-980E-31A9905C3C72@czichos.net> Hi list, where can I find the meaning of lowercase file flags ? I found this information: http://cr.yp.to/proto/maildir.html But what does the lowercase a & d mean as in ...,S=2364,W=2411:2,Sad ? Thanx for your help ! From skdovecot at smail.inf.fh-brs.de Wed Feb 8 06:58:01 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 8 Feb 2017 07:58:01 +0100 (CET) Subject: Filenames and flags In-Reply-To: <3116A510-27E4-4E1B-980E-31A9905C3C72@czichos.net> References: <3116A510-27E4-4E1B-980E-31A9905C3C72@czichos.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 8 Feb 2017, Carsten Czichos wrote: > where can I find the meaning of lowercase file flags ? > > But what does the lowercase a & d mean as in ...,S=2364,W=2411:2,Sad ? it's documented in the Wiki: http://wiki2.dovecot.org/MailboxFormat/Maildir - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWJrBeXz1H7kL/d9rAQJltAgAhDHzl3pjJsmDLF1kwnURYwmY+xzidIf/ /eeRGPBulqyfzNCsnbg1UX++SgXKrKLRpNZfFYlF/VaQDtmfwt6YEhdzhH59UFMB otx26+RwJUymW4UxuIQmwf8fkwqtHf9npQhFyhj2sju2StNeUz3ZR8xc4nFcKSPp oipW7ovq3Wc0Q8sWKBG3gPprnr7C1cqjCwgqkQRLQ3JfWqdyVluH8vNh12Djokz7 XacQfClt0GmeCpc/NqZchp53X/W519ncZoRWQv2qHjnktmb7uCDOIXyYjkd2Awmi 4yp4X1ydjcddvzE6eLiT8wpnxD2P92zYFNNqYbQWrB1eWByci7REcA== =z4v8 -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Wed Feb 8 07:10:15 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 8 Feb 2017 08:10:15 +0100 (CET) Subject: Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22. In-Reply-To: <9df9b8cb09e1c47a6c5f72f5bfa1b696@mail2.ca> References: <9df9b8cb09e1c47a6c5f72f5bfa1b696@mail2.ca> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 5 Feb 2017, ygrishin-lists at mail2.ca wrote: > service dict { > unix_listener dict { > mode = 0660 > user = Debian-exim > group = Debian-exim > } > } > > dovecot-lda-erros.log: > ********************** > Feb 04 14:23:33 lda(testuser at XXX): Error: read(/var/run/dovecot/dict) failed: > Remote disconnected > > dovecot.log: > ************ > Feb 04 13:57:06 imap(YYY at XXX): Error: write(/var/run/dovecot/dict) failed: > Broken pipe > ... > > dovecot-debug.log: > ****************** > Feb 04 13:18:12 lda(YYY at XXX): Error: read(/var/run/dovecot/dict) failed: > Remote disconnected > Feb 04 13:18:12 lda(YYY at XXX): Error: dict quota: Quota update failed, it's > now desynced > Feb 04 13:57:07 lda(testuser at XXX): Error: write(/var/run/dovecot/dict) > failed: Broken pipe Does a process listens on /var/run/dovecot/dict ? The socket is accessable by Debian-exim:Debian-exim only (0660). As what user and group does the LDA and imap service run as? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEUAwUBWJrEV3z1H7kL/d9rAQLEGQf4gv/br0Q6AqJAyY5PNDotR+L0BBi5oV4v LUn5qeocEFVQGY9N/ESXv5LTffoFzcAvLiFJtBKzki0S4aiUZa000RfbAbIRPaeQ O9ZMFd/8AEqIOGYtSgoTTk0XenQ+x7GedAE5BT8KFd6K0fZp4FqL/9CVko28wL91 sdhssd42dRnFfgk/+AjeRrnIq/bxFsL9uwbV6MUUMlSBAFlTw4l8+Z129q8pXvsI te41cjWHIBR4wqF+6UoeIF7fLaRmYW/zCRRm3gOCmLGn/Ui/fGqk/k+rG5gg4As5 JKkAZT9JvbmqpXOPnE4kIn5qYyUXej1/nT7ZCPhJspyNtlQ/LW5W =w20l -----END PGP SIGNATURE----- From mail at tomsommer.dk Wed Feb 8 09:02:46 2017 From: mail at tomsommer.dk (Tom Sommer) Date: Wed, 08 Feb 2017 10:02:46 +0100 Subject: Changes to userdb not picked up In-Reply-To: References: <5f233d2bfff9cd222192ea56d680e2d7@tomsommer.dk> Message-ID: I know, but I'm changing the quota in dict:sql so running commands on the mailserver itself it something I would like to avoid. --- Tom On 2017-02-06 14:29, Urban Loesch wrote: > You can flush the cache with: "doveadm auth cache flush $USER" > > Regards > Urban > > > Am 06.02.2017 um 13:59 schrieb Tom Sommer: >> I have my quota limits stored in userdb and auth_cache enabled >> (default settings). >> >> When I change the quota limit the old limit is still cached for the >> user for another hour. Is there any way to prevent this from >> happening? >> >> Thanks >> From skdovecot at smail.inf.fh-brs.de Wed Feb 8 10:14:00 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 8 Feb 2017 11:14:00 +0100 (CET) Subject: Changes to userdb not picked up In-Reply-To: References: <5f233d2bfff9cd222192ea56d680e2d7@tomsommer.dk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 8 Feb 2017, Tom Sommer wrote: > I know, but I'm changing the quota in dict:sql so running commands on the > mailserver itself it something I would like to avoid. hm: http://wiki2.dovecot.org/Design/DoveadmProtocol/HTTP it is essentially the same, though > On 2017-02-06 14:29, Urban Loesch wrote: >> You can flush the cache with: "doveadm auth cache flush $USER" >> >> Am 06.02.2017 um 13:59 schrieb Tom Sommer: >>> I have my quota limits stored in userdb and auth_cache enabled (default >>> settings). >>> >>> When I change the quota limit the old limit is still cached for the user >>> for another hour. Is there any way to prevent this from happening? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWJrvaHz1H7kL/d9rAQJO7gf/QCikbuxZ1bG9BAE8xn052d69JysrnH9N +0YtWjnlZW+zyGcy0mGZ9Vp08FBfYQx2ikTLpN3smjlPtvyUQ4ECXNsZAUVKPkY2 FwJ1/tnWVNfP/Ss7GqrkiekemzrbWBPdaF2upFjsyK1DZoSP/jAznrgwKSD8UhqC kXuxmBQNWp8ifiDnSGk3shbg8FEFuUsHhZ55pMxdI5Rs4LbL5Ga1sCaAy+oTvi2r tDW1Y2uhOKfGlDk3M4fiDG1/SE/9WMjXW6eP6D1CC3NcENnS2uADfWtpGEIzt4id ULDHjdi+M6IDTYDx7q3/oH3dJ7JKptVcSH4epdO0PpYRwW54AziibQ== =YuMf -----END PGP SIGNATURE----- From dbetz at df.eu Wed Feb 8 10:21:37 2017 From: dbetz at df.eu (Daniel Betz) Date: Wed, 8 Feb 2017 10:21:37 +0000 Subject: define auth timeout Message-ID: <761eb03b283f431eb19310730edde296@EXDAG08-1.EXCHANGE.INT> Hello list, when reloading dovecot via "doveadm reload" it throws me tons of errors like this: Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 imap: Error: net_connect_unix(/run/dovecot/auth-master) failed: Resource temporarily unavailable Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 imap: Error: net_connect_unix(/run/dovecot/auth-master) failed: Resource temporarily unavailable Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 imap: Error: net_connect_unix(/run/dovecot/auth-master) failed: Resource temporarily unavailable Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 imap: Error: net_connect_unix(/run/dovecot/auth-master) failed: Resource temporarily unavailable Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 imap: Error: net_connect_unix(/run/dovecot/auth-master) failed: Resource temporarily unavailable this is so long, till all client logins time out: Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 8576.416 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 8574.736 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 13163.1758 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 13163.1760 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 13163.1761 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 13163.1762 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 8566.121 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 8583.1702 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: login(mail at adress,xx.xx.xx.xx.): Request 8582.1834 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: plain(mail at adress,xx.xx.xx.xx.): Request 8576.340 timed out after 150 secs, state=3 Feb 08 10:57:30 server1 dovecot[18243]: Feb 08 10:57:30 auth: Error: login(mail at adress,xx.xx.xx.xx.): Request 8576.335 timed out after 150 secs, state=3 I have searched for this timeout in dovecot config, but havent found something. Can you tell me how to define this timeout, or is this an LDAP timeout in userdb/passdb and dovecot-ldap.conf ? Or better, how to prevent this failures? I can see them sometimes in the log too w/o reloading. Regards, Daniel dovecot-ldap.conf: uris = ldapi://%2Fvar%2Frun%2Fldapi dn = cn=xxxxxxx,o=domain,c=com dnpass = xxxxxxxxxxxxx auth_bind = no ldap_version = 3 base = o=domain,c=com user_attrs = mail=user,mailMessageStore=home,\ mailQuota=quota_rule=*:storage=%$ iterate_filter= (|(mailHost=server1.domain.com)(mailHost=popserver1.domain.com)) user_filter = (&(accountstatus=active)(|(uid=%u)(mail=%u))) pass_attrs = mail=user,userPassword=password,=proxy_maybe=y,mailHost=host,=destuser=%u[%r] pass_filter = (&(accountstatus=active)(|(uid=%u)(mail=%u))) dovecot.conf: # 2.2.25 (7be1766): /usr/local/dovecot2/etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.36.3.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_cache_negative_ttl = 1 mins auth_cache_size = 64 M auth_cache_ttl = 2 hours auth_mechanisms = plain login auth_username_chars = auth_verbose = yes base_dir = /var/run/dovecot/ debug_log_path = /dev/null default_login_user = dovecot disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 first_valid_gid = 1001 first_valid_uid = 1001 info_log_path = /var/log/dovecot/messages lda_mailbox_autocreate = yes lda_original_recipient_header = X-Envelope-To log_path = /dev/stderr login_log_format_elements = user=[%u] method=%m rip=%r lip=%l %c mail_gid = 1001 mail_location = mdbox:~:INDEX=%h/INDEX mail_plugins = " notify replication stats" mail_uid = 1001 mbox_write_locks = fcntl namespace { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox "Gesendete Elemente" { auto = no special_use = \Sent } mailbox "Infizierte Objekte" { auto = no special_use = \Junk } mailbox Sent { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = no special_use = \Trash } prefix = separator = . type = private } namespace inbox { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap2.conf driver = ldap } plugin { quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_warning = storage=85%% quota-warning 85 %u stats_refresh = 30 secs stats_track_cmds = yes } replication_max_conns = 30 sendmail_path = /usr/local/exim/bin/exim service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = popuser } unix_listener replication-notify { mode = 0666 user = popuser } } service anvil { client_limit = 60000 } service auth { client_limit = 60000 unix_listener auth-userdb { mode = 0666 user = popuser } unix_listener auth { mode = 0666 user = popuser } } service config { unix_listener config { user = popuser } } service dict { unix_listener dict { mode = 0666 user = popuser } } service doveadm { inet_listener { port = 12345 } user = popuser } service imap-login { chroot = login client_limit = 600 process_limit = 100 process_min_avail = 16 service_count = 0 } service imap { executable = /usr/local/dovecot2/libexec/dovecot/imap process_limit = 250000 } service ipc { client_limit = 60000 unix_listener ipc { mode = 0650 user = dovecot } unix_listener login/ipc-proxy { mode = 0650 user = dovecot } } service lmtp { unix_listener lmtp { mode = 0666 user = popuser } } service pop3-login { chroot = login client_limit = 600 process_limit = 100 process_min_avail = 16 service_count = 0 } service pop3 { executable = /usr/local/dovecot2/libexec/dovecot/pop3 process_limit = 250000 } service replicator { unix_listener replicator-doveadm { mode = 0600 user = popuser } } service stats { fifo_listener stats-mail { mode = 0600 user = popuser } } ssl_cert = Hello Please I need to add "Maurizio" to this MSSQL DB, but I don't now how to add this step in my opinion I'think that the user can't be found, so I will have the "unknown user" Can give here any little help to fix this? Feb 8 12:09:56 caloro dovecot: auth: Debug: auth client connected (pid=13300) Feb 8 12:09:56 caloro dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011session=dfsfdsdsJITACX+KIh#011lip=37.1 20.190.188#011rip=151.248.162.33#011lport=110#011rport=54604#011resp= Feb 8 12:09:56 caloro dovecot: auth-worker(13303): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Feb 8 12:09:56 caloro dovecot: auth-worker(13303): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Feb 8 12:09:56 caloro dovecot: auth-worker(13303): Debug: sql(maurizio at caloro.ch,151.248.162.33): query: SELECT username AS user, password_enc AS password, CONCAT(homedir, maildir) AS userdb_home, uid AS userdb_uid, gid AS userdb_gid, CONCAT('maildir:', homedir, maildir) AS userdb_mail, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM mail_users WHERE (username = 'maurizio at caloro.ch' OR email = 'maurizio at caloro.ch') AND ((imap = 1 AND 'pop3' = 'imap') OR (pop3 = 1 AND 'pop3' = 'pop3') OR 'pop3' = 'smtp' OR 'pop3' = 'sieve') Feb 8 12:09:56 caloro dovecot: auth-worker(13303): sql(maurizio at caloro.ch,151.248.162.33): unknown user Feb 8 12:09:58 caloro dovecot: auth: Debug: client passdb out: FAIL#0111#011user=maurizio at caloro.ch Feb 8 12:09:58 caloro dovecot: pop3-login: Disconnected (auth failed, 1 attempts in 2 secs): user=, method=PLAIN, rip=151.248.162.33, lip=37.120.190.188, session= Regards and thank for any hint or help mauri From skdovecot at smail.inf.fh-brs.de Wed Feb 8 11:52:23 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 8 Feb 2017 12:52:23 +0100 (CET) Subject: How to add User in MSSQL DB - error unknown user In-Reply-To: <000b01d281fd$ae85d0c0$0b917240$@gmx.ch> References: <000b01d281fd$ae85d0c0$0b917240$@gmx.ch> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 8 Feb 2017, Maurizio Caloro wrote: > Please I need to add "Maurizio" to this MSSQL DB, but I don't now how to > add this step add this step to which workflow, action, ... ? > in my opinion I'think that the user can't be found, so I will have the > "unknown user" > > Can give here any little help to fix this? INSERT INTO mail_users (...) VALUES (*correct* data of maurizio); Who has installed the Dovecot server and the user DB? This person should know how to fill the *correct* data into the SQL table. > Feb 8 12:09:56 caloro dovecot: auth-worker(13303): Debug: > sql(maurizio at caloro.ch,151.248.162.33): query: SELECT username AS user, > password_enc AS password, CONCAT(homedir, maildir) AS userdb_home, uid AS > userdb_uid, gid AS userdb_gid, CONCAT('maildir:', homedir, maildir) AS > userdb_mail, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM > mail_users WHERE (username = 'maurizio at caloro.ch' OR email = > 'maurizio at caloro.ch') AND ((imap = 1 AND 'pop3' = 'imap') OR (pop3 = 1 AND > 'pop3' = 'pop3') OR 'pop3' = 'smtp' OR 'pop3' = 'sieve') - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWJsGd3z1H7kL/d9rAQJA+Af9EQk22pq64AYSxwQhkjm0a5D3qA72oW+M +k+46ivZXYvZiqGPYzY7YpeUQAKSRw6ihkIeF+hrm8Li6bauZJ5mDt7+DBZNCXc4 5S0+qZpuVBFGrC/k/grajnkRiiB56ejMkGjNLiB6tukUVGoeT3U5Q7hzmW0q0hlf vECIswRv6Yct0ZfsmHpy1apeB3HwOb3z4C8a6oaZBMkgo3GhSeJZQVKfeJtjaG2O HIPMHjvFKJjw9P8DfRjP5rqqUxI2MZLYq5ShI/pgOIqNoTSTvRCALbcAdOj0DO8R 2xCH+0vB3vEMk4kz0bQxlW0EXdJYJkoqx2O19dNDHVdYkPBexes78w== =tFGQ -----END PGP SIGNATURE----- From mauric at gmx.ch Wed Feb 8 12:14:45 2017 From: mauric at gmx.ch (Maurizio Caloro) Date: Wed, 8 Feb 2017 13:14:45 +0100 Subject: AW: How to add User in MSSQL DB - error unknown user In-Reply-To: References: <000b01d281fd$ae85d0c0$0b917240$@gmx.ch> Message-ID: <001001d28204$f00d4fd0$d027ef70$@gmx.ch> Linux caloro.ch 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1 (2016-12-30) x86_64 GNU/Linux Apt-get install postfix dovecot Adduser -m maurizio Add config steps to dovecot.conf, main.cf, master.cf Email from CLI running, but if connect from Outlook or Thunderbird I have this error, I think that maurizio this user aren?t opened correct on this sqldb? root at caloro:/home/maurizio# postconf -n alias_maps = $alias_database append_dot_mydomain = no biff = no broken_sasl_auth_clients = yes command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/lib/postfix data_directory = /var/lib/postfix debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 dovecot_destination_recipient_limit = 1 home_mailbox = Maildir/ html_directory = no inet_interfaces = all inet_protocols = ipv4 local_transport = local mailbox_size_limit = 0 mailq_path = /usr/bin/mailq manpage_directory = /usr/share/man message_size_limit = 52428800 mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain mydomain = nmail.caloro.ch myhostname = nmail.caloro.ch mynetworks = 127.0.0.0/8 192.168.1.0/27 myorigin = $mydomain newaliases_path = /usr/bin/newaliases queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix sample_directory = /usr/share/doc/postfix sendmail_path = /usr/sbin/sendmail setgid_group = postdrop smtp_tls_note_starttls_offer = yes smtp_tls_security_level = may smtpd_banner = $myhostname ESMTP $mail_name smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_client_hostname smtpd_helo_required = yes smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_unauth_pipelining, reject_non_fqdn_recipient smtpd_relay_restrictions = smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = $myhostname smtpd_sasl_path = private/auth smtpd_sasl_type = dovecot smtpd_sender_login_maps = mysql:/etc/postfix/mysql-virtual_sender_permissions.cf smtpd_sender_restrictions = permit_mynetworks, reject_sender_login_mismatch, permit_sasl_authenticated, reject_unknown_helo_hostname, reject_unknown_recipient_domain, reject_unknown_sender_domain smtpd_tls_cert_file = /etc/ssl/server/servername.pem smtpd_tls_key_file = $smtpd_tls_cert_file smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_security_level = may unknown_local_recipient_reject_code = 550 virtual_alias_maps = mysql:/etc/postfix/mysql-virtual_alias_maps.cf virtual_gid_maps = static:2000 virtual_mailbox_base = /var/customers/mail/ virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual_mailbox_domains.cf virtual_mailbox_limit = 0 virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual_mailbox_maps.cf virtual_transport = dovecot virtual_uid_maps = static:2000 -----Urspr?ngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Steffen Kaiser Gesendet: Mittwoch, 8. Februar 2017 12:52 An: Maurizio Caloro Cc: dovecot at dovecot.org Betreff: Re: How to add User in MSSQL DB - error unknown user -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 8 Feb 2017, Maurizio Caloro wrote: > Please I need to add "Maurizio" to this MSSQL DB, but I don't now how > to add this step add this step to which workflow, action, ... ? > in my opinion I'think that the user can't be found, so I will have the > "unknown user" > > Can give here any little help to fix this? INSERT INTO mail_users (...) VALUES (*correct* data of maurizio); Who has installed the Dovecot server and the user DB? This person should know how to fill the *correct* data into the SQL table. > Feb 8 12:09:56 caloro dovecot: auth-worker(13303): Debug: > sql(maurizio at caloro.ch,151.248.162.33): query: SELECT username AS > user, password_enc AS password, CONCAT(homedir, maildir) AS > userdb_home, uid AS userdb_uid, gid AS userdb_gid, CONCAT('maildir:', > homedir, maildir) AS userdb_mail, CONCAT('maildir:storage=', > (quota*1024)) as userdb_quota FROM mail_users WHERE (username = > 'maurizio at caloro.ch' OR email = > 'maurizio at caloro.ch') AND ((imap = 1 AND 'pop3' = 'imap') OR (pop3 = 1 > AND 'pop3' = 'pop3') OR 'pop3' = 'smtp' OR 'pop3' = 'sieve') - -- Steffen Kaiser From tom at whyscream.net Wed Feb 8 15:33:04 2017 From: tom at whyscream.net (Tom Hendrikx) Date: Wed, 8 Feb 2017 16:33:04 +0100 Subject: AW: How to add User in MSSQL DB - error unknown user In-Reply-To: <001001d28204$f00d4fd0$d027ef70$@gmx.ch> References: <000b01d281fd$ae85d0c0$0b917240$@gmx.ch> <001001d28204$f00d4fd0$d027ef70$@gmx.ch> Message-ID: Hi, Your postfix and dovecot configurations refer to a MySQL database that contains email accounts. You didn't get all of that by doing 'apt-get install postfix dovecot', the database setup is something you added yourself (or someone else did that for you). Running 'adduser -m maurizio' will create a system user on the server, but no email account in the MySQL database setup. Please refer to the installation manual you (or someone else on your behalf) followed during 'Add config steps to dovecot.conf, main.cf, master.cf' which should tell you how to manage your accounts. Regards, Tom On 08-02-17 13:14, Maurizio Caloro wrote: > Linux caloro.ch 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1 (2016-12-30) x86_64 > GNU/Linux > > Apt-get install postfix dovecot > Adduser -m maurizio > Add config steps to dovecot.conf, main.cf, master.cf > > Email from CLI running, but if connect from Outlook or Thunderbird I have > this error, I think > that maurizio this user aren?t opened correct on this sqldb? > > root at caloro:/home/maurizio# postconf -n > alias_maps = $alias_database > append_dot_mydomain = no > biff = no > broken_sasl_auth_clients = yes > command_directory = /usr/sbin > config_directory = /etc/postfix > daemon_directory = /usr/lib/postfix > data_directory = /var/lib/postfix > debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd > $daemon_directory/$process_name $process_id & sleep 5 > dovecot_destination_recipient_limit = 1 > home_mailbox = Maildir/ > html_directory = no > inet_interfaces = all > inet_protocols = ipv4 > local_transport = local > mailbox_size_limit = 0 > mailq_path = /usr/bin/mailq > manpage_directory = /usr/share/man > message_size_limit = 52428800 > mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain > mydomain = nmail.caloro.ch > myhostname = nmail.caloro.ch > mynetworks = 127.0.0.0/8 192.168.1.0/27 > myorigin = $mydomain > newaliases_path = /usr/bin/newaliases > queue_directory = /var/spool/postfix > readme_directory = /usr/share/doc/postfix > sample_directory = /usr/share/doc/postfix > sendmail_path = /usr/sbin/sendmail > setgid_group = postdrop > smtp_tls_note_starttls_offer = yes > smtp_tls_security_level = may > smtpd_banner = $myhostname ESMTP $mail_name > smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, > reject_unknown_client_hostname > smtpd_helo_required = yes > smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, > reject_unauth_destination, reject_unauth_pipelining, > reject_non_fqdn_recipient > smtpd_relay_restrictions = > smtpd_sasl_auth_enable = yes > smtpd_sasl_local_domain = $myhostname > smtpd_sasl_path = private/auth > smtpd_sasl_type = dovecot > smtpd_sender_login_maps = > mysql:/etc/postfix/mysql-virtual_sender_permissions.cf > smtpd_sender_restrictions = permit_mynetworks, reject_sender_login_mismatch, > permit_sasl_authenticated, reject_unknown_helo_hostname, > reject_unknown_recipient_domain, reject_unknown_sender_domain > smtpd_tls_cert_file = /etc/ssl/server/servername.pem > smtpd_tls_key_file = $smtpd_tls_cert_file > smtpd_tls_loglevel = 1 > smtpd_tls_received_header = yes > smtpd_tls_security_level = may > unknown_local_recipient_reject_code = 550 > virtual_alias_maps = mysql:/etc/postfix/mysql-virtual_alias_maps.cf > virtual_gid_maps = static:2000 > virtual_mailbox_base = /var/customers/mail/ > virtual_mailbox_domains = > mysql:/etc/postfix/mysql-virtual_mailbox_domains.cf > virtual_mailbox_limit = 0 > virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual_mailbox_maps.cf > virtual_transport = dovecot > virtual_uid_maps = static:2000 > > > -----Urspr?ngliche Nachricht----- > Von: dovecot [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Steffen > Kaiser > Gesendet: Mittwoch, 8. Februar 2017 12:52 > An: Maurizio Caloro > Cc: dovecot at dovecot.org > Betreff: Re: How to add User in MSSQL DB - error unknown user > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 8 Feb 2017, Maurizio Caloro wrote: > >> Please I need to add "Maurizio" to this MSSQL DB, but I don't now how >> to add this step > > add this step to which workflow, action, ... ? > >> in my opinion I'think that the user can't be found, so I will have the >> "unknown user" >> >> Can give here any little help to fix this? > > INSERT INTO mail_users (...) VALUES (*correct* data of maurizio); > > Who has installed the Dovecot server and the user DB? This person should > know how to fill the *correct* data into the SQL table. > >> Feb 8 12:09:56 caloro dovecot: auth-worker(13303): Debug: >> sql(maurizio at caloro.ch,151.248.162.33): query: SELECT username AS >> user, password_enc AS password, CONCAT(homedir, maildir) AS >> userdb_home, uid AS userdb_uid, gid AS userdb_gid, CONCAT('maildir:', >> homedir, maildir) AS userdb_mail, CONCAT('maildir:storage=', >> (quota*1024)) as userdb_quota FROM mail_users WHERE (username = >> 'maurizio at caloro.ch' OR email = >> 'maurizio at caloro.ch') AND ((imap = 1 AND 'pop3' = 'imap') OR (pop3 = 1 >> AND 'pop3' = 'pop3') OR 'pop3' = 'smtp' OR 'pop3' = 'sieve') > > - -- > Steffen Kaiser > From mail at jan-von.de Wed Feb 8 20:07:36 2017 From: mail at jan-von.de (Jan Vonde) Date: Wed, 8 Feb 2017 21:07:36 +0100 Subject: fts_solr and connection via https:// In-Reply-To: <69269fb5-65a9-8cfa-ea06-4e6a3cf0232d@rename-it.nl> References: <3e405043-9e5b-39a5-6ff1-0c462a7bb8cb@jan-von.de> <836bf373-79be-a2cf-3012-45b238014b1d@rename-it.nl> <4c177121-f649-11e5-7f34-f91ea20ceb79@jan-von.de> <69269fb5-65a9-8cfa-ea06-4e6a3cf0232d@rename-it.nl> Message-ID: Am 07.02.2017 um 12:29 schrieb Stephan Bosch: > > > Op 31-1-2017 om 6:33 schreef Jan Vonde: >> Am 31.01.2017 um 00:04 schrieb Stephan Bosch: >>> Op 1/22/2017 om 12:01 PM schreef Stephan Bosch: >>>> Op 1/22/2017 om 10:01 AM schreef Jan Vonde: >>>>> I tried adding the following settings but that didn't help: >>>>> ssl_ca = < /etc/ssl/certs/ca-certificates.crt >>>>> ssl_client_ca_dir = /etc/ssl/certs >>>>> >>>>> Can you give me a hint how I can get the ssl certificate accepted? >>>> That should normally have done the trick. However, the sources tell me >>>> that no ssl_client settings are propagated to the http_client used by >>>> fts-solr, so SSL is not currently supported it seems. >>>> >>>> I'll check how easy it is to add that. >>> >>> Just to keep you informed: I created a patch, but it is still being >>> tested. >>> >> >> Thanks for the update Stephan! Awesome! Looking forward to test it >> myself :-) > > https://github.com/dovecot/core/commit/526631052ca3175357302af8fa7dcbf763b40c53 > Thank you. I am using now the following version: 2.3.0.alpha0 (2eeea57) [XI:2:2.3.0~alpha0-1~auto+650] The error messages I am getting now are like this: doveadm(user at host): Info: Received invalid SSL certificate: unable to get local issuer certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 doveadm(user at host): Error: fts_solr: Lookup failed: 9002 SSL handshaking with 5.45.106.248:443 failed: read(SSL 5.45.106.248:443) failed: Received invalid SSL certificate: unable to get local issuer certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 You can connect to 5.45.106.248:443 and IMHO everything is correct with the chain. I am no SSL expert, but I am reading it as "doveadm and its ssl part cannot verify the Let's Encrypt certificate". It would need the DST Root CA X3 and this is in the local trust store (ssl_client_ca_dir...) Do you have another hint maybe? Thanks in advance and good night, Jan :-) -- Jan Vonde Hermann-Rein-Str. 6 37075 G?ttingen Tel: 0551 - 200 47 58 2 Mobil: 0176 - 83 110 775 http://www.vonde.eu From dmiller at amfes.com Thu Feb 9 05:54:32 2017 From: dmiller at amfes.com (Daniel Miller) Date: Wed, 8 Feb 2017 21:54:32 -0800 Subject: Solr 6.4.1 Message-ID: I've been running Solr for a while (4.10.3) - wanted to make the jump to the latest & greatest. I installed 6.4.1, copied over my schema.xml - and after a couple false starts where I needed to tweak it work with the new version...it works! I did not copy the database, started from scratch, and executed a "doveadm fts rescan -A". But... Judging solely from at least one client - it's fine. But looking in the logs I see: 1. The first scan of a mailbox dovecot's error log gives: dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: Bad Request 2. Subsequent scans do not appear to generate any dovecot error logs - but I'm not certain. Each new mailbox/subfolder scanned will each have one error on the initial scan. 3. Solr's log gives me the following - on every search. 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad contentType for search handler :text/xml request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} at org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:72) at org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:163) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2306) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:296) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 2017-02-09 05:50:12.412 INFO (qtp205125520-15) [ x:dovecot] o.a.s.c.S.Request [dovecot] webapp=/solr path=/select params={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} status=400 QTime=1 My managed-schema file is: id -- Daniel From aki.tuomi at dovecot.fi Thu Feb 9 06:57:50 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Thu, 9 Feb 2017 08:57:50 +0200 Subject: Solr 6.4.1 In-Reply-To: References: Message-ID: On 09.02.2017 07:54, Daniel Miller wrote: > I've been running Solr for a while (4.10.3) - wanted to make the jump > to the latest & greatest. I installed 6.4.1, copied over my > schema.xml - and after a couple false starts where I needed to tweak > it work with the new version...it works! I did not copy the database, > started from scratch, and executed a "doveadm fts rescan -A". But... > > Judging solely from at least one client - it's fine. But looking in > the logs I see: > 1. The first scan of a mailbox dovecot's error log gives: > dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: > Bad Request > > 2. Subsequent scans do not appear to generate any dovecot error logs > - but I'm not certain. Each new mailbox/subfolder scanned will each > have one error on the initial scan. > > 3. Solr's log gives me the following - on every search. > 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad > contentType for search handler :text/xml > request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} Hi! can you please use tcpdump or wireshark to capture the actual HTTP request causing this exception? Aki From 24x7server at 24x7server.net Thu Feb 9 10:30:22 2017 From: 24x7server at 24x7server.net (Rajesh M) Date: Thu, 9 Feb 2017 16:00:22 +0530 Subject: dovecot logout issues Message-ID: <3112B2DB02AA4C97871A9E3C19D6C9B8.MAI@ns1.24x7server.net> hi we are using dovecot version 2.2.7 (config file given below) centos 6, qmail, vpopmail, mysql server configuration hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 gb hdd for data (No raid) busy server with around 4000 email ids --- load is around 2 to 10 the issue is that SQUIRRELMAIL webmail users suddenly lose connection while they are working on the webmail. after logging in, if the user tries to open a mail then the interface gives error invalid user id or password. this happens on an extremely random basis. also htop always shows a few delayed processes of dovecot (shown as D) on top -- pop3 and imap dovecot logs do not show any login error when such a logout takes place. the said mailbox contained just around 30 emails and it is not related to the timeout plugin of squirrelmail either since the same webmail folders works on other servers of ours without any issues. webmail load slowly in general however when it works normally webmail is very fast and able to handle several 10 s of thousands of emails in the inbox. ram consumed is 2 - 5 gb during peak hours. rebooted server but issue not solved issue is present for the last around 1 month and was not present earlier. help required please. thanks rajesh settings as such # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) # NOTE: Send doveconf -n output instead when asking for help. auth_anonymous_username = anonymous auth_cache_negative_ttl = 0 auth_cache_size = 0 auth_cache_ttl = 0 auth_debug = no auth_debug_passwords = yes auth_default_realm = auth_failure_delay = 2 secs auth_gssapi_hostname = auth_krb5_keytab = auth_master_user_separator = auth_mechanisms = plain login digest-md5 cram-md5 auth_proxy_self = auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = %Lu auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 1 mins default_internal_user = dovecot default_login_user = vpopmail default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins director_username_hash = %u disable_plaintext_auth = no dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 dsync_alt_char = _ dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U first_valid_gid = 89 first_valid_uid = 89 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = name * imap_idle_notify_interval = 2 mins imap_logout_format = in=%i out=%o imap_max_line_length = 64 k imap_metadata = no imap_urlauth_host = imap_urlauth_logout_format = in=%i out=%o imap_urlauth_port = 143 imapc_features = imapc_host = imapc_list_prefix = imapc_master_user = imapc_max_idle_time = 29 mins imapc_password = imapc_port = 143 imapc_rawlog_dir = imapc_ssl = no imapc_ssl_verify = yes imapc_user = import_environment = TZ DEBUG_OUTOFMEM info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/libexec/dovecot listen = *, :: lmtp_address_translate = lmtp_proxy = no lmtp_rcpt_check_quota = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = /var/log/dovecot.log log_timestamp = "%b %d %H:%M:%S " login_access_sockets = login_greeting = ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c session=<%{session}> login_trusted_networks = mail_access_groups = mail_always_cache_fields = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_attribute_dict = mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = mail_home = mail_location = mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib64/dovecot mail_plugins = " quota" mail_prefetch_count = 0 mail_privileged_group = mail_save_crlf = no mail_shared_explicit_inbox = no mail_temp_dir = /tmp mail_temp_scan_interval = 1 weeks mail_uid = mailbox_idle_check_interval = 30 secs mailbox_list_index = no maildir_broken_filename_sizes = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no managesieve_client_workarounds = managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_md5 = apop3d mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no namespace { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = cache_key=%u webmail=127.0.0.1 default_fields = deny = no driver = vpopmail master = no override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never } plugin { quota = maildir:ignore=Trash quota_rule = ?:storage=0 } pop3_client_workarounds = pop3_deleted_flag = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_duplicates = allow pop3_uidl_format = %08Xu%08Xv pop3c_host = pop3c_master_user = pop3c_password = pop3c_port = 110 pop3c_rawlog_dir = pop3c_ssl = no pop3c_ssl_verify = yes pop3c_user = %u postmaster_address = protocols = imap pop3 quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s replication_full_sync_interval = 1 days replication_max_conns = 10 replicator_host = replicator replicator_port = 0 sendmail_path = /usr/sbin/sendmail service aggregator { chroot = . client_limit = 0 drop_priv_before_exec = no executable = aggregator extra_groups = fifo_listener replication-notify-fifo { group = mode = 0600 user = } group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener replication-notify { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 0 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = $default_internal_user } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0666 user = $default_internal_user } unix_listener login/login { group = mode = 0666 user = } unix_listener token-login/tokenlogin { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = . client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 reuse_port = no ssl = no } inet_listener imaps { address = port = 993 reuse_port = no ssl = yes } privileged_group = process_limit = 256 process_min_avail = 50 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap-urlauth-login { chroot = token-login client_limit = 0 drop_priv_before_exec = no executable = imap-urlauth-login extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login unix_listener imap-urlauth { group = mode = 0666 user = } user = $default_login_user vsz_limit = 18446744073709551615 B } service imap-urlauth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap-urlauth-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener imap-urlauth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service imap-urlauth { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap-urlauth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener token-login/imap-urlauth { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 2048 process_min_avail = 50 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 512 M } service indexer-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = indexer-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 10 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service indexer { chroot = client_limit = 0 drop_priv_before_exec = no executable = indexer extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log unix_listener log-errors { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service managesieve-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = managesieve-login extra_groups = group = idle_kill = 0 inet_listener sieve { address = port = 4190 reuse_port = no ssl = no } privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service managesieve { chroot = client_limit = 1 drop_priv_before_exec = no executable = managesieve extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = unix_listener login/sieve { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 reuse_port = no ssl = no } inet_listener pop3s { address = port = 995 reuse_port = no ssl = yes } privileged_group = process_limit = 256 process_min_avail = 25 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 256 process_min_avail = 25 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service replicator { chroot = client_limit = 0 drop_priv_before_exec = no executable = replicator extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener replicator-doveadm { group = mode = 00 user = $default_internal_user } unix_listener replicator { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } unix_listener ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0600 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = References: <1486420747.1665.4.camel@degga.net> <20170207082259.406481aa@pennywise.havleik.net> Message-ID: <00905D2C-F645-424B-AA31-92E36318AC5F@degga.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Well for other mailing lists I have noticed that a lot of lists add text to the body or subject saying what list the email is from which would cause the signature not to match. But the dovecot list doesn't do that so that's why I found it strange that so many emails fail dkim. - -- Steven Mainor On February 7, 2017 2:22:59 AM EST, "Bj?rn T Johansen" wrote: >On Mon, 06 Feb 2017 17:39:07 -0500 >Steven Mainor wrote: > >> Hello, >> >> It seems that I get several emails a week from this list in my spam >> folder. Usually because the DKIM signature fails. Has anyone else >> noticed this problem or is it just me? >> >> -- >> Steven mainor >> steve at degga.net > >Yes, same here... Having the same problem on other maillists as well, >not sure why. > >BTJ -----BEGIN PGP SIGNATURE----- Version: APG v1.1.1 iQI9BAEBCgAnBQJYnE7hIBxTdGV2ZW4gTWFpbm9yIDxzdGV2ZUBkZWdnYS5uZXQ+ AAoJEHEwUCz4iIxrOVcP/A6eW2CqDs6wl53cPRgBszBVCJeoiWPesdR2ZTBEK/k3 agFgeBoknDLxlsAqnqDpi8vAVOjc1DvUTAFDs0btp8e5qirFjwzSMy74PpdBAWWJ m7B/8g69TnhDwYi9RQITJ1Ev5gUX/S5BqpV7zFjsr5BP1fhPEEwDYlnZLWrFXe+e Z3N/skPKvr9Obyd+dREiL9OVjrU+2SNLnbLjPn69JKDQZxarCq2wqz6DTJ+YAtRs ompbEgjAFTQ2p/618KQeOKRy59BIDWKp0TB797FiWYCjwVS0M9+vuecro2otplj6 xHWPhkF9paoZzxi5YXpFH/M5rXwg52jaUJWLVEiPszANHVgkZcolPkf71JcLOU01 H5LG66qXdy7spDjKvbXA0vYRzB/BaviJi4TIk/gK2qQpalXcz00G1EDmSQA7XqA1 uXf2TOpfIOQYjzN2YajtNM1+fnQUHw65SWC/vVEuN3nbjaNsmNSrTwximG5J5JZK ffYzr934TCXw65l+WIJS4nUafJKyuX7BKmz6kBcBQOr8Rw3udxiMzyC4ZelTLBw+ akj88JlHbYA2H2nA8xAsqwVxGOfJnvxb9IDiGwJyI0WoL4CRIcpZVyNelsrJMRZg v7U4wRKX7J485PSB9NDpQTGpDkUJCjtmWZx7cJpYQbiTHZiCJ4cMJyNO/T/nywvR =XJnU -----END PGP SIGNATURE----- From yacinechaouche at yahoo.com Thu Feb 9 12:48:40 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Thu, 9 Feb 2017 12:48:40 +0000 (UTC) Subject: Maildirsize not updated References: <824227861.1251135.1486644520643.ref@mail.yahoo.com> Message-ID: <824227861.1251135.1486644520643@mail.yahoo.com> Hello dovecot Problem : maildirsize and dovecot quota get have different values dovecot version : 2.2.13 happens with : some mailboxes This issue happens only to some mailboxes and only for a specific period of time until maildirsize gets updated by something (not me). I can't reproduce this now because it happened this morning and now its 2 PM and the problem is gone (maildirsize was updated). Anybody experienced the same situation ? -- Yassine. From aki.tuomi at dovecot.fi Thu Feb 9 13:02:14 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Thu, 9 Feb 2017 15:02:14 +0200 Subject: Maildirsize not updated In-Reply-To: <824227861.1251135.1486644520643@mail.yahoo.com> References: <824227861.1251135.1486644520643.ref@mail.yahoo.com> <824227861.1251135.1486644520643@mail.yahoo.com> Message-ID: <21b1ac9e-43fb-32e7-5ca8-e6267491eb38@dovecot.fi> how do you measure the difference? dovecot quota takes only into consideration either the physical or virtual size of the files. Aki On 09.02.2017 14:48, chaouche yacine wrote: > Hello dovecot > > Problem : maildirsize and dovecot quota get have different values > > dovecot version : 2.2.13 > happens with : some mailboxes > > > This issue happens only to some mailboxes and only for a specific period of time until maildirsize gets updated by something (not me). I can't reproduce this now because it happened this morning and now its 2 PM and the problem is gone (maildirsize was updated). > > Anybody experienced the same situation ? > > > -- Yassine. From yacinechaouche at yahoo.com Thu Feb 9 13:20:10 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Thu, 9 Feb 2017 13:20:10 +0000 (UTC) Subject: Maildirsize not updated In-Reply-To: <21b1ac9e-43fb-32e7-5ca8-e6267491eb38@dovecot.fi> References: <824227861.1251135.1486644520643.ref@mail.yahoo.com> <824227861.1251135.1486644520643@mail.yahoo.com> <21b1ac9e-43fb-32e7-5ca8-e6267491eb38@dovecot.fi> Message-ID: <758119982.1220532.1486646410142@mail.yahoo.com> On Thursday, February 9, 2017 2:02 PM, Aki Tuomi wrote: > how do you measure the difference? header of maildirsize and output of doveadm quota get -u user at domain.tld Here's another occurence that happened a couple weeks ago on 25-01-2017 (retrieved from a mail I sent to the user and to teammates): in some unit in % of quota in Mb maildirsize value : 785744731 73% 749.34 doveadm quota get value : 597777 57% 583.76 command line traces : 1) doveadm quota get root at messagerie[10.10.10.19] ~/SCRIPTS/MAIL # doveadm quota get -u hakim.b at domain.tld Quota name Type Value Limit % User quota STORAGE 597777 1048576 57 User quota MESSAGE 7210 - 0 root at messagerie[10.10.10.19] ~/SCRIPTS/MAIL # 2) maildirsize header : root at messagerie[10.10.10.19] /var/vmail/domain.tld/hakim.b # head maildirsize 1073741824S 785744731 7414 420741 1 420854 1 420567 1 420632 1 420607 1 420721 1 421077 1 420863 1 root at messagerie[10.10.10.19] /var/vmail/domain.tld/hakim.b # From rke at xxxlgroup.com Thu Feb 9 15:33:55 2017 From: rke at xxxlgroup.com (Riecken Jens) Date: Thu, 9 Feb 2017 15:33:55 +0000 Subject: Quota error for disabled namespace Message-ID: <195FDE37AE77AB419A45AE1F0392751D9E4CEBE2@SRV00-EX2010-02.lutz.gmbh> Hello, Some of our users use a dedicated archive mailbox, so we've setup an archive namespace for it, which is by default disabled. This namespace has its own quota which is defined by quota2* settings. Every time a user without archive authenticates, dsync kicks in, lmtp delivers a message or imap is used an Error message gets recorded: "imap(user1): Error: quota: Unknown namespace: Archive/" This happens about 441,756 times per hour - putting heavy load on the whole logging chain. In my opinion it makes sense suppressing quota error messages for disabled namespaces or flag it as debug messages. I'm thankful for every solution for this problem! Best regards Jens Environment: Dovecot Version 2.2.27 (c0f36b0) Red Hat Enterprise Linux Server release 7.3 (Maipo) Linux 3.10.0-514.6.1.el7.x86_64 doveadm user '*' |wc -l: 15946 Relevant configuration sections: namespace archive { disabled = yes hidden = no inbox = no list = yes location = maildir:/archive/%h/Archive mailbox "Auto Archive" { auto = subscribe special_use = \Archive } prefix = Archive/ separator = / subscriptions = no type = private } The archive gets enabled by the authentication backend: user_attrs = ou=home=/mail/%U$/%u,=uid=500,=gid=500,quota=quota_rule=*:bytes=%{ldap:quota},archiveDisabled=namespace/archive/disabled=%{ldap:archiveDisabled:yes} To ensure that the archive storage stays within sane boundaries there is a quota defined for the archive: plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = proxy::acl mail_replica = tcp:mail-replica:12345 quota = maildir:User_quota quota2 = maildir:Archive_quota:ns=Archive/ quota2_rule = *:storage=10G quota_rule = *:storage=2G quota_rule2 = INBOX/Trash:storage=+10%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=85%% quota-warning 85 %u quota_warning4 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } From nlekkas at gmail.com Thu Feb 9 15:45:56 2017 From: nlekkas at gmail.com (Nick Lekkas) Date: Thu, 9 Feb 2017 17:45:56 +0200 Subject: dovecot Digest, Vol 166, Issue 22 In-Reply-To: References: Message-ID: Hello to all After a lot of tries using dovecot 2.2.10 on centos 7 latest ...i could not manage make dovecot dsync work ... i have tried a lot of scenarios using mysql as backend , custom user file. Using by command line the dsync it works fine Has anyone managed to use 2.2.10 ..? -----Original Message----- From: dovecot [mailto:dovecot-bounces at dovecot.org] On Behalf Of dovecot-request at dovecot.org Sent: Thursday, February 09, 2017 1:05 PM To: dovecot at dovecot.org Subject: dovecot Digest, Vol 166, Issue 22 Send dovecot mailing list submissions to dovecot at dovecot.org To subscribe or unsubscribe via the World Wide Web, visit http://dovecot.org/cgi-bin/mailman/listinfo/dovecot or, via email, send a message with subject or body 'help' to dovecot-request at dovecot.org You can reach the person managing the list at dovecot-owner at dovecot.org When replying, please edit your Subject line so it is more specific than "Re: Contents of dovecot digest..." Today's Topics: 1. Re: Solr 6.4.1 (Aki Tuomi) 2. dovecot logout issues (Rajesh M) 3. Re: Messages on this list are often marked as spam. (Steven Mainor) ---------------------------------------------------------------------- Message: 1 Date: Thu, 9 Feb 2017 08:57:50 +0200 From: Aki Tuomi To: dovecot at dovecot.org Subject: Re: Solr 6.4.1 Message-ID: Content-Type: text/plain; charset=utf-8 On 09.02.2017 07:54, Daniel Miller wrote: > I've been running Solr for a while (4.10.3) - wanted to make the jump > to the latest & greatest. I installed 6.4.1, copied over my > schema.xml - and after a couple false starts where I needed to tweak > it work with the new version...it works! I did not copy the database, > started from scratch, and executed a "doveadm fts rescan -A". But... > > Judging solely from at least one client - it's fine. But looking in > the logs I see: > 1. The first scan of a mailbox dovecot's error log gives: > dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: > Bad Request > > 2. Subsequent scans do not appear to generate any dovecot error logs > - but I'm not certain. Each new mailbox/subfolder scanned will each > have one error on the initial scan. > > 3. Solr's log gives me the following - on every search. > 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad > contentType for search handler :text/xml > request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:" test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+% 2Buser:"dmiller at amfes.com"&rows=67135} Hi! can you please use tcpdump or wireshark to capture the actual HTTP request causing this exception? Aki ------------------------------ Message: 2 Date: Thu, 9 Feb 2017 16:00:22 +0530 From: "Rajesh M" <24x7server at 24x7server.net> To: Subject: dovecot logout issues Message-ID: <3112B2DB02AA4C97871A9E3C19D6C9B8.MAI at ns1.24x7server.net> Content-Type: text/plain; charset="UTF-8" hi we are using dovecot version 2.2.7 (config file given below) centos 6, qmail, vpopmail, mysql server configuration hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 gb hdd for data (No raid) busy server with around 4000 email ids --- load is around 2 to 10 the issue is that SQUIRRELMAIL webmail users suddenly lose connection while they are working on the webmail. after logging in, if the user tries to open a mail then the interface gives error invalid user id or password. this happens on an extremely random basis. also htop always shows a few delayed processes of dovecot (shown as D) on top -- pop3 and imap dovecot logs do not show any login error when such a logout takes place. the said mailbox contained just around 30 emails and it is not related to the timeout plugin of squirrelmail either since the same webmail folders works on other servers of ours without any issues. webmail load slowly in general however when it works normally webmail is very fast and able to handle several 10 s of thousands of emails in the inbox. ram consumed is 2 - 5 gb during peak hours. rebooted server but issue not solved issue is present for the last around 1 month and was not present earlier. help required please. thanks rajesh settings as such # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) # NOTE: Send doveconf -n output instead when asking for help. auth_anonymous_username = anonymous auth_cache_negative_ttl = 0 auth_cache_size = 0 auth_cache_ttl = 0 auth_debug = no auth_debug_passwords = yes auth_default_realm = auth_failure_delay = 2 secs auth_gssapi_hostname = auth_krb5_keytab = auth_master_user_separator = auth_mechanisms = plain login digest-md5 cram-md5 auth_proxy_self = auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = %Lu auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 1 mins default_internal_user = dovecot default_login_user = vpopmail default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins director_username_hash = %u disable_plaintext_auth = no dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 dsync_alt_char = _ dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U first_valid_gid = 89 first_valid_uid = 89 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = name * imap_idle_notify_interval = 2 mins imap_logout_format = in=%i out=%o imap_max_line_length = 64 k imap_metadata = no imap_urlauth_host = imap_urlauth_logout_format = in=%i out=%o imap_urlauth_port = 143 imapc_features = imapc_host = imapc_list_prefix = imapc_master_user = imapc_max_idle_time = 29 mins imapc_password = imapc_port = 143 imapc_rawlog_dir = imapc_ssl = no imapc_ssl_verify = yes imapc_user = import_environment = TZ DEBUG_OUTOFMEM info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/libexec/dovecot listen = *, :: lmtp_address_translate = lmtp_proxy = no lmtp_rcpt_check_quota = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = /var/log/dovecot.log log_timestamp = "%b %d %H:%M:%S " login_access_sockets = login_greeting = ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c session=<%{session}> login_trusted_networks = mail_access_groups = mail_always_cache_fields = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_attribute_dict = mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = mail_home = mail_location = mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib64/dovecot mail_plugins = " quota" mail_prefetch_count = 0 mail_privileged_group = mail_save_crlf = no mail_shared_explicit_inbox = no mail_temp_dir = /tmp mail_temp_scan_interval = 1 weeks mail_uid = mailbox_idle_check_interval = 30 secs mailbox_list_index = no maildir_broken_filename_sizes = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no managesieve_client_workarounds = managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_md5 = apop3d mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no namespace { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = cache_key=%u webmail=127.0.0.1 default_fields = deny = no driver = vpopmail master = no override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never } plugin { quota = maildir:ignore=Trash quota_rule = ?:storage=0 } pop3_client_workarounds = pop3_deleted_flag = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_duplicates = allow pop3_uidl_format = %08Xu%08Xv pop3c_host = pop3c_master_user = pop3c_password = pop3c_port = 110 pop3c_rawlog_dir = pop3c_ssl = no pop3c_ssl_verify = yes pop3c_user = %u postmaster_address = protocols = imap pop3 quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s replication_full_sync_interval = 1 days replication_max_conns = 10 replicator_host = replicator replicator_port = 0 sendmail_path = /usr/sbin/sendmail service aggregator { chroot = . client_limit = 0 drop_priv_before_exec = no executable = aggregator extra_groups = fifo_listener replication-notify-fifo { group = mode = 0600 user = } group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener replication-notify { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 0 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = $default_internal_user } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0666 user = $default_internal_user } unix_listener login/login { group = mode = 0666 user = } unix_listener token-login/tokenlogin { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = . client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 reuse_port = no ssl = no } inet_listener imaps { address = port = 993 reuse_port = no ssl = yes } privileged_group = process_limit = 256 process_min_avail = 50 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap-urlauth-login { chroot = token-login client_limit = 0 drop_priv_before_exec = no executable = imap-urlauth-login extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login unix_listener imap-urlauth { group = mode = 0666 user = } user = $default_login_user vsz_limit = 18446744073709551615 B } service imap-urlauth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap-urlauth-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener imap-urlauth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service imap-urlauth { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap-urlauth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener token-login/imap-urlauth { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 2048 process_min_avail = 50 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 512 M } service indexer-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = indexer-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 10 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service indexer { chroot = client_limit = 0 drop_priv_before_exec = no executable = indexer extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log unix_listener log-errors { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service managesieve-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = managesieve-login extra_groups = group = idle_kill = 0 inet_listener sieve { address = port = 4190 reuse_port = no ssl = no } privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service managesieve { chroot = client_limit = 1 drop_priv_before_exec = no executable = managesieve extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = unix_listener login/sieve { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 reuse_port = no ssl = no } inet_listener pop3s { address = port = 995 reuse_port = no ssl = yes } privileged_group = process_limit = 256 process_min_avail = 25 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 256 process_min_avail = 25 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service replicator { chroot = client_limit = 0 drop_priv_before_exec = no executable = replicator extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener replicator-doveadm { group = mode = 00 user = $default_internal_user } unix_listener replicator { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } unix_listener ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0600 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = To: Bj?rn T Johansen ,dovecot at dovecot.org Subject: Re: Messages on this list are often marked as spam. Message-ID: <00905D2C-F645-424B-AA31-92E36318AC5F at degga.net> Content-Type: text/plain; charset=UTF-8 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Well for other mailing lists I have noticed that a lot of lists add text to the body or subject saying what list the email is from which would cause the signature not to match. But the dovecot list doesn't do that so that's why I found it strange that so many emails fail dkim. - -- Steven Mainor On February 7, 2017 2:22:59 AM EST, "Bj?rn T Johansen" wrote: >On Mon, 06 Feb 2017 17:39:07 -0500 >Steven Mainor wrote: > >> Hello, >> >> It seems that I get several emails a week from this list in my spam >> folder. Usually because the DKIM signature fails. Has anyone else >> noticed this problem or is it just me? >> >> -- >> Steven mainor >> steve at degga.net > >Yes, same here... Having the same problem on other maillists as well, >not sure why. > >BTJ -----BEGIN PGP SIGNATURE----- Version: APG v1.1.1 iQI9BAEBCgAnBQJYnE7hIBxTdGV2ZW4gTWFpbm9yIDxzdGV2ZUBkZWdnYS5uZXQ+ AAoJEHEwUCz4iIxrOVcP/A6eW2CqDs6wl53cPRgBszBVCJeoiWPesdR2ZTBEK/k3 agFgeBoknDLxlsAqnqDpi8vAVOjc1DvUTAFDs0btp8e5qirFjwzSMy74PpdBAWWJ m7B/8g69TnhDwYi9RQITJ1Ev5gUX/S5BqpV7zFjsr5BP1fhPEEwDYlnZLWrFXe+e Z3N/skPKvr9Obyd+dREiL9OVjrU+2SNLnbLjPn69JKDQZxarCq2wqz6DTJ+YAtRs ompbEgjAFTQ2p/618KQeOKRy59BIDWKp0TB797FiWYCjwVS0M9+vuecro2otplj6 xHWPhkF9paoZzxi5YXpFH/M5rXwg52jaUJWLVEiPszANHVgkZcolPkf71JcLOU01 H5LG66qXdy7spDjKvbXA0vYRzB/BaviJi4TIk/gK2qQpalXcz00G1EDmSQA7XqA1 uXf2TOpfIOQYjzN2YajtNM1+fnQUHw65SWC/vVEuN3nbjaNsmNSrTwximG5J5JZK ffYzr934TCXw65l+WIJS4nUafJKyuX7BKmz6kBcBQOr8Rw3udxiMzyC4ZelTLBw+ akj88JlHbYA2H2nA8xAsqwVxGOfJnvxb9IDiGwJyI0WoL4CRIcpZVyNelsrJMRZg v7U4wRKX7J485PSB9NDpQTGpDkUJCjtmWZx7cJpYQbiTHZiCJ4cMJyNO/T/nywvR =XJnU -----END PGP SIGNATURE----- ------------------------------ Subject: Digest Footer _______________________________________________ dovecot mailing list dovecot at dovecot.org http://dovecot.org/cgi-bin/mailman/listinfo/dovecot ------------------------------ End of dovecot Digest, Vol 166, Issue 22 **************************************** __________ Information from ESET Endpoint Antivirus, version of virus signature database 14908 (20170209) __________ The message was checked by ESET Endpoint Antivirus. part000.txt - is OK http://www.eset.com __________ Information from ESET Endpoint Antivirus, version of virus signature database 14909 (20170209) __________ The message was checked by ESET Endpoint Antivirus. Email message - is OK http://www.eset.com From juri at koschikode.com Thu Feb 9 16:14:01 2017 From: juri at koschikode.com (Juri Haberland) Date: Thu, 9 Feb 2017 17:14:01 +0100 Subject: Messages on this list are often marked as spam. In-Reply-To: <00905D2C-F645-424B-AA31-92E36318AC5F@degga.net> References: <1486420747.1665.4.camel@degga.net> <20170207082259.406481aa@pennywise.havleik.net> <00905D2C-F645-424B-AA31-92E36318AC5F@degga.net> Message-ID: <75d36358-0812-2205-d74f-abad77f12d27@koschikode.com> On 09.02.2017 12:13, Steven Mainor wrote: > Well for other mailing lists I have noticed that a lot of lists add text to the body or subject saying what list the email is from which would cause the signature not to match. > > But the dovecot list doesn't do that so that's why I found it strange that so many emails fail dkim. But it uses MimeDel, presumably to delete the HTML part of some messages thus invalidating the DKIM signature... Juri From sca at andreasschulze.de Thu Feb 9 16:37:39 2017 From: sca at andreasschulze.de (A. Schulze) Date: Thu, 9 Feb 2017 17:37:39 +0100 Subject: Messages on this list are often marked as spam. In-Reply-To: <75d36358-0812-2205-d74f-abad77f12d27@koschikode.com> References: <1486420747.1665.4.camel@degga.net> <20170207082259.406481aa@pennywise.havleik.net> <00905D2C-F645-424B-AA31-92E36318AC5F@degga.net> <75d36358-0812-2205-d74f-abad77f12d27@koschikode.com> Message-ID: Am 09.02.2017 um 17:14 schrieb Juri Haberland: > But it uses MimeDel, presumably to delete the HTML part of some messages > thus invalidating the DKIM signature... > X-Mailman-Version: 2.1.17 and it's using an old version. Newer releases fix some points where Mailman modify messages in subtile ways. Updating to 2.1.23 is suggested.. Andreas From bill-dovecot at carpenter.org Thu Feb 9 18:54:05 2017 From: bill-dovecot at carpenter.org (WJCarpenter) Date: Thu, 9 Feb 2017 10:54:05 -0800 Subject: Maildirsize not updated In-Reply-To: <758119982.1220532.1486646410142@mail.yahoo.com> References: <824227861.1251135.1486644520643.ref@mail.yahoo.com> <824227861.1251135.1486644520643@mail.yahoo.com> <21b1ac9e-43fb-32e7-5ca8-e6267491eb38@dovecot.fi> <758119982.1220532.1486646410142@mail.yahoo.com> Message-ID: <589CBACD.2000908@carpenter.org> Who delievers incoming mail, dovecot LDA or something else? This is what caused a similar problem for me: https://dovecot.org/list/dovecot/2016-April/104091.html From 24x7server at 24x7server.net Thu Feb 9 19:43:20 2017 From: 24x7server at 24x7server.net (Rajesh M) Date: Fri, 10 Feb 2017 01:13:20 +0530 Subject: dovecot config for 1500 simultaneous connection Message-ID: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> hello could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. my server server configuration hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 gb hdd for data (No raid) thanks rajesh my current config file settings as such # 2.2.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) # NOTE: Send doveconf -n output instead when asking for help. auth_anonymous_username = anonymous auth_cache_negative_ttl = 0 auth_cache_size = 0 auth_cache_ttl = 0 auth_debug = no auth_debug_passwords = yes auth_default_realm = auth_failure_delay = 2 secs auth_gssapi_hostname = auth_krb5_keytab = auth_master_user_separator = auth_mechanisms = plain login digest-md5 cram-md5 auth_proxy_self = auth_realms = auth_socket_path = auth-userdb auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_use_winbind = no auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth_username_format = %Lu auth_username_translation = auth_verbose = no auth_verbose_passwords = no auth_winbind_helper_path = /usr/bin/ntlm_auth auth_worker_max_count = 30 base_dir = /var/run/dovecot config_cache_size = 1 M debug_log_path = default_client_limit = 1000 default_idle_kill = 1 mins default_internal_user = dovecot default_login_user = vpopmail default_process_limit = 100 default_vsz_limit = 256 M deliver_log_format = msgid=%m: %$ dict_db_config = director_doveadm_port = 0 director_mail_servers = director_servers = director_user_expire = 15 mins director_username_hash = %u disable_plaintext_auth = no dotlock_use_excl = yes doveadm_allowed_commands = doveadm_password = doveadm_port = 0 doveadm_socket_path = doveadm-server doveadm_worker_count = 0 dsync_alt_char = _ dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U first_valid_gid = 89 first_valid_uid = 89 hostname = imap_capability = imap_client_workarounds = imap_id_log = imap_id_send = name * imap_idle_notify_interval = 2 mins imap_logout_format = in=%i out=%o imap_max_line_length = 64 k imap_metadata = no imap_urlauth_host = imap_urlauth_logout_format = in=%i out=%o imap_urlauth_port = 143 imapc_features = imapc_host = imapc_list_prefix = imapc_master_user = imapc_max_idle_time = 29 mins imapc_password = imapc_port = 143 imapc_rawlog_dir = imapc_ssl = no imapc_ssl_verify = yes imapc_user = import_environment = TZ DEBUG_OUTOFMEM info_log_path = instance_name = dovecot last_valid_gid = 0 last_valid_uid = 0 lda_mailbox_autocreate = no lda_mailbox_autosubscribe = no lda_original_recipient_header = libexec_dir = /usr/libexec/dovecot listen = *, :: lmtp_address_translate = lmtp_proxy = no lmtp_rcpt_check_quota = no lmtp_save_to_detail_mailbox = no lock_method = fcntl log_path = /var/log/dovecot.log log_timestamp = "%b %d %H:%M:%S " login_access_sockets = login_greeting = ready. login_log_format = %$: %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c session=<%{session}> login_trusted_networks = mail_access_groups = mail_always_cache_fields = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_attribute_dict = mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = mail_home = mail_location = mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/lib64/dovecot mail_plugins = " quota" mail_prefetch_count = 0 mail_privileged_group = mail_save_crlf = no mail_shared_explicit_inbox = no mail_temp_dir = /tmp mail_temp_scan_interval = 1 weeks mail_uid = mailbox_idle_check_interval = 30 secs mailbox_list_index = no maildir_broken_filename_sizes = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no managesieve_client_workarounds = managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_md5 = apop3d mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no namespace { disabled = no hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = cache_key=%u webmail=127.0.0.1 default_fields = deny = no driver = vpopmail master = no override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never } plugin { quota = maildir:ignore=Trash quota_rule = ?:storage=0 } pop3_client_workarounds = pop3_deleted_flag = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_duplicates = allow pop3_uidl_format = %08Xu%08Xv pop3c_host = pop3c_master_user = pop3c_password = pop3c_port = 110 pop3c_rawlog_dir = pop3c_ssl = no pop3c_ssl_verify = yes pop3c_user = %u postmaster_address = protocols = imap pop3 quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s replication_full_sync_interval = 1 days replication_max_conns = 10 replicator_host = replicator replicator_port = 0 sendmail_path = /usr/sbin/sendmail service aggregator { chroot = . client_limit = 0 drop_priv_before_exec = no executable = aggregator extra_groups = fifo_listener replication-notify-fifo { group = mode = 0600 user = } group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener replication-notify { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 0 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = $default_internal_user } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0666 user = $default_internal_user } unix_listener login/login { group = mode = 0666 user = } unix_listener token-login/tokenlogin { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = . client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 reuse_port = no ssl = no } inet_listener imaps { address = port = 993 reuse_port = no ssl = yes } privileged_group = process_limit = 256 process_min_avail = 50 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap-urlauth-login { chroot = token-login client_limit = 0 drop_priv_before_exec = no executable = imap-urlauth-login extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login unix_listener imap-urlauth { group = mode = 0666 user = } user = $default_login_user vsz_limit = 18446744073709551615 B } service imap-urlauth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap-urlauth-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener imap-urlauth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service imap-urlauth { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap-urlauth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener token-login/imap-urlauth { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 2048 process_min_avail = 50 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 512 M } service indexer-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = indexer-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 10 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service indexer { chroot = client_limit = 0 drop_priv_before_exec = no executable = indexer extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log unix_listener log-errors { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service managesieve-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = managesieve-login extra_groups = group = idle_kill = 0 inet_listener sieve { address = port = 4190 reuse_port = no ssl = no } privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service managesieve { chroot = client_limit = 1 drop_priv_before_exec = no executable = managesieve extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = sieve service_count = 1 type = unix_listener login/sieve { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 reuse_port = no ssl = no } inet_listener pop3s { address = port = 995 reuse_port = no ssl = yes } privileged_group = process_limit = 256 process_min_avail = 25 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 256 process_min_avail = 25 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service replicator { chroot = client_limit = 0 drop_priv_before_exec = no executable = replicator extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener replicator-doveadm { group = mode = 00 user = $default_internal_user } unix_listener replicator { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } unix_listener ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0600 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = References: <91215f50-fff4-b8ad-fa9e-031d18b52d53@amfes.com> Message-ID: Does this work (pcap attached)? Daniel On 2/8/2017 10:57 PM, Aki Tuomi wrote: > > On 09.02.2017 07:54, Daniel Miller wrote: >> I've been running Solr for a while (4.10.3) - wanted to make the jump >> to the latest & greatest. I installed 6.4.1, copied over my >> schema.xml - and after a couple false starts where I needed to tweak >> it work with the new version...it works! I did not copy the database, >> started from scratch, and executed a "doveadm fts rescan -A". But... >> >> Judging solely from at least one client - it's fine. But looking in >> the logs I see: >> 1. The first scan of a mailbox dovecot's error log gives: >> dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: >> Bad Request >> >> 2. Subsequent scans do not appear to generate any dovecot error logs >> - but I'm not certain. Each new mailbox/subfolder scanned will each >> have one error on the initial scan. >> >> 3. Solr's log gives me the following - on every search. >> 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] >> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad >> contentType for search handler :text/xml >> request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} > Hi! > > can you please use tcpdump or wireshark to capture the actual HTTP > request causing this exception? > > Aki -------------- next part -------------- A non-text attachment was scrubbed... Name: solr-cap.pcapng Type: application/octet-stream Size: 15988 bytes Desc: not available URL: From aki.tuomi at dovecot.fi Fri Feb 10 07:12:04 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 10 Feb 2017 09:12:04 +0200 Subject: Solr 6.4.1 In-Reply-To: References: <91215f50-fff4-b8ad-fa9e-031d18b52d53@amfes.com> Message-ID: <192b8a17-598a-9ea6-8557-3ff0572bbe6f@dovecot.fi> Yeah, thanks. It seems that there indeed is content-type header (there should not be). We'll look into it. Aki On 10.02.2017 01:44, Daniel Miller wrote: > Does this work (pcap attached)? > > Daniel > > On 2/8/2017 10:57 PM, Aki Tuomi wrote: >> >> On 09.02.2017 07:54, Daniel Miller wrote: >>> I've been running Solr for a while (4.10.3) - wanted to make the jump >>> to the latest & greatest. I installed 6.4.1, copied over my >>> schema.xml - and after a couple false starts where I needed to tweak >>> it work with the new version...it works! I did not copy the database, >>> started from scratch, and executed a "doveadm fts rescan -A". But... >>> >>> Judging solely from at least one client - it's fine. But looking in >>> the logs I see: >>> 1. The first scan of a mailbox dovecot's error log gives: >>> dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: >>> Bad Request >>> >>> 2. Subsequent scans do not appear to generate any dovecot error logs >>> - but I'm not certain. Each new mailbox/subfolder scanned will each >>> have one error on the initial scan. >>> >>> 3. Solr's log gives me the following - on every search. >>> 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] >>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad >>> contentType for search handler :text/xml >>> request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} >>> >> Hi! >> >> can you please use tcpdump or wireshark to capture the actual HTTP >> request causing this exception? >> >> Aki > From aki.tuomi at dovecot.fi Fri Feb 10 07:13:09 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 10 Feb 2017 09:13:09 +0200 Subject: Solr 6.4.1 In-Reply-To: <192b8a17-598a-9ea6-8557-3ff0572bbe6f@dovecot.fi> References: <91215f50-fff4-b8ad-fa9e-031d18b52d53@amfes.com> <192b8a17-598a-9ea6-8557-3ff0572bbe6f@dovecot.fi> Message-ID: <36bf499b-9e55-d437-1faa-3ce2ff1531a9@dovecot.fi> What dovecot version are you using? On 10.02.2017 09:12, Aki Tuomi wrote: > Yeah, thanks. > > It seems that there indeed is content-type header (there should not be). > We'll look into it. > > Aki > > On 10.02.2017 01:44, Daniel Miller wrote: >> Does this work (pcap attached)? >> >> Daniel >> >> On 2/8/2017 10:57 PM, Aki Tuomi wrote: >>> On 09.02.2017 07:54, Daniel Miller wrote: >>>> I've been running Solr for a while (4.10.3) - wanted to make the jump >>>> to the latest & greatest. I installed 6.4.1, copied over my >>>> schema.xml - and after a couple false starts where I needed to tweak >>>> it work with the new version...it works! I did not copy the database, >>>> started from scratch, and executed a "doveadm fts rescan -A". But... >>>> >>>> Judging solely from at least one client - it's fine. But looking in >>>> the logs I see: >>>> 1. The first scan of a mailbox dovecot's error log gives: >>>> dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: >>>> Bad Request >>>> >>>> 2. Subsequent scans do not appear to generate any dovecot error logs >>>> - but I'm not certain. Each new mailbox/subfolder scanned will each >>>> have one error on the initial scan. >>>> >>>> 3. Solr's log gives me the following - on every search. >>>> 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] >>>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad >>>> contentType for search handler :text/xml >>>> request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} >>>> >>> Hi! >>> >>> can you please use tcpdump or wireshark to capture the actual HTTP >>> request causing this exception? >>> >>> Aki From aki.tuomi at dovecot.fi Fri Feb 10 08:06:25 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 10 Feb 2017 10:06:25 +0200 Subject: Replacement for antispam plugin Message-ID: Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome. https://wiki.dovecot.org/HowTo/AntispamWithSieve --- Aki Tuomi Dovecot oy From tom at whyscream.net Fri Feb 10 08:35:16 2017 From: tom at whyscream.net (Tom Hendrikx) Date: Fri, 10 Feb 2017 09:35:16 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: On 10-02-17 09:06, Aki Tuomi wrote: > Hi! > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with > IMAPSieve. Comments and suggestions are most welcome. > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > Could you elaborate on the fact that it's deprecated? I never saw anything about that? The wiki page wasn't mentioning that it's deprecated since this morning? Kind regards, Tom From aki.tuomi at dovecot.fi Fri Feb 10 08:36:25 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 10 Feb 2017 10:36:25 +0200 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <7db87626-e235-9f05-ee21-0d571d3bf507@dovecot.fi> On 10.02.2017 10:35, Tom Hendrikx wrote: > > On 10-02-17 09:06, Aki Tuomi wrote: >> Hi! >> Since antispam plugin is deprecated and we would really prefer people >> not to use it, we wrote instructions on how to replace it with >> IMAPSieve. Comments and suggestions are most welcome. >> >> https://wiki.dovecot.org/HowTo/AntispamWithSieve >> > Could you elaborate on the fact that it's deprecated? I never saw > anything about that? The wiki page wasn't mentioning that it's > deprecated since this morning? > > Kind regards, > Tom It has not been maintained for ages and we have decided to officially deprecate it now. We believe that IMAPSieve does this much better than this plugin. Aki From chibi at gol.com Fri Feb 10 08:58:58 2017 From: chibi at gol.com (Christian Balzer) Date: Fri, 10 Feb 2017 17:58:58 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> Message-ID: <20170210175858.0079da4d@batzmaru.gol.ad.jp> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > hello > > could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > Be very precise here, you expect to see 1500 as the result of "doveadm who |grep pop3 |wc -l"? Because that implies an ungodly number of POP3 connects per second, given the typically short duration of these. 1500 IMAP connections (note that frequently a client will have more than the INBOX open and thus have more than one session and thus process on the server) are a much easier proposition, provided they are of the typical long lasting type. So can you put a number to your expected logins per second (both protocols)? > my server > > server configuration > hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > gb hdd for data (No raid) > No RAID and no other replication like DRBD? Why would you even bother? How many users/mailboxes in total with what quota? 1500 IMAP sessions will eat up about 3GB alone. You will want more memory, simply to keep all relevant SLAB bits (inodes, dentries) in RAM. If you really have several hundreds logins/s, you're facing several bottlenecks: 1. Login processes themselves (easily fixed by high performance mode) 2. Auth processes (that will depend on your backends, method mostly) 3. Dovecot master process (spawning mail processes) The later is a single-threaded process, so it will benefit from a faster CPU core. It can be dramatically improved by enabling process re-usage, see: http://wiki.dovecot.org/PerformanceTuning However that also means more memory usage. Christian > > thanks > rajesh > [snip] -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From hakon at alstadheim.priv.no Fri Feb 10 10:52:52 2017 From: hakon at alstadheim.priv.no (=?UTF-8?Q?H=c3=a5kon_Alstadheim?=) Date: Fri, 10 Feb 2017 11:52:52 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <32a3c7a6-5156-fff7-6645-dfa9f7791d83@alstadheim.priv.no> Nice. Finally got around to switching. Had to add filter to my dspam pipe (another one I'm overdue for replacing) . Antispam plugin seems to have automatically converted line endings from "mail-type" '\r\n' into linux plain '\n' . Like so: sed -e 's/\r$//' | dspam ... Den 10. feb. 2017 09:06, skrev Aki Tuomi: > Hi! > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with > IMAPSieve. Comments and suggestions are most welcome. > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > > --- > Aki Tuomi > Dovecot oy > From b.sebode at linet-services.de Fri Feb 10 11:11:23 2017 From: b.sebode at linet-services.de (Bastian Sebode) Date: Fri, 10 Feb 2017 12:11:23 +0100 Subject: dovecot logout issues In-Reply-To: <3112B2DB02AA4C97871A9E3C19D6C9B8.MAI@ns1.24x7server.net> References: <3112B2DB02AA4C97871A9E3C19D6C9B8.MAI@ns1.24x7server.net> Message-ID: <86b17125-37c8-7e10-04bb-5b38097d2ded@linet-services.de> Hello Rajesh, > also htop always shows a few delayed processes of dovecot (shown as D) on top -- pop3 and imap > Processes shown as "D" are waiting for Disk. That also explains your high Load on the Server, because every Process waiting for disk increases the Load by 1! Probably your disks are simply to slow for your scenario. Best Regards Bastian Am 09.02.2017 um 11:30 schrieb Rajesh M: > hi > > we are using dovecot version 2.2.7 (config file given below) > centos 6, qmail, vpopmail, mysql > > server configuration > hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 gb hdd for data (No raid) > > busy server with around 4000 email ids --- load is around 2 to 10 > > the issue is that SQUIRRELMAIL webmail users suddenly lose connection while they are working on the webmail. > > after logging in, if the user tries to open a mail then the interface gives error invalid user id or password. > > this happens on an extremely random basis. > > also htop always shows a few delayed processes of dovecot (shown as D) on top -- pop3 and imap > > dovecot logs do not show any login error when such a logout takes place. > > the said mailbox contained just around 30 emails > > and it is not related to the timeout plugin of squirrelmail either since the same webmail folders works on other servers of ours without any issues. > > webmail load slowly in general > > however when it works normally webmail is very fast and able to handle several 10 s of thousands of emails in the inbox. > > ram consumed is 2 - 5 gb during peak hours. > > rebooted server but issue not solved > > issue is present for the last around 1 month and was not present earlier. > > help required please. > > thanks > rajesh > > > settings as such > # 2.2.7: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) > # NOTE: Send doveconf -n output instead when asking for help. > auth_anonymous_username = anonymous > auth_cache_negative_ttl = 0 > auth_cache_size = 0 > auth_cache_ttl = 0 > auth_debug = no > auth_debug_passwords = yes > auth_default_realm = > auth_failure_delay = 2 secs > auth_gssapi_hostname = > auth_krb5_keytab = > auth_master_user_separator = > auth_mechanisms = plain login digest-md5 cram-md5 > auth_proxy_self = > auth_realms = > auth_socket_path = auth-userdb > auth_ssl_require_client_cert = no > auth_ssl_username_from_cert = no > auth_use_winbind = no > auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ > auth_username_format = %Lu > auth_username_translation = > auth_verbose = no > auth_verbose_passwords = no > auth_winbind_helper_path = /usr/bin/ntlm_auth > auth_worker_max_count = 30 > base_dir = /var/run/dovecot > config_cache_size = 1 M > debug_log_path = > default_client_limit = 1000 > default_idle_kill = 1 mins > default_internal_user = dovecot > default_login_user = vpopmail > default_process_limit = 100 > default_vsz_limit = 256 M > deliver_log_format = msgid=%m: %$ > dict_db_config = > director_doveadm_port = 0 > director_mail_servers = > director_servers = > director_user_expire = 15 mins > director_username_hash = %u > disable_plaintext_auth = no > dotlock_use_excl = yes > doveadm_allowed_commands = > doveadm_password = > doveadm_port = 0 > doveadm_socket_path = doveadm-server > doveadm_worker_count = 0 > dsync_alt_char = _ > dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U > first_valid_gid = 89 > first_valid_uid = 89 > hostname = > imap_capability = > imap_client_workarounds = > imap_id_log = > imap_id_send = name * > imap_idle_notify_interval = 2 mins > imap_logout_format = in=%i out=%o > imap_max_line_length = 64 k > imap_metadata = no > imap_urlauth_host = > imap_urlauth_logout_format = in=%i out=%o > imap_urlauth_port = 143 > imapc_features = > imapc_host = > imapc_list_prefix = > imapc_master_user = > imapc_max_idle_time = 29 mins > imapc_password = > imapc_port = 143 > imapc_rawlog_dir = > imapc_ssl = no > imapc_ssl_verify = yes > imapc_user = > import_environment = TZ DEBUG_OUTOFMEM > info_log_path = > instance_name = dovecot > last_valid_gid = 0 > last_valid_uid = 0 > lda_mailbox_autocreate = no > lda_mailbox_autosubscribe = no > lda_original_recipient_header = > libexec_dir = /usr/libexec/dovecot > listen = *, :: > lmtp_address_translate = > lmtp_proxy = no > lmtp_rcpt_check_quota = no > lmtp_save_to_detail_mailbox = no > lock_method = fcntl > log_path = /var/log/dovecot.log > log_timestamp = "%b %d %H:%M:%S " > login_access_sockets = > login_greeting = ready. > login_log_format = %$: %s > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c session=<%{session}> > login_trusted_networks = > mail_access_groups = > mail_always_cache_fields = > mail_attachment_dir = > mail_attachment_fs = sis posix > mail_attachment_hash = %{sha1} > mail_attachment_min_size = 128 k > mail_attribute_dict = > mail_cache_fields = flags > mail_cache_min_mail_count = 0 > mail_chroot = > mail_debug = no > mail_fsync = optimized > mail_full_filesystem_access = no > mail_gid = > mail_home = > mail_location = > mail_log_prefix = "%s(%u): " > mail_max_keyword_length = 50 > mail_max_lock_timeout = 0 > mail_max_userip_connections = 10 > mail_never_cache_fields = imap.envelope > mail_nfs_index = no > mail_nfs_storage = no > mail_plugin_dir = /usr/lib64/dovecot > mail_plugins = " quota" > mail_prefetch_count = 0 > mail_privileged_group = > mail_save_crlf = no > mail_shared_explicit_inbox = no > mail_temp_dir = /tmp > mail_temp_scan_interval = 1 weeks > mail_uid = > mailbox_idle_check_interval = 30 secs > mailbox_list_index = no > maildir_broken_filename_sizes = no > maildir_copy_with_hardlinks = yes > maildir_stat_dirs = no > maildir_very_dirty_syncs = no > managesieve_client_workarounds = > managesieve_implementation_string = Dovecot Pigeonhole > managesieve_logout_format = bytes=%i/%o > managesieve_max_compile_errors = 5 > managesieve_max_line_length = 65536 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave > master_user_separator = > mbox_dirty_syncs = yes > mbox_dotlock_change_timeout = 2 mins > mbox_lazy_writes = yes > mbox_lock_timeout = 5 mins > mbox_md5 = apop3d > mbox_min_index_size = 0 > mbox_read_locks = fcntl > mbox_very_dirty_syncs = no > mbox_write_locks = dotlock fcntl > mdbox_preallocate_space = no > mdbox_rotate_interval = 0 > mdbox_rotate_size = 2 M > mmap_disable = no > namespace { > disabled = no > hidden = no > ignore_on_failure = no > inbox = yes > list = yes > location = > prefix = > separator = . > subscriptions = yes > type = private > } > passdb { > args = cache_key=%u webmail=127.0.0.1 > default_fields = > deny = no > driver = vpopmail > master = no > override_fields = > pass = no > result_failure = continue > result_internalfail = continue > result_success = return-ok > skip = never > } > plugin { > quota = maildir:ignore=Trash > quota_rule = ?:storage=0 > } > pop3_client_workarounds = > pop3_deleted_flag = > pop3_enable_last = no > pop3_fast_size_lookups = no > pop3_lock_session = no > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > pop3_no_flag_updates = no > pop3_reuse_xuidl = no > pop3_save_uidl = no > pop3_uidl_duplicates = allow > pop3_uidl_format = %08Xu%08Xv > pop3c_host = > pop3c_master_user = > pop3c_password = > pop3c_port = 110 > pop3c_rawlog_dir = > pop3c_ssl = no > pop3c_ssl_verify = yes > pop3c_user = %u > postmaster_address = > protocols = imap pop3 > quota_full_tempfail = no > recipient_delimiter = + > rejection_reason = Your message to <%t> was automatically rejected:%n%r > rejection_subject = Rejected: %s > replication_full_sync_interval = 1 days > replication_max_conns = 10 > replicator_host = replicator > replicator_port = 0 > sendmail_path = /usr/sbin/sendmail > service aggregator { > chroot = . > client_limit = 0 > drop_priv_before_exec = no > executable = aggregator > extra_groups = > fifo_listener replication-notify-fifo { > group = > mode = 0600 > user = > } > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener replication-notify { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service anvil { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = anvil > extra_groups = > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 1 > protocol = > service_count = 0 > type = anvil > unix_listener anvil-auth-penalty { > group = > mode = 0600 > user = > } > unix_listener anvil { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service auth-worker { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = auth -w > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener auth-worker { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service auth { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = auth > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener auth-client { > group = > mode = 0600 > user = $default_internal_user > } > unix_listener auth-login { > group = > mode = 0600 > user = $default_internal_user > } > unix_listener auth-master { > group = > mode = 0600 > user = > } > unix_listener auth-userdb { > group = > mode = 0666 > user = $default_internal_user > } > unix_listener login/login { > group = > mode = 0666 > user = > } > unix_listener token-login/tokenlogin { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service config { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = config > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = config > unix_listener config { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service dict { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dict > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dict { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service director { > chroot = . > client_limit = 0 > drop_priv_before_exec = no > executable = director > extra_groups = > fifo_listener login/proxy-notify { > group = > mode = 00 > user = > } > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener director-admin { > group = > mode = 0600 > user = > } > unix_listener login/director { > group = > mode = 00 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service dns_client { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = dns-client > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener dns-client { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service doveadm { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = doveadm-server > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 1 > type = > unix_listener doveadm-server { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service imap-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = imap-login > extra_groups = > group = > idle_kill = 0 > inet_listener imap { > address = > port = 143 > reuse_port = no > ssl = no > } > inet_listener imaps { > address = > port = 993 > reuse_port = no > ssl = yes > } > privileged_group = > process_limit = 256 > process_min_avail = 50 > protocol = imap > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service imap-urlauth-login { > chroot = token-login > client_limit = 0 > drop_priv_before_exec = no > executable = imap-urlauth-login > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = login > unix_listener imap-urlauth { > group = > mode = 0666 > user = > } > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service imap-urlauth-worker { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = imap-urlauth-worker > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = > unix_listener imap-urlauth-worker { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service imap-urlauth { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = imap-urlauth > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1024 > process_min_avail = 0 > protocol = imap > service_count = 1 > type = > unix_listener token-login/imap-urlauth { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service imap { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = imap > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 2048 > process_min_avail = 50 > protocol = imap > service_count = 1 > type = > unix_listener login/imap { > group = > mode = 0666 > user = > } > user = > vsz_limit = 512 M > } > service indexer-worker { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = indexer-worker > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 10 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener indexer-worker { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service indexer { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = indexer > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener indexer { > group = > mode = 0666 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service ipc { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = ipc > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener ipc { > group = > mode = 0600 > user = > } > unix_listener login/ipc-proxy { > group = > mode = 0600 > user = $default_login_user > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > service lmtp { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = lmtp > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = lmtp > service_count = 0 > type = > unix_listener lmtp { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service log { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = log > extra_groups = > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = log > unix_listener log-errors { > group = > mode = 0600 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service managesieve-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = managesieve-login > extra_groups = > group = > idle_kill = 0 > inet_listener sieve { > address = > port = 4190 > reuse_port = no > ssl = no > } > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = sieve > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service managesieve { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = managesieve > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = sieve > service_count = 1 > type = > unix_listener login/sieve { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service pop3-login { > chroot = login > client_limit = 0 > drop_priv_before_exec = no > executable = pop3-login > extra_groups = > group = > idle_kill = 0 > inet_listener pop3 { > address = > port = 110 > reuse_port = no > ssl = no > } > inet_listener pop3s { > address = > port = 995 > reuse_port = no > ssl = yes > } > privileged_group = > process_limit = 256 > process_min_avail = 25 > protocol = pop3 > service_count = 1 > type = login > user = $default_login_user > vsz_limit = 18446744073709551615 B > } > service pop3 { > chroot = > client_limit = 1 > drop_priv_before_exec = no > executable = pop3 > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 256 > process_min_avail = 25 > protocol = pop3 > service_count = 1 > type = > unix_listener login/pop3 { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service replicator { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = replicator > extra_groups = > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener replicator-doveadm { > group = > mode = 00 > user = $default_internal_user > } > unix_listener replicator { > group = > mode = 0600 > user = $default_internal_user > } > user = > vsz_limit = 18446744073709551615 B > } > service ssl-params { > chroot = > client_limit = 0 > drop_priv_before_exec = no > executable = ssl-params > extra_groups = > group = > idle_kill = 0 > privileged_group = > process_limit = 0 > process_min_avail = 0 > protocol = > service_count = 0 > type = startup > unix_listener login/ssl-params { > group = > mode = 0666 > user = > } > unix_listener ssl-params { > group = > mode = 0666 > user = > } > user = > vsz_limit = 18446744073709551615 B > } > service stats { > chroot = empty > client_limit = 0 > drop_priv_before_exec = no > executable = stats > extra_groups = > fifo_listener stats-mail { > group = > mode = 0600 > user = > } > group = > idle_kill = 4294967295 secs > privileged_group = > process_limit = 1 > process_min_avail = 0 > protocol = > service_count = 0 > type = > unix_listener stats { > group = > mode = 0600 > user = > } > user = $default_internal_user > vsz_limit = 18446744073709551615 B > } > shutdown_clients = yes > ssl = yes > ssl_ca = > ssl_cert = ssl_cert_username_field = commonName > ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > ssl_client_ca_dir = > ssl_client_ca_file = > ssl_client_cert = > ssl_client_key = > ssl_crypto_device = > ssl_dh_parameters_length = 2048 > ssl_key = ssl_key_password = > ssl_parameters_regenerate = 0 > ssl_prefer_server_ciphers = no > ssl_protocols = !SSLv2 > ssl_require_crl = yes > ssl_verify_client_cert = no > state_dir = /var/lib/dovecot > stats_command_min_time = 1 mins > stats_domain_min_time = 12 hours > stats_ip_min_time = 12 hours > stats_memory_limit = 16 M > stats_session_min_time = 15 mins > stats_user_min_time = 1 hours > submission_host = > syslog_facility = mail > userdb { > args = cache_key=%u quota_template=quota_rule=*:backend=%q > default_fields = > driver = vpopmail > override_fields = > } > valid_chroot_dirs = > verbose_proctitle = no > verbose_ssl = no > version_ignore = no > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 200 > mail_plugins = " quota imap_quota" > } > protocol pop3 { > mail_max_userip_connections = 40 > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_fast_size_lookups = yes > pop3_lock_session = no > pop3_no_flag_updates = yes > } > -- Bastian Sebode Fachinformatiker Systemintegration LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de LINET in den sozialen Netzwerken: www.twitter.com/linetservices | www.facebook.com/linetservices Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ Gesch?ftsf?hrung: Timo Springmann, Mirko Savic und Moritz Bunkus HR B 9170 Amtsgericht Braunschweig USt-IdNr. DE 259 526 516 From aki.tuomi at dovecot.fi Fri Feb 10 12:14:02 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 10 Feb 2017 14:14:02 +0200 Subject: dovecot Digest, Vol 166, Issue 22 In-Reply-To: References: Message-ID: <7a4a3dae-9267-bc29-3f33-f00cda803a60@dovecot.fi> I have not tried it at all. Was just an suggestion. You can try something like protocol lda { namespace archive { ... } } On 10.02.2017 14:12, Nick Lekkas wrote: > > Hi Aki > > > > Thanks for your answer .... > > What version of dovecot have you tried it ? Mine is 2.2.10 > > > > > > > > > > *From:*Aki Tuomi [mailto:aki.tuomi at dovecot.fi] > *Sent:* Friday, February 10, 2017 2:06 PM > *To:* nlekkas at gmail.com; dovecot at dovecot.org > *Subject:* Re: dovecot Digest, Vol 166, Issue 22 > > > > You could enable it for LDA/LMTP? > > Aki > > On 09.02.2017 17:45, Nick Lekkas wrote: > > Hello to all > > > > > > After a lot of tries using dovecot 2.2.10 on centos 7 latest ...i could not > > > manage make dovecot dsync work ... > > > i have tried a lot of scenarios using mysql as backend , custom user file. > > > Using by command line the dsync it works fine > > > > > > Has anyone managed to use 2.2.10 ..? > > > > > > > > > -----Original Message----- > > > From: dovecot [mailto:dovecot-bounces at dovecot.org] On Behalf Of > > > dovecot-request at dovecot.org > > > Sent: Thursday, February 09, 2017 1:05 PM > > > To: dovecot at dovecot.org > > > Subject: dovecot Digest, Vol 166, Issue 22 > > > > > > Send dovecot mailing list submissions to > > > dovecot at dovecot.org > > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot > > > or, via email, send a message with subject or body 'help' to > > > dovecot-request at dovecot.org > > > > > > You can reach the person managing the list at > > > dovecot-owner at dovecot.org > > > > > > When replying, please edit your Subject line so it is more specific > > > than "Re: Contents of dovecot digest..." > > > > > > > > > Today's Topics: > > > > > > 1. Re: Solr 6.4.1 (Aki Tuomi) > > > 2. dovecot logout issues (Rajesh M) > > > 3. Re: Messages on this list are often marked as spam. > > > (Steven Mainor) > > > > > > > > > ---------------------------------------------------------------------- > > > > > > Message: 1 > > > Date: Thu, 9 Feb 2017 08:57:50 +0200 > > > From: Aki Tuomi > > > To: dovecot at dovecot.org > > > Subject: Re: Solr 6.4.1 > > > Message-ID: > > > > Content-Type: text/plain; charset=utf-8 > > > > > > > > > > > > On 09.02.2017 07:54, Daniel Miller wrote: > > >> I've been running Solr for a while (4.10.3) - wanted to make the jump > > >> to the latest & greatest. I installed 6.4.1, copied over my > > >> schema.xml - and after a couple false starts where I needed to tweak > > >> it work with the new version...it works! I did not copy the database, > > >> started from scratch, and executed a "doveadm fts rescan -A". But... > > >> > > >> Judging solely from at least one client - it's fine. But looking in > > >> the logs I see: > > >> 1. The first scan of a mailbox dovecot's error log gives: > > >> dovecot: imap(dmiller at amfes.com ): Error: > fts_solr: Lookup failed: > > >> Bad Request > > >> > > >> 2. Subsequent scans do not appear to generate any dovecot error logs > > >> - but I'm not certain. Each new mailbox/subfolder scanned will each > > >> have one error on the initial scan. > > >> > > >> 3. Solr's log gives me the following - on every search. > > >> 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] > > >> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad > > >> contentType for search handler :text/xml > > >> > > > request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:" > > > test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+% > > > 2Buser:"dmiller at amfes.com" &rows=67135} > > > > > > Hi! > > > > > > can you please use tcpdump or wireshark to capture the actual HTTP > > > request causing this exception? > > > > > > Aki > > > > > > > > > ------------------------------ > > > > > > Message: 2 > > > Date: Thu, 9 Feb 2017 16:00:22 +0530 > > > From: "Rajesh M" <24x7server at 24x7server.net> > > > To: > > > Subject: dovecot logout issues > > > Message-ID: <3112B2DB02AA4C97871A9E3C19D6C9B8.MAI at ns1.24x7server.net> > > > > Content-Type: text/plain; charset="UTF-8" > > > > > > hi > > > > > > we are using dovecot version 2.2.7 (config file given below) > > > centos 6, qmail, vpopmail, mysql > > > > > > server configuration > > > hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X > > > 2000 gb hdd for data (No raid) > > > > > > busy server with around 4000 email ids --- load is around 2 to 10 > > > > > > the issue is that SQUIRRELMAIL webmail users suddenly lose connection while > > > they are working on the webmail. > > > > > > after logging in, if the user tries to open a mail then the interface gives > > > error invalid user id or password. > > > > > > this happens on an extremely random basis. > > > > > > also htop always shows a few delayed processes of dovecot (shown as D) on > > > top -- pop3 and imap > > > > > > dovecot logs do not show any login error when such a logout takes place. > > > > > > the said mailbox contained just around 30 emails > > > > > > and it is not related to the timeout plugin of squirrelmail either since the > > > same webmail folders works on other servers of ours without any issues. > > > > > > webmail load slowly in general > > > > > > however when it works normally webmail is very fast and able to handle > > > several 10 s of thousands of emails in the inbox. > > > > > > ram consumed is 2 - 5 gb during peak hours. > > > > > > rebooted server but issue not solved > > > > > > issue is present for the last around 1 month and was not present earlier. > > > > > > help required please. > > > > > > thanks > > > rajesh > > > > > > > > > settings as such > > > # 2.2.7: /etc/dovecot/dovecot.conf > > > # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) > > > # NOTE: Send doveconf -n output instead when asking for help. > > > auth_anonymous_username = anonymous > > > auth_cache_negative_ttl = 0 > > > auth_cache_size = 0 > > > auth_cache_ttl = 0 > > > auth_debug = no > > > auth_debug_passwords = yes > > > auth_default_realm = > > > auth_failure_delay = 2 secs > > > auth_gssapi_hostname = > > > auth_krb5_keytab = > > > auth_master_user_separator = > > > auth_mechanisms = plain login digest-md5 cram-md5 > > > auth_proxy_self = > > > auth_realms = > > > auth_socket_path = auth-userdb > > > auth_ssl_require_client_cert = no > > > auth_ssl_username_from_cert = no > > > auth_use_winbind = no > > > auth_username_chars = > > > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ > > > auth_username_format = %Lu > > > auth_username_translation = > > > auth_verbose = no > > > auth_verbose_passwords = no > > > auth_winbind_helper_path = /usr/bin/ntlm_auth > > > auth_worker_max_count = 30 > > > base_dir = /var/run/dovecot > > > config_cache_size = 1 M > > > debug_log_path = > > > default_client_limit = 1000 > > > default_idle_kill = 1 mins > > > default_internal_user = dovecot > > > default_login_user = vpopmail > > > default_process_limit = 100 > > > default_vsz_limit = 256 M > > > deliver_log_format = msgid=%m: %$ > > > dict_db_config = > > > director_doveadm_port = 0 > > > director_mail_servers = > > > director_servers = > > > director_user_expire = 15 mins > > > director_username_hash = %u > > > disable_plaintext_auth = no > > > dotlock_use_excl = yes > > > doveadm_allowed_commands = > > > doveadm_password = > > > doveadm_port = 0 > > > doveadm_socket_path = doveadm-server > > > doveadm_worker_count = 0 > > > dsync_alt_char = _ > > > dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U > > > first_valid_gid = 89 > > > first_valid_uid = 89 > > > hostname = > > > imap_capability = > > > imap_client_workarounds = > > > imap_id_log = > > > imap_id_send = name * > > > imap_idle_notify_interval = 2 mins > > > imap_logout_format = in=%i out=%o > > > imap_max_line_length = 64 k > > > imap_metadata = no > > > imap_urlauth_host = > > > imap_urlauth_logout_format = in=%i out=%o > > > imap_urlauth_port = 143 > > > imapc_features = > > > imapc_host = > > > imapc_list_prefix = > > > imapc_master_user = > > > imapc_max_idle_time = 29 mins > > > imapc_password = > > > imapc_port = 143 > > > imapc_rawlog_dir = > > > imapc_ssl = no > > > imapc_ssl_verify = yes > > > imapc_user = > > > import_environment = TZ DEBUG_OUTOFMEM > > > info_log_path = > > > instance_name = dovecot > > > last_valid_gid = 0 > > > last_valid_uid = 0 > > > lda_mailbox_autocreate = no > > > lda_mailbox_autosubscribe = no > > > lda_original_recipient_header = > > > libexec_dir = /usr/libexec/dovecot > > > listen = *, :: > > > lmtp_address_translate = > > > lmtp_proxy = no > > > lmtp_rcpt_check_quota = no > > > lmtp_save_to_detail_mailbox = no > > > lock_method = fcntl > > > log_path = /var/log/dovecot.log > > > log_timestamp = "%b %d %H:%M:%S " > > > login_access_sockets = > > > login_greeting = ready. > > > login_log_format = %$: %s > > > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > > > session=<%{session}> > > > login_trusted_networks = > > > mail_access_groups = > > > mail_always_cache_fields = > > > mail_attachment_dir = > > > mail_attachment_fs = sis posix > > > mail_attachment_hash = %{sha1} > > > mail_attachment_min_size = 128 k > > > mail_attribute_dict = > > > mail_cache_fields = flags > > > mail_cache_min_mail_count = 0 > > > mail_chroot = > > > mail_debug = no > > > mail_fsync = optimized > > > mail_full_filesystem_access = no > > > mail_gid = > > > mail_home = > > > mail_location = > > > mail_log_prefix = "%s(%u): " > > > mail_max_keyword_length = 50 > > > mail_max_lock_timeout = 0 > > > mail_max_userip_connections = 10 > > > mail_never_cache_fields = imap.envelope > > > mail_nfs_index = no > > > mail_nfs_storage = no > > > mail_plugin_dir = /usr/lib64/dovecot > > > mail_plugins = " quota" > > > mail_prefetch_count = 0 > > > mail_privileged_group = > > > mail_save_crlf = no > > > mail_shared_explicit_inbox = no > > > mail_temp_dir = /tmp > > > mail_temp_scan_interval = 1 weeks > > > mail_uid = > > > mailbox_idle_check_interval = 30 secs > > > mailbox_list_index = no > > > maildir_broken_filename_sizes = no > > > maildir_copy_with_hardlinks = yes > > > maildir_stat_dirs = no > > > maildir_very_dirty_syncs = no > > > managesieve_client_workarounds = > > > managesieve_implementation_string = Dovecot Pigeonhole > > > managesieve_logout_format = bytes=%i/%o > > > managesieve_max_compile_errors = 5 > > > managesieve_max_line_length = 65536 > > > managesieve_notify_capability = mailto > > > managesieve_sieve_capability = fileinto reject envelope encoded-character > > > vacation subaddress comparator-i;ascii-numeric relational regex > imap4flags > > > copy include variables body enotify environment mailbox date ihave > > > master_user_separator = > > > mbox_dirty_syncs = yes > > > mbox_dotlock_change_timeout = 2 mins > > > mbox_lazy_writes = yes > > > mbox_lock_timeout = 5 mins > > > mbox_md5 = apop3d > > > mbox_min_index_size = 0 > > > mbox_read_locks = fcntl > > > mbox_very_dirty_syncs = no > > > mbox_write_locks = dotlock fcntl > > > mdbox_preallocate_space = no > > > mdbox_rotate_interval = 0 > > > mdbox_rotate_size = 2 M > > > mmap_disable = no > > > namespace { > > > disabled = no > > > hidden = no > > > ignore_on_failure = no > > > inbox = yes > > > list = yes > > > location = > > > prefix = > > > separator = . > > > subscriptions = yes > > > type = private > > > } > > > passdb { > > > args = cache_key=%u webmail=127.0.0.1 > > > default_fields = > > > deny = no > > > driver = vpopmail > > > master = no > > > override_fields = > > > pass = no > > > result_failure = continue > > > result_internalfail = continue > > > result_success = return-ok > > > skip = never > > > } > > > plugin { > > > quota = maildir:ignore=Trash > > > quota_rule = ?:storage=0 > > > } > > > pop3_client_workarounds = > > > pop3_deleted_flag = > > > pop3_enable_last = no > > > pop3_fast_size_lookups = no > > > pop3_lock_session = no > > > pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s > > > pop3_no_flag_updates = no > > > pop3_reuse_xuidl = no > > > pop3_save_uidl = no > > > pop3_uidl_duplicates = allow > > > pop3_uidl_format = %08Xu%08Xv > > > pop3c_host = > > > pop3c_master_user = > > > pop3c_password = > > > pop3c_port = 110 > > > pop3c_rawlog_dir = > > > pop3c_ssl = no > > > pop3c_ssl_verify = yes > > > pop3c_user = %u > > > postmaster_address = > > > protocols = imap pop3 > > > quota_full_tempfail = no > > > recipient_delimiter = + > > > rejection_reason = Your message to <%t> was automatically rejected:%n%r > > > rejection_subject = Rejected: %s > > > replication_full_sync_interval = 1 days > > > replication_max_conns = 10 > > > replicator_host = replicator > > > replicator_port = 0 > > > sendmail_path = /usr/sbin/sendmail > > > service aggregator { > > > chroot = . > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = aggregator > > > extra_groups = > > > fifo_listener replication-notify-fifo { > > > group = > > > mode = 0600 > > > user = > > > } > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener replication-notify { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service anvil { > > > chroot = empty > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = anvil > > > extra_groups = > > > group = > > > idle_kill = 4294967295 secs > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 1 > > > protocol = > > > service_count = 0 > > > type = anvil > > > unix_listener anvil-auth-penalty { > > > group = > > > mode = 0600 > > > user = > > > } > > > unix_listener anvil { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service auth-worker { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = auth -w > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 1 > > > type = > > > unix_listener auth-worker { > > > group = > > > mode = 0600 > > > user = $default_internal_user > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service auth { > > > chroot = > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = auth > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener auth-client { > > > group = > > > mode = 0600 > > > user = $default_internal_user > > > } > > > unix_listener auth-login { > > > group = > > > mode = 0600 > > > user = $default_internal_user > > > } > > > unix_listener auth-master { > > > group = > > > mode = 0600 > > > user = > > > } > > > unix_listener auth-userdb { > > > group = > > > mode = 0666 > > > user = $default_internal_user > > > } > > > unix_listener login/login { > > > group = > > > mode = 0666 > > > user = > > > } > > > unix_listener token-login/tokenlogin { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service config { > > > chroot = > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = config > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = config > > > unix_listener config { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service dict { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = dict > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener dict { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service director { > > > chroot = . > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = director > > > extra_groups = > > > fifo_listener login/proxy-notify { > > > group = > > > mode = 00 > > > user = > > > } > > > group = > > > idle_kill = 4294967295 secs > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener director-admin { > > > group = > > > mode = 0600 > > > user = > > > } > > > unix_listener login/director { > > > group = > > > mode = 00 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service dns_client { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = dns-client > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener dns-client { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service doveadm { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = doveadm-server > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 1 > > > type = > > > unix_listener doveadm-server { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service imap-login { > > > chroot = login > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = imap-login > > > extra_groups = > > > group = > > > idle_kill = 0 > > > inet_listener imap { > > > address = > > > port = 143 > > > reuse_port = no > > > ssl = no > > > } > > > inet_listener imaps { > > > address = > > > port = 993 > > > reuse_port = no > > > ssl = yes > > > } > > > privileged_group = > > > process_limit = 256 > > > process_min_avail = 50 > > > protocol = imap > > > service_count = 1 > > > type = login > > > user = $default_login_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service imap-urlauth-login { > > > chroot = token-login > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = imap-urlauth-login > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = imap > > > service_count = 1 > > > type = login > > > unix_listener imap-urlauth { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = $default_login_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service imap-urlauth-worker { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = imap-urlauth-worker > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 1024 > > > process_min_avail = 0 > > > protocol = imap > > > service_count = 1 > > > type = > > > unix_listener imap-urlauth-worker { > > > group = > > > mode = 0600 > > > user = $default_internal_user > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service imap-urlauth { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = imap-urlauth > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 1024 > > > process_min_avail = 0 > > > protocol = imap > > > service_count = 1 > > > type = > > > unix_listener token-login/imap-urlauth { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service imap { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = imap > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 2048 > > > process_min_avail = 50 > > > protocol = imap > > > service_count = 1 > > > type = > > > unix_listener login/imap { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = > > > vsz_limit = 512 M > > > } > > > service indexer-worker { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = indexer-worker > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 10 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener indexer-worker { > > > group = > > > mode = 0600 > > > user = $default_internal_user > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service indexer { > > > chroot = > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = indexer > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener indexer { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service ipc { > > > chroot = empty > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = ipc > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener ipc { > > > group = > > > mode = 0600 > > > user = > > > } > > > unix_listener login/ipc-proxy { > > > group = > > > mode = 0600 > > > user = $default_login_user > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service lmtp { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = lmtp > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = lmtp > > > service_count = 0 > > > type = > > > unix_listener lmtp { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service log { > > > chroot = > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = log > > > extra_groups = > > > group = > > > idle_kill = 4294967295 secs > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = log > > > unix_listener log-errors { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service managesieve-login { > > > chroot = login > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = managesieve-login > > > extra_groups = > > > group = > > > idle_kill = 0 > > > inet_listener sieve { > > > address = > > > port = 4190 > > > reuse_port = no > > > ssl = no > > > } > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = sieve > > > service_count = 1 > > > type = login > > > user = $default_login_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service managesieve { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = managesieve > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = sieve > > > service_count = 1 > > > type = > > > unix_listener login/sieve { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service pop3-login { > > > chroot = login > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = pop3-login > > > extra_groups = > > > group = > > > idle_kill = 0 > > > inet_listener pop3 { > > > address = > > > port = 110 > > > reuse_port = no > > > ssl = no > > > } > > > inet_listener pop3s { > > > address = > > > port = 995 > > > reuse_port = no > > > ssl = yes > > > } > > > privileged_group = > > > process_limit = 256 > > > process_min_avail = 25 > > > protocol = pop3 > > > service_count = 1 > > > type = login > > > user = $default_login_user > > > vsz_limit = 18446744073709551615 B > > > } > > > service pop3 { > > > chroot = > > > client_limit = 1 > > > drop_priv_before_exec = no > > > executable = pop3 > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 256 > > > process_min_avail = 25 > > > protocol = pop3 > > > service_count = 1 > > > type = > > > unix_listener login/pop3 { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service replicator { > > > chroot = > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = replicator > > > extra_groups = > > > group = > > > idle_kill = 4294967295 secs > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener replicator-doveadm { > > > group = > > > mode = 00 > > > user = $default_internal_user > > > } > > > unix_listener replicator { > > > group = > > > mode = 0600 > > > user = $default_internal_user > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service ssl-params { > > > chroot = > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = ssl-params > > > extra_groups = > > > group = > > > idle_kill = 0 > > > privileged_group = > > > process_limit = 0 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = startup > > > unix_listener login/ssl-params { > > > group = > > > mode = 0666 > > > user = > > > } > > > unix_listener ssl-params { > > > group = > > > mode = 0666 > > > user = > > > } > > > user = > > > vsz_limit = 18446744073709551615 B > > > } > > > service stats { > > > chroot = empty > > > client_limit = 0 > > > drop_priv_before_exec = no > > > executable = stats > > > extra_groups = > > > fifo_listener stats-mail { > > > group = > > > mode = 0600 > > > user = > > > } > > > group = > > > idle_kill = 4294967295 secs > > > privileged_group = > > > process_limit = 1 > > > process_min_avail = 0 > > > protocol = > > > service_count = 0 > > > type = > > > unix_listener stats { > > > group = > > > mode = 0600 > > > user = > > > } > > > user = $default_internal_user > > > vsz_limit = 18446744073709551615 B > > > } > > > shutdown_clients = yes > > > ssl = yes > > > ssl_ca = > > > ssl_cert = > > ssl_cert_username_field = commonName > > > ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > > > ssl_client_ca_dir = > > > ssl_client_ca_file = > > > ssl_client_cert = > > > ssl_client_key = > > > ssl_crypto_device = > > > ssl_dh_parameters_length = 2048 > > > ssl_key = > > ssl_key_password = > > > ssl_parameters_regenerate = 0 > > > ssl_prefer_server_ciphers = no > > > ssl_protocols = !SSLv2 > > > ssl_require_crl = yes > > > ssl_verify_client_cert = no > > > state_dir = /var/lib/dovecot > > > stats_command_min_time = 1 mins > > > stats_domain_min_time = 12 hours > > > stats_ip_min_time = 12 hours > > > stats_memory_limit = 16 M > > > stats_session_min_time = 15 mins > > > stats_user_min_time = 1 hours > > > submission_host = > > > syslog_facility = mail > > > userdb { > > > args = cache_key=%u quota_template=quota_rule=*:backend=%q > > > default_fields = > > > driver = vpopmail > > > override_fields = > > > } > > > valid_chroot_dirs = > > > verbose_proctitle = no > > > verbose_ssl = no > > > version_ignore = no > > > protocol imap { > > > imap_client_workarounds = delay-newmail > > > mail_max_userip_connections = 200 > > > mail_plugins = " quota imap_quota" > > > } > > > protocol pop3 { > > > mail_max_userip_connections = 40 > > > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > > > pop3_fast_size_lookups = yes > > > pop3_lock_session = no > > > pop3_no_flag_updates = yes > > > } > > > > > > > > > > > > > > > ------------------------------ > > > > > > Message: 3 > > > Date: Thu, 09 Feb 2017 06:13:37 -0500 > > > From: Steven Mainor > > > To: Bj?rn T Johansen > ,dovecot at dovecot.org > > > Subject: Re: Messages on this list are often marked as spam. > > > Message-ID: <00905D2C-F645-424B-AA31-92E36318AC5F at degga.net> > > > > Content-Type: text/plain; charset=UTF-8 > > > > > Well for other mailing lists I have noticed that a lot of lists add > text to > the body or subject saying what list the email is from which would > cause the > signature not to match. > > But the dovecot list doesn't do that so that's why I found it strange that > so many emails fail dkim. > > > > > > > > > > > > ------------------------------ > > > > > > Subject: Digest Footer > > > > > > _______________________________________________ > > > dovecot mailing list > > > dovecot at dovecot.org > > > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot > > > > > > ------------------------------ > > > > > > End of dovecot Digest, Vol 166, Issue 22 > > > **************************************** > > > > > > > > > __________ Information from ESET Endpoint Antivirus, version of virus > > > signature database 14908 (20170209) __________ > > > > > > The message was checked by ESET Endpoint Antivirus. > > > > > > part000.txt - is OK > > > > > > http://www.eset.com > > > > > > > > > > > > > > > __________ Information from ESET Endpoint Antivirus, version of virus > > > signature database 14909 (20170209) __________ > > > > > > The message was checked by ESET Endpoint Antivirus. > > > > > > Email message - is OK > > > > > > http://www.eset.com > > > > > __________ Information from ESET Endpoint Antivirus, version of virus > signature database 14914 (20170210) __________ > > The message was checked by ESET Endpoint Antivirus. > > part000.txt - is OK > part001.htm - is OK > > http://www.eset.com > > > > __________ Information from ESET Endpoint Antivirus, version of virus > signature database 14914 (20170210) __________ > > The message was checked by ESET Endpoint Antivirus. > > Email message - is OK > > http://www.eset.com > From dovecot-list at mohtex.net Fri Feb 10 12:52:38 2017 From: dovecot-list at mohtex.net (Tamsy) Date: Fri, 10 Feb 2017 19:52:38 +0700 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <66a3e5c6-8e87-2641-3529-1d94e22935e6@mohtex.net> Aki Tuomi wrote on 10.02.2017 15:06: > Hi! > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with > IMAPSieve. Comments and suggestions are most welcome. > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > > --- > Aki Tuomi > Dovecot oy We just implemented this according to the wiki. Works great and without any hiccups. Thanks, Aki From kevin at my.walr.us Fri Feb 10 12:59:52 2017 From: kevin at my.walr.us (KT Walrus) Date: Fri, 10 Feb 2017 07:59:52 -0500 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <20170210175858.0079da4d@batzmaru.gol.ad.jp> References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> <20170210175858.0079da4d@batzmaru.gol.ad.jp> Message-ID: > 1500 IMAP sessions will eat up about 3GB alone. Are you saying that Dovecot needs 2MB of physical memory per IMAP session? If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > > On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > >> hello >> >> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. >> > > Be very precise here, you expect to see 1500 as the result of > "doveadm who |grep pop3 |wc -l"? > > Because that implies an ungodly number of POP3 connects per second, given > the typically short duration of these. > > 1500 IMAP connections (note that frequently a client will have more than > the INBOX open and thus have more than one session and thus process on the > server) are a much easier proposition, provided they are of the typical > long lasting type. > > So can you put a number to your expected logins per second (both protocols)? > >> my server >> >> server configuration >> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 >> gb hdd for data (No raid) >> > No RAID and no other replication like DRBD? > Why would you even bother? > > How many users/mailboxes in total with what quota? > > 1500 IMAP sessions will eat up about 3GB alone. > You will want more memory, simply to keep all relevant SLAB bits (inodes, > dentries) in RAM. > > If you really have several hundreds logins/s, you're facing several > bottlenecks: > 1. Login processes themselves (easily fixed by high performance mode) > 2. Auth processes (that will depend on your backends, method mostly) > 3. Dovecot master process (spawning mail processes) > > The later is a single-threaded process, so it will benefit from a faster > CPU core. > It can be dramatically improved by enabling process re-usage, see: > http://wiki.dovecot.org/PerformanceTuning > > However that also means more memory usage. > > > > Christian > >> >> thanks >> rajesh >> > > [snip] > -- > Christian Balzer Network/Systems Engineer > chibi at gol.com Global OnLine Japan/Rakuten Communications > http://www.gol.com/ From alessio at skye.it Fri Feb 10 13:20:22 2017 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 10 Feb 2017 14:20:22 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: Il 10/02/2017 09:06, Aki Tuomi ha scritto: > Hi! > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with > IMAPSieve. Comments and suggestions are most welcome. > > https://wiki.dovecot.org/HowTo/AntispamWithSieve Hi, imap_stats plugin is required? -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice From dovecot-ml at seichter.de Fri Feb 10 14:52:52 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Fri, 10 Feb 2017 15:52:52 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> On 10.02.2017 09:06, Aki Tuomi wrote: > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with IMAPSieve. In my setup, I use the following sieve script globally for all users: if header :is "X-Spam-Flag" "YES" { fileinto "Junk"; stop; } This allows processing based on spam flags set by Amavis/SpamAssassin. I wonder if the method shown in https://wiki.dovecot.org/HowTo/AntispamWithSieve will cause incoming mail (via LMTP) that is already flagged as spam to be processed by report-spam.sieve and, in consequence, will be learned as spam for a second time, which would of course be undesirable? -Ralph From mailinglist at darac.org.uk Fri Feb 10 15:09:36 2017 From: mailinglist at darac.org.uk (Darac Marjal) Date: Fri, 10 Feb 2017 15:09:36 +0000 Subject: Replacement for antispam plugin In-Reply-To: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> Message-ID: <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> On Fri, Feb 10, 2017 at 03:52:52PM +0100, Ralph Seichter wrote: >On 10.02.2017 09:06, Aki Tuomi wrote: > >> Since antispam plugin is deprecated and we would really prefer people >> not to use it, we wrote instructions on how to replace it with IMAPSieve. > >In my setup, I use the following sieve script globally for all users: > > if header :is "X-Spam-Flag" "YES" { > fileinto "Junk"; > stop; > } > >This allows processing based on spam flags set by Amavis/SpamAssassin. >I wonder if the method shown in > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > >will cause incoming mail (via LMTP) that is already flagged as spam to >be processed by report-spam.sieve and, in consequence, will be learned >as spam for a second time, which would of course be undesirable? It shouldn't do. Check out https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally only used at delivery time, but the sieve_imapsieve plugin runs a *different* sieve script based on IMAP actions (for example, COPY). So, when you deliver, your main script tells dovecot WHERE to deliver to. When you issue and IMAP COPY command, the Antispam scripts tell dovecot to pipe the message to spamassassin. The delivery does not involve IMAP and the IMAP COPY does not constitute redelivery. -- For more information, please reread. From drbobllc at yahoo.com Fri Feb 10 15:37:31 2017 From: drbobllc at yahoo.com (drbobllc at yahoo.com) Date: Fri, 10 Feb 2017 15:37:31 +0000 (UTC) Subject: please help this newbie get started In-Reply-To: <320424774.1204094.1486311151591@mail.yahoo.com> References: <1894124593.523567.1486153137926.ref@mail.yahoo.com> <1894124593.523567.1486153137926@mail.yahoo.com> <1687514593.624092.1486163761261@mail.yahoo.com> <19662EB0-3B67-40C9-8A9F-F9D7159D4BC9@valo.at> <80096989.830779.1486219047023@mail.yahoo.com> <1348327411.1107599.1486274134291@mail.yahoo.com> <85AED38A-A83F-4021-B42A-324D251970C8@valo.at> <347428067.1212522.1486304091270@mail.yahoo.com> <320424774.1204094.1486311151591@mail.yahoo.com> Message-ID: <795174969.2023422.1486741051573@mail.yahoo.com> I appreciated the help I received here. To try to give back a little, I contributed something I learned to the wiki: Passwd as a password databasePasswd as a password database on FreeBSD Thanks again, Bob On Sunday, February 5, 2017 10:12 AM, "drbobllc at yahoo.com" wrote: Hi, everyone, Got through for the first time! In fact the trick was to switch to: passdb { ? driver = passwd-file ? args = path-to-file-with-encrypted-passwords } Thanks for steering me in the right direction. Next I guess is SSL for more security. Bob On Sunday, February 5, 2017 8:14 AM, "drbobllc at yahoo.com" wrote: Do I need to tell dovecot to check master.passwd instead of passwd? 2. Is my (simple) passdb OK? passdb { ? args = blocking=no ? driver = passwd } From dovelist at tesla.demon.nl Fri Feb 10 16:05:15 2017 From: dovelist at tesla.demon.nl (dovelist) Date: Fri, 10 Feb 2017 17:05:15 +0100 Subject: Managesieve cannot access script store In-Reply-To: References: Message-ID: <6fed7b56fee93b911001ece16bace8ee@tesla.demon.nl> Hi Stephan, > Normally, Dovecot permission errors are more helpful than that. So, > this > error message in itself is a bit of a bug: I'm glad to h've been able to help with this beta-test ;-) > About the cause of this error: keep in mind that the whole directory > path needs read/execute permission, not only the leaf directory. Have checked. They are... > You could try a command other than LISTSCRIPTS in your manual debugging > efforts. That should take a different code path that provides a more > detailed error. I tried: PUTSCRIPT "hutsefluts" {6+} keep; Gives the same result: Feb 10 15:43:26 p150 dovecot[2042]: managesieve(rogier): Error: sieve: file storage: save: open(/home/rogier/sieve/tmp/hutsefluts_1486737806.M728733P6414.p150.sieve) failed: Permission denied I have put a script named "std.sieve" in the sieve directory manually. Then the GETSCRIPT command gives some more information: Feb 10 15:50:07 p150 dovecot[2042]: managesieve(rogier): Debug: sieve: file script: Opened script `std' from `/home/rogier/sieve/std.sieve' Feb 10 15:50:07 p150 dovecot[2042]: managesieve(rogier): Error: sieve: file script: Failed to open sieve script: open(/home/rogier/sieve/std.sieve) failed: Permission denied (euid=1000(rogier) egid=100(users) UNIX perms appear ok (ACL/MAC wrong?)) So the UNIX permissions seem not to be the problem. The mentioning of ACL made me look into the audit.log. There I found this: type=AVC msg=audit(1486738207.203:354): apparmor="DENIED" operation="open" profile="/usr/lib/dovecot/managesieve" name="/home/rogier/sieve/std.sieve" pid=6414 comm="managesieve" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000 type=SYSCALL msg=audit(1486738207.203:354): arch=c000003e syscall=2 success=no exit=-13 a0=55e8920917d8 a1=0 a2=7fff73b41a14 a3=65766569732f7265 items=0 ppid=1861 pid=6414 auid=429 4967295 uid=1000 gid=100 euid=1000 suid=1000 fsuid=1000 egid=100 sgid=100 fsgid=100 tty=(none) ses=4294967295 comm="managesieve" exe="/usr/lib/dovecot/managesieve" key=(null) type=UNKNOWN[1327] msg=audit(1486738207.203:354): proctitle="dovecot/managesieve" Looks like AppArmor says NO... Does the apparmor profile for managesieve account for this or any other script store location? Or is the user expected to tweak apparmor profiles in such cases? Then I have to figure out how... Regards, Rogier From chibi at gol.com Fri Feb 10 16:07:14 2017 From: chibi at gol.com (Christian Balzer) Date: Sat, 11 Feb 2017 01:07:14 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> <20170210175858.0079da4d@batzmaru.gol.ad.jp> Message-ID: <20170211010714.5790a617@batzmaru.gol.ad.jp> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > > 1500 IMAP sessions will eat up about 3GB alone. > > Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > That depends on the IMAP session, read the mailbox size and index size, etc. Some are significantly larger: --- PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap --- But yes, as somebody who has mailbox servers with 55k+ session the average is around 1.6MB. > If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > You will want: 1. 256GB of real RAM, swap is for chums. 2. Understanding how to tune Dovecot and more importantly the overall system to such a task (see that PID up there?). 3. Be willing to deal with stuff like top and ps taking ages to start/run and others like atop actually killing dovecot (performance wise, not literally) when doing their obviously flawed cleanup on exit. Some things clearly do NOT scale well. My current goal is to have 100k capable servers that work well, 200k in a failover scenario, but that won't be particular enjoyable. Christian > > On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > > > > On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > > > >> hello > >> > >> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > >> > > > > Be very precise here, you expect to see 1500 as the result of > > "doveadm who |grep pop3 |wc -l"? > > > > Because that implies an ungodly number of POP3 connects per second, given > > the typically short duration of these. > > > > 1500 IMAP connections (note that frequently a client will have more than > > the INBOX open and thus have more than one session and thus process on the > > server) are a much easier proposition, provided they are of the typical > > long lasting type. > > > > So can you put a number to your expected logins per second (both protocols)? > > > >> my server > >> > >> server configuration > >> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > >> gb hdd for data (No raid) > >> > > No RAID and no other replication like DRBD? > > Why would you even bother? > > > > How many users/mailboxes in total with what quota? > > > > 1500 IMAP sessions will eat up about 3GB alone. > > You will want more memory, simply to keep all relevant SLAB bits (inodes, > > dentries) in RAM. > > > > If you really have several hundreds logins/s, you're facing several > > bottlenecks: > > 1. Login processes themselves (easily fixed by high performance mode) > > 2. Auth processes (that will depend on your backends, method mostly) > > 3. Dovecot master process (spawning mail processes) > > > > The later is a single-threaded process, so it will benefit from a faster > > CPU core. > > It can be dramatically improved by enabling process re-usage, see: > > http://wiki.dovecot.org/PerformanceTuning > > > > However that also means more memory usage. > > > > > > > > Christian > > > >> > >> thanks > >> rajesh > >> > > > > [snip] > > -- > > Christian Balzer Network/Systems Engineer > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > http://www.gol.com/ > -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From marek at grondecki.de Fri Feb 10 16:09:06 2017 From: marek at grondecki.de (Marek Grondecki) Date: Fri, 10 Feb 2017 17:09:06 +0100 Subject: Prohibit dots in folder names In-Reply-To: References: <6aba8b7257a3d49fdc40e17b4d418c7b@grondecki.de> <74e2c3b9-97b0-91b9-4dcf-d4c111db4619@dovecot.fi> Message-ID: <687a77370b4386986d22977eef18e7e6@grondecki.de> Using "doveadm mailbox create" I tried to create a mailbox with dots, it's not possible. doveadm(marek): Error: Can't create mailbox test.test1.test2.test3: Character not allowed in mailbox name: '.' So actually dovecot is already prohibiting a creation of folders with dots in it's names. Unfortunately Outlook 2013 is creating those folders in user's profile and showing them as (Only this computer). Any idea how to fix this behaviour? Marek On 2017-01-11 10:46, Aki Tuomi wrote: > On 11.01.2017 11:35, Steffen Kaiser wrote: >> On Wed, 11 Jan 2017, Aki Tuomi wrote: >> > On 10.01.2017 17:38, Marek Grondecki wrote: >> >> Hello Dovecot Community, >> >> >> >> I am using Dovecot 2.2.13-12~deb8u1 (Debian Jessie). >> >> >> >> separator = / >> >> listescape plugin is NOT active >> >> LAYOUT=fs is NOT active >> >> >> >> I would like to prohibit a creation of folders with dot "." in the >> >> names - >> >> so creation of a folder "foo.bar" should NOT be possible. >> >> Currently, when I create "foo.bar" in Outlook 2013 it will be created >> >> but won't be >> >> synchronised with the mail server - Outlook shows it as "foo.bar (Only >> >> this computer)". >> >> Ideally the user should receive an error message informing that "the >> >> folder foo.bar >> >> could not be created." >> >> >> >> Thank you for your help. >> >> >> >> Regards, >> >> Marek Grondecki >> >> > Are you sure the folder is actually created in dovecot instance? >> > Can you run doveadm mailbox status -u username foo.bar? >> >> Yes, they are. If you issue >> >> * create t.t.t.t.t.t.t >> >> the complete set of t's is created. (I'm using Maildir as backend.) >> >> This case is what the listescape plugin is to help. I don't use it >> myself, however. >> >> -- Steffen Kaiser > > I am sure that it is created like that, but his Outlook is saying (Only > this computer), which is the reason I am asking. I am sure the command > works over IMAP protocol. > > Aki From gkontos.mail at gmail.com Fri Feb 10 16:25:27 2017 From: gkontos.mail at gmail.com (George Kontostanos) Date: Fri, 10 Feb 2017 18:25:27 +0200 Subject: Replacement for antispam plugin In-Reply-To: <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> Message-ID: On Fri, Feb 10, 2017 at 5:09 PM, Darac Marjal wrote: > On Fri, Feb 10, 2017 at 03:52:52PM +0100, Ralph Seichter wrote: > >> On 10.02.2017 09:06, Aki Tuomi wrote: >> >> Since antispam plugin is deprecated and we would really prefer people >>> not to use it, we wrote instructions on how to replace it with IMAPSieve. >>> >> >> In my setup, I use the following sieve script globally for all users: >> >> if header :is "X-Spam-Flag" "YES" { >> fileinto "Junk"; >> stop; >> } >> >> This allows processing based on spam flags set by Amavis/SpamAssassin. >> I wonder if the method shown in >> >> https://wiki.dovecot.org/HowTo/AntispamWithSieve >> >> will cause incoming mail (via LMTP) that is already flagged as spam to >> be processed by report-spam.sieve and, in consequence, will be learned >> as spam for a second time, which would of course be undesirable? >> > > It shouldn't do. Check out https://wiki.dovecot.org/Pigeo > nhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally only > used at delivery time, but the sieve_imapsieve plugin runs a *different* > sieve script based on IMAP actions (for example, COPY). > > So, when you deliver, your main script tells dovecot WHERE to deliver to. > When you issue and IMAP COPY command, the Antispam scripts tell dovecot to > pipe the message to spamassassin. The delivery does not involve IMAP and > the IMAP COPY does not constitute redelivery. > > > -- > For more information, please reread. > I think that this needs some change: # From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve When a message from Spam is moved to Trash then the report-ham.sieve is being executed. Can we add an exception for the Trash folder? Thanks -- George Kontostanos --- From 24x7server at 24x7server.net Fri Feb 10 16:29:49 2017 From: 24x7server at 24x7server.net (Rajesh M) Date: Fri, 10 Feb 2017 21:59:49 +0530 Subject: dovecot config for 1500 simultaneous connection Message-ID: <250E69D9124241839D4DFFA56C38E81B.MAI@ns1.24x7server.net> ----- Original Message ----- From: Christian Balzer [mailto:chibi at gol.com] To: dovecot at dovecot.org Cc: 24x7server at 24x7server.net Sent: Fri, 10 Feb 2017 17:58:58 +0900 Subject: On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > hello > > could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > Be very precise here, you expect to see 1500 as the result of "doveadm who |grep pop3 |wc -l"? Because that implies an ungodly number of POP3 connects per second, given the typically short duration of these. 1500 IMAP connections (note that frequently a client will have more than the INBOX open and thus have more than one session and thus process on the server) are a much easier proposition, provided they are of the typical long lasting type. So can you put a number to your expected logins per second (both protocols)? > my server > > server configuration > hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > gb hdd for data (No raid) > No RAID and no other replication like DRBD? Why would you even bother? How many users/mailboxes in total with what quota? 1500 IMAP sessions will eat up about 3GB alone. You will want more memory, simply to keep all relevant SLAB bits (inodes, dentries) in RAM. If you really have several hundreds logins/s, you're facing several bottlenecks: 1. Login processes themselves (easily fixed by high performance mode) 2. Auth processes (that will depend on your backends, method mostly) 3. Dovecot master process (spawning mail processes) The later is a single-threaded process, so it will benefit from a faster CPU core. It can be dramatically improved by enabling process re-usage, see: http://wiki.dovecot.org/PerformanceTuning However that also means more memory usage. Christian > > thanks > rajesh > [snip] -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ thanks christian during peak times here are the results for connections [root at ns1 domains]# doveadm who |grep imap |wc -l username # proto (pids) (ips) 631 [root at ns1 domains]# doveadm who |grep pop3 |wc -l username # proto (pids) (ips) 233 could you please guide me concerning the dovecot config files settings to handle the above 631 imap and 233 pop connections. number of mailboxes is around 4000 -- some users would consume 25 GB while others would be just around 10 MB this is a hex core machine with hyperthreading -- so 12 cores [root at ns1 domains]# iostat Linux 2.6.32-431.29.2.el6.x86_64 (ns1.bizmailserver.net) 02/10/2017 _x86_64_ (12 CPU) avg-cpu: %user %nice %system %iowait %steal %idle 2.67 0.00 0.65 3.43 0.00 93.25 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn sdd 44.95 1094.25 765.10 720884842 504041712 sdc 1.92 32.15 0.03 21178186 21248 sdb 34.71 1377.37 625.54 907398402 412102224 sda 49.88 124.29 2587.32 81879548 1704506408 thanks rajesh From gkontos.mail at gmail.com Fri Feb 10 16:45:26 2017 From: gkontos.mail at gmail.com (George Kontostanos) Date: Fri, 10 Feb 2017 18:45:26 +0200 Subject: Replacement for antispam plugin In-Reply-To: References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> Message-ID: On Fri, Feb 10, 2017 at 6:25 PM, George Kontostanos wrote: > > > On Fri, Feb 10, 2017 at 5:09 PM, Darac Marjal > wrote: > >> On Fri, Feb 10, 2017 at 03:52:52PM +0100, Ralph Seichter wrote: >> >>> On 10.02.2017 09:06, Aki Tuomi wrote: >>> >>> Since antispam plugin is deprecated and we would really prefer people >>>> not to use it, we wrote instructions on how to replace it with >>>> IMAPSieve. >>>> >>> >>> In my setup, I use the following sieve script globally for all users: >>> >>> if header :is "X-Spam-Flag" "YES" { >>> fileinto "Junk"; >>> stop; >>> } >>> >>> This allows processing based on spam flags set by Amavis/SpamAssassin. >>> I wonder if the method shown in >>> >>> https://wiki.dovecot.org/HowTo/AntispamWithSieve >>> >>> will cause incoming mail (via LMTP) that is already flagged as spam to >>> be processed by report-spam.sieve and, in consequence, will be learned >>> as spam for a second time, which would of course be undesirable? >>> >> >> It shouldn't do. Check out https://wiki.dovecot.org/Pigeo >> nhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally >> only used at delivery time, but the sieve_imapsieve plugin runs a >> *different* sieve script based on IMAP actions (for example, COPY). >> >> So, when you deliver, your main script tells dovecot WHERE to deliver to. >> When you issue and IMAP COPY command, the Antispam scripts tell dovecot to >> pipe the message to spamassassin. The delivery does not involve IMAP and >> the IMAP COPY does not constitute redelivery. >> >> >> -- >> For more information, please reread. >> > > I think that this needs some change: > > # From Spam folder to elsewhere > imapsieve_mailbox2_name = * > imapsieve_mailbox2_from = Spam > imapsieve_mailbox2_causes = COPY > imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve > > When a message from Spam is moved to Trash then the report-ham.sieve is > being executed. > > Can we add an exception for the Trash folder? > > Thanks > > > -- > George Kontostanos > --- > Could we use something like this: imapsieve_mailbox2_name = ! Trash -- George Kontostanos --- From dovecot-ml at seichter.de Fri Feb 10 16:50:46 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Fri, 10 Feb 2017 17:50:46 +0100 Subject: Replacement for antispam plugin In-Reply-To: <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> Message-ID: <6f85c290-14b1-83c3-a9df-61576ea65520@seichter.de> On 10.02.2017 16:09, Darac Marjal wrote: > Check out https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve, > which explains that sieve is normally only used at delivery time, but > the sieve_imapsieve plugin runs a *different* sieve script based on > IMAP actions (for example, COPY). Thanks for the pointer, Darac. The following confirms that imapsieve_* is not involved during delivery: "Note that the imapsieve extension can only be used in a Sieve script that is invoked from IMAP. When it is used in the active delivery script, it will cause runtime errors." -Ralph From zhb at iredmail.org Fri Feb 10 17:22:20 2017 From: zhb at iredmail.org (Zhang Huangbin) Date: Sat, 11 Feb 2017 01:22:20 +0800 Subject: Replacement for antispam plugin In-Reply-To: <6f85c290-14b1-83c3-a9df-61576ea65520@seichter.de> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <6f85c290-14b1-83c3-a9df-61576ea65520@seichter.de> Message-ID: > On Feb 11, 2017, at 12:50 AM, Ralph Seichter wrote: > > Check out https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve, My concern is, will you experience any lag while moving message? According to the doc, this plugin works like ?pipe? backend of old ?antispam? plugin, when you move a message from INBOX to Junk, the antispam plugin calls sa-learn, and you will experience noticeable lag on webmail (i used Roundcube webmail for testing) until sa-learn finished. It's much worse if you move multiple emails at the same time, because we have to wait for sa-learn to learn all moved messages. I switched to ?spool2dir? backend of old ?antispam? plugin, and call sa-learn hourly to learn spam/ham. Since it simply copies moved message(s), no lag on webmail side at all, our users are satisfied. ---- Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ Time zone: GMT+8 (China/Beijing). From michael.slusarz at dovecot.fi Fri Feb 10 17:34:19 2017 From: michael.slusarz at dovecot.fi (Michael Slusarz) Date: Fri, 10 Feb 2017 10:34:19 -0700 (MST) Subject: Replacement for antispam plugin In-Reply-To: References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> Message-ID: <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> > On February 10, 2017 at 9:25 AM George Kontostanos wrote: [snip] > I think that this needs some change: > > # From Spam folder to elsewhere > imapsieve_mailbox2_name = * > imapsieve_mailbox2_from = Spam > imapsieve_mailbox2_causes = COPY > imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve > > When a message from Spam is moved to Trash then the report-ham.sieve is > being executed. > > Can we add an exception for the Trash folder? This is handled in the sieve script. E.g.: require "environment"; if environment "imap.mailbox" "Trash" { stop; } michael From adam at shostack.org Fri Feb 10 18:30:13 2017 From: adam at shostack.org (Adam Shostack) Date: Fri, 10 Feb 2017 13:30:13 -0500 Subject: Safely piping to a shell script Message-ID: <20170210183013.GA15969@calypso.stonekeep.com> Thanks for the previous answer on :execute. In thinking about malicious input, I am worried about the possibility that mail will be sent with a clever from line. (Section 7 of http://www.ietf.org/rfc/rfc5229.txt is great, btw) To address this, I'm considering the following, and would appreciate feedback. I'm aware that this doesn't capture all emails, those with non-alphanum are legit, and badly handled. I think that this restricts the input of the grepfrom script to be a single string, matching "a-zA-Z09 at ." if not address :regex "from" "^[:alnum]*@[:alnum]*([:alnum].)*$" { fileinto :create "wierd"; stop ; } elsif address :regex "from" "*" { set "sender" ${1}; if execute "grepfrom" "${sender}" { keep ; stop; } else {fileinto :create "neversent"} As an aside, https://www.joachim-breitner.de/blog/441-Goodbye_procmail,_Hello_Sieve contains a nice pattern, creating an include file to test, and that addresses many, but not all of my use cases. Should I worry if the match there is 5000+ strings? Adam From dovecot-ml at seichter.de Fri Feb 10 18:48:08 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Fri, 10 Feb 2017 19:48:08 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <6f85c290-14b1-83c3-a9df-61576ea65520@seichter.de> Message-ID: <6fb0541b-a965-4724-49e1-6012e732581f@seichter.de> On 10.02.17 18:22, Zhang Huangbin wrote: > My concern is, will you experience any lag while moving message? I don't use direct calls to sa-learn, but store the piped e-mails on disk, and a periodic cron-job picks them up and invokes sa-learn. This way, there is no noticeable lag. -Ralph From dovecot-ml at seichter.de Fri Feb 10 19:13:00 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Fri, 10 Feb 2017 20:13:00 +0100 Subject: Replacement for antispam plugin In-Reply-To: <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> Message-ID: <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> On 10.02.17 18:34, Michael Slusarz wrote: > > Can we add an exception for the Trash folder? > > This is handled in the sieve script. E.g.: > > require "environment"; > if environment "imap.mailbox" "Trash" { > stop; > } This does not work for me, and I don't really expect it to work either. https://tools.ietf.org/html/rfc6785#section-4.4 states: The implementation MUST set the Environment [RFC5183] item "imap.mailbox" to the name of the mailbox that the affected message is in, in the case of existing messages, or is targeted to be stored into, in the case of new messages. The message already exists in the Spam folder, hence imap.mailbox should be "Spam" instead of "Trash", correct? Is there perhaps another way to ensure that manually deleted spam is not erroneously learned as ham? -Ralph From michael.slusarz at dovecot.fi Fri Feb 10 19:34:50 2017 From: michael.slusarz at dovecot.fi (Michael Slusarz) Date: Fri, 10 Feb 2017 12:34:50 -0700 (MST) Subject: Replacement for antispam plugin In-Reply-To: <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> Message-ID: <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> > On February 10, 2017 at 12:13 PM Ralph Seichter wrote: > > On 10.02.17 18:34, Michael Slusarz wrote: > > > Can we add an exception for the Trash folder? > > This is handled in the sieve script. E.g.: > > > > require "environment"; > > if environment "imap.mailbox" "Trash" { > > stop; > > } > > This does not work for me, and I don't really expect it to work either. > https://tools.ietf.org/html/rfc6785#section-4.4 states: > > The implementation MUST set the Environment [RFC5183] item "imap.mailbox" > to the name of the mailbox that the affected message is in, in the > case of existing messages, or is targeted to be stored into, in the > case of new messages. > > The message already exists in the Spam folder, hence imap.mailbox should > be "Spam" instead of "Trash", correct? Incorrect. When you move a message to a new mailbox, that is a "new message" event (a new UID in the target mailbox is created; the message count increases).? So imap.mailbox is set to the name of the *target* mailbox. > Is there perhaps another way to ensure that manually deleted spam is not > erroneously learned as ham? > > -Ralph From dave.mehler at gmail.com Fri Feb 10 19:46:50 2017 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 10 Feb 2017 14:46:50 -0500 Subject: Replacement for antispam plugin In-Reply-To: <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> Message-ID: Hello, Chiming in on this with a question, and will be getting to it over the weekend or later this evening time permitting. Does retraining a message as either spam or ham alter message headers for example x-spam or the spamassassin-modified subject header? If not is it possible to do so after processing? For example, I have a message inadvertently tagged as spam, from Spamassassin it gets an x-spam header added as well as a modified subject. Retraining that message as ham moving it to say any other folder but spam i'd like for that x-spam header to be set to as it is not spam, and the spamassassin subject to be removed. Is this doable? Thanks. Dave. On 2/10/17, Michael Slusarz wrote: > >> On February 10, 2017 at 12:13 PM Ralph Seichter wrote: >> >> On 10.02.17 18:34, Michael Slusarz wrote: >> > > Can we add an exception for the Trash folder? >> > This is handled in the sieve script. E.g.: >> > >> > require "environment"; >> > if environment "imap.mailbox" "Trash" { >> > stop; >> > } >> >> This does not work for me, and I don't really expect it to work either. >> https://tools.ietf.org/html/rfc6785#section-4.4 states: >> >> The implementation MUST set the Environment [RFC5183] item >> "imap.mailbox" >> to the name of the mailbox that the affected message is in, in the >> case of existing messages, or is targeted to be stored into, in the >> case of new messages. >> >> The message already exists in the Spam folder, hence imap.mailbox should >> be "Spam" instead of "Trash", correct? > > Incorrect. > > When you move a message to a new mailbox, that is a "new message" event (a > new UID in the target mailbox is created; the message count increases).? So > imap.mailbox is set to the name of the *target* mailbox. > >> Is there perhaps another way to ensure that manually deleted spam is not >> erroneously learned as ham? >> >> -Ralph > From kevin at my.walr.us Fri Feb 10 19:50:03 2017 From: kevin at my.walr.us (KT Walrus) Date: Fri, 10 Feb 2017 14:50:03 -0500 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <20170211010714.5790a617@batzmaru.gol.ad.jp> References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> <20170210175858.0079da4d@batzmaru.gol.ad.jp> <20170211010714.5790a617@batzmaru.gol.ad.jp> Message-ID: <307B1E3A-F63C-4A01-A9D7-13AD27E083CB@my.walr.us> > 1. 256GB of real RAM, swap is for chums. Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > > On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > >>> 1500 IMAP sessions will eat up about 3GB alone. >> >> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? >> > That depends on the IMAP session, read the mailbox size and index size, > etc. > Some are significantly larger: > --- > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > --- > > But yes, as somebody who has mailbox servers with 55k+ session the average > is around 1.6MB. > >> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? >> > You will want: > 2. Understanding how to tune Dovecot and more importantly the overall > system to such a task (see that PID up there?). > 3. Be willing to deal with stuff like top and ps taking ages to start/run > and others like atop actually killing dovecot (performance wise, not > literally) when doing their obviously flawed cleanup on exit. Some things > clearly do NOT scale well. > > My current goal is to have 100k capable servers that work well, 200k in a > failover scenario, but that won't be particular enjoyable. > > Christian > >>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: >>> >>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: >>> >>>> hello >>>> >>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. >>>> >>> >>> Be very precise here, you expect to see 1500 as the result of >>> "doveadm who |grep pop3 |wc -l"? >>> >>> Because that implies an ungodly number of POP3 connects per second, given >>> the typically short duration of these. >>> >>> 1500 IMAP connections (note that frequently a client will have more than >>> the INBOX open and thus have more than one session and thus process on the >>> server) are a much easier proposition, provided they are of the typical >>> long lasting type. >>> >>> So can you put a number to your expected logins per second (both protocols)? >>> >>>> my server >>>> >>>> server configuration >>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 >>>> gb hdd for data (No raid) >>>> >>> No RAID and no other replication like DRBD? >>> Why would you even bother? >>> >>> How many users/mailboxes in total with what quota? >>> >>> 1500 IMAP sessions will eat up about 3GB alone. >>> You will want more memory, simply to keep all relevant SLAB bits (inodes, >>> dentries) in RAM. >>> >>> If you really have several hundreds logins/s, you're facing several >>> bottlenecks: >>> 1. Login processes themselves (easily fixed by high performance mode) >>> 2. Auth processes (that will depend on your backends, method mostly) >>> 3. Dovecot master process (spawning mail processes) >>> >>> The later is a single-threaded process, so it will benefit from a faster >>> CPU core. >>> It can be dramatically improved by enabling process re-usage, see: >>> http://wiki.dovecot.org/PerformanceTuning >>> >>> However that also means more memory usage. >>> >>> >>> >>> Christian >>> >>>> >>>> thanks >>>> rajesh >>>> >>> >>> [snip] >>> -- >>> Christian Balzer Network/Systems Engineer >>> chibi at gol.com Global OnLine Japan/Rakuten Communications >>> http://www.gol.com/ >> > > > -- > Christian Balzer Network/Systems Engineer > chibi at gol.com Global OnLine Japan/Rakuten Communications > http://www.gol.com/ From dovecot-ml at seichter.de Fri Feb 10 19:59:00 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Fri, 10 Feb 2017 20:59:00 +0100 Subject: Replacement for antispam plugin In-Reply-To: <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> Message-ID: <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> On 10.02.17 20:34, Michael Slusarz wrote: > When you move a message to a new mailbox, that is a "new message" > event (a new UID in the target mailbox is created; the message count > increases). So imap.mailbox is set to the name of the *target* mailbox. My tests seem to indicate otherwise. Deleting a message currently located in the Junk folder causes the report-ham.sieve script to be invoked, which in my case contains the following: require ["vnd.dovecot.pipe", "copy", "environment"]; if environment "imap.mailbox" "Trash" { stop; } elsif environment "imap.mailbox" "Junk" { pipe :copy "debug-junk"; } else { pipe :copy "learn-ham"; } I can see that "learn-ham" is always invoked when a message is deleted from or moved out of the Junk folder, so my guess is that imap.mailbox is neither "Trash" nor "Junk" ("debug-junk" is never called, according to the Dovecot logs). Unfortunately I don't know how to debug this further. I don't want "learn-ham" to be run when a message located in the Junk folder is manually deleted or moved to Trash, and right now I don't know how to accomplish this. -Ralph From gkontos.mail at gmail.com Fri Feb 10 21:15:13 2017 From: gkontos.mail at gmail.com (George Kontostanos) Date: Fri, 10 Feb 2017 23:15:13 +0200 Subject: Replacement for antispam plugin In-Reply-To: <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> Message-ID: On Fri, Feb 10, 2017 at 9:59 PM, Ralph Seichter wrote: > On 10.02.17 20:34, Michael Slusarz wrote: > > > When you move a message to a new mailbox, that is a "new message" > > event (a new UID in the target mailbox is created; the message count > > increases). So imap.mailbox is set to the name of the *target* mailbox. > > My tests seem to indicate otherwise. Deleting a message currently > located in the Junk folder causes the report-ham.sieve script to be > invoked, which in my case contains the following: > > require ["vnd.dovecot.pipe", "copy", "environment"]; > if environment "imap.mailbox" "Trash" { > stop; > } elsif environment "imap.mailbox" "Junk" { > pipe :copy "debug-junk"; > } else { > pipe :copy "learn-ham"; > } > > I can see that "learn-ham" is always invoked when a message is deleted > from or moved out of the Junk folder, so my guess is that imap.mailbox > is neither "Trash" nor "Junk" ("debug-junk" is never called, according > to the Dovecot logs). Unfortunately I don't know how to debug this > further. > > I don't want "learn-ham" to be run when a message located in the Junk > folder is manually deleted or moved to Trash, and right now I don't know > how to accomplish this. > > -Ralph > Same problem here. As a workaround I tried the following: # From Spam folder to Inbox imapsieve_mailbox2_name = Inbox imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve In theory this should trigger the report-ham script, only if a mail is moved from Spam to Inbox. However, it does not seem to work either. -- George Kontostanos --- From jtam.home at gmail.com Fri Feb 10 22:59:51 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 10 Feb 2017 14:59:51 -0800 (PST) Subject: dovecot config for 1500 simultaneous connection In-Reply-To: References: Message-ID: "Rajesh M" <24x7server at 24x7server.net> writes: > during peak times here are the results for connections > > [root at ns1 domains]# doveadm who |grep imap |wc -l > username # proto (pids) (ips) > 631 > [root at ns1 domains]# doveadm who |grep pop3 |wc -l > username # proto (pids) (ips) > 233 Thare are user counts, not connections. Ae a user can launch multiple IMAP connections, and if they have some MaxOSX reader, that can peak to several dozens during searches. This counts connections doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' Or you can parse the output of netstat. I'm suprised you have so many POP3 connections though -- they tend to be connect/process/disconnect. n=0 most of the time on my modest server. Joseph Tam From dmiller at amfes.com Fri Feb 10 23:15:30 2017 From: dmiller at amfes.com (Daniel Miller) Date: Fri, 10 Feb 2017 15:15:30 -0800 Subject: Solr 6.4.1 In-Reply-To: <36bf499b-9e55-d437-1faa-3ce2ff1531a9@dovecot.fi> References: <91215f50-fff4-b8ad-fa9e-031d18b52d53@amfes.com> <192b8a17-598a-9ea6-8557-3ff0572bbe6f@dovecot.fi> <36bf499b-9e55-d437-1faa-3ce2ff1531a9@dovecot.fi> Message-ID: 2.2.18 On 2/9/2017 11:13 PM, Aki Tuomi wrote: > What dovecot version are you using? > > > On 10.02.2017 09:12, Aki Tuomi wrote: >> Yeah, thanks. >> >> It seems that there indeed is content-type header (there should not be). >> We'll look into it. >> >> Aki >> >> On 10.02.2017 01:44, Daniel Miller wrote: >>> Does this work (pcap attached)? >>> >>> Daniel >>> >>> On 2/8/2017 10:57 PM, Aki Tuomi wrote: >>>> On 09.02.2017 07:54, Daniel Miller wrote: >>>>> I've been running Solr for a while (4.10.3) - wanted to make the jump >>>>> to the latest & greatest. I installed 6.4.1, copied over my >>>>> schema.xml - and after a couple false starts where I needed to tweak >>>>> it work with the new version...it works! I did not copy the database, >>>>> started from scratch, and executed a "doveadm fts rescan -A". But... >>>>> >>>>> Judging solely from at least one client - it's fine. But looking in >>>>> the logs I see: >>>>> 1. The first scan of a mailbox dovecot's error log gives: >>>>> dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup failed: >>>>> Bad Request >>>>> >>>>> 2. Subsequent scans do not appear to generate any dovecot error logs >>>>> - but I'm not certain. Each new mailbox/subfolder scanned will each >>>>> have one error on the initial scan. >>>>> >>>>> 3. Solr's log gives me the following - on every search. >>>>> 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] >>>>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad >>>>> contentType for search handler :text/xml >>>>> request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} >>>>> >>>> Hi! >>>> >>>> can you please use tcpdump or wireshark to capture the actual HTTP >>>> request causing this exception? >>>> >>>> Aki From chibi at gol.com Sat Feb 11 05:06:56 2017 From: chibi at gol.com (Christian Balzer) Date: Sat, 11 Feb 2017 14:06:56 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <250E69D9124241839D4DFFA56C38E81B.MAI@ns1.24x7server.net> References: <250E69D9124241839D4DFFA56C38E81B.MAI@ns1.24x7server.net> Message-ID: <20170211140656.1bf251eb@batzmaru.gol.ad.jp> Hello, On Fri, 10 Feb 2017 21:59:49 +0530 Rajesh M wrote: You replied below my signature, making a normal reply/quotation impossible for decent mail clients, which is worse than top-quoting. Please reply in-line or at the top if must be. > thanks christian > > during peak times here are the results for connections > > [root at ns1 domains]# doveadm who |grep imap |wc -l > username # proto (pids) (ips) > 631 > [root at ns1 domains]# doveadm who |grep pop3 |wc -l > username # proto (pids) (ips) > 233 > As Joseph mentioned, these are users, not sessions. And while this gives us some ideas, it doesn't answer my question about login rates. Do something like this: "grep Login: /var/log/mail.log.1 |wc -l" with the mail.log being of a typical, busy day. On my larger servers that average to 35 logins per second, with obviously higher peaks. Without the mail process re-usage (idling), that would have the dovecot master process use about 35% of a decent cpu core, with 100% being a hard limit. > > could you please guide me concerning the dovecot config files settings to handle the above 631 imap and 233 pop connections. > What do you mean, isn't your current system handling that? See the various tuning, performance hints on the wiki, but w/o more info where your system is stalling, dovecot config changes might not be enough. > number of mailboxes is around 4000 -- some users would consume 25 GB while others would be just around 10 MB > So that at least puts an upper limit to the users, however w/o quotas your users could easily swamp your storage. And those 25GB mailbox users will have a rather large IMAP mail process memory footprint. > this is a hex core machine with hyperthreading -- so 12 cores > With the exception of that dovecot master forking issue, I've never run out of CPU resources with it. > [root at ns1 domains]# iostat > Linux 2.6.32-431.29.2.el6.x86_64 (ns1.bizmailserver.net) 02/10/2017 _x86_64_ (12 CPU) > > avg-cpu: %user %nice %system %iowait %steal %idle > 2.67 0.00 0.65 3.43 0.00 93.25 > > Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn > sdd 44.95 1094.25 765.10 720884842 504041712 > sdc 1.92 32.15 0.03 21178186 21248 > sdb 34.71 1377.37 625.54 907398402 412102224 > sda 49.88 124.29 2587.32 81879548 1704506408 > > Rather meaningless w/o knowning which drive is which. Also an "iostat -x" oneshot summary and a few samples of when the machine is busy would be vastly more informative. atop is a good tool (when not running with 20k+ processes) to give you an idea about bottlenecks and what resource is being utilized how much. Christian > > thanks > rajesh > ----- Original Message ----- > From: Christian Balzer [mailto:chibi at gol.com] > To: dovecot at dovecot.org > Cc: 24x7server at 24x7server.net > Sent: Fri, 10 Feb 2017 17:58:58 +0900 > Subject: > > On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > > > hello > > > > could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > > > > Be very precise here, you expect to see 1500 as the result of > "doveadm who |grep pop3 |wc -l"? > > Because that implies an ungodly number of POP3 connects per second, given > the typically short duration of these. > > 1500 IMAP connections (note that frequently a client will have more than > the INBOX open and thus have more than one session and thus process on the > server) are a much easier proposition, provided they are of the typical > long lasting type. > > So can you put a number to your expected logins per second (both protocols)? > > > my server > > > > server configuration > > hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > > gb hdd for data (No raid) > > > No RAID and no other replication like DRBD? > Why would you even bother? > > How many users/mailboxes in total with what quota? > > 1500 IMAP sessions will eat up about 3GB alone. > You will want more memory, simply to keep all relevant SLAB bits (inodes, > dentries) in RAM. > > If you really have several hundreds logins/s, you're facing several > bottlenecks: > 1. Login processes themselves (easily fixed by high performance mode) > 2. Auth processes (that will depend on your backends, method mostly) > 3. Dovecot master process (spawning mail processes) > > The later is a single-threaded process, so it will benefit from a faster > CPU core. > It can be dramatically improved by enabling process re-usage, see: > http://wiki.dovecot.org/PerformanceTuning > > However that also means more memory usage. > > > > Christian > > > > > thanks > > rajesh > > > > [snip] -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From chibi at gol.com Sat Feb 11 05:17:47 2017 From: chibi at gol.com (Christian Balzer) Date: Sat, 11 Feb 2017 14:17:47 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: References: Message-ID: <20170211141747.78d795e9@batzmaru.gol.ad.jp> On Fri, 10 Feb 2017 14:59:51 -0800 (PST) Joseph Tam wrote: > "Rajesh M" <24x7server at 24x7server.net> writes: > > > during peak times here are the results for connections > > > > [root at ns1 domains]# doveadm who |grep imap |wc -l > > username # proto (pids) (ips) > > 631 > > [root at ns1 domains]# doveadm who |grep pop3 |wc -l > > username # proto (pids) (ips) > > 233 > > Thare are user counts, not connections. Ae a user can launch multiple > IMAP connections, and if they have some MaxOSX reader, that can peak to > several dozens during searches. > > This counts connections > > doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' > > Or you can parse the output of netstat. > > I'm suprised you have so many POP3 connections though -- they tend to be > connect/process/disconnect. n=0 most of the time on my modest server. > That vastly depends on things like client network speed and mailbox size. People pop'ing multi GB mailboxes (leave mail on server) with crappy clients (a big factor, some get disconnected for inactivity and thus hang around for 10 minutes) and over slow links tend to linger for quite a while. Typical example: --- Feb 11 13:54:27 mbx09 dovecot: pop3-login: Login: user=, method=PLAIN, rip=redacted, lip=redacted, mpid=381958, secured, session= Feb 11 14:05:18 mbx09 dovecot: pop3(redacted): Disconnected: Logged out top=0/0, retr=0/0, del=0/6188, size=103540794 session= --- This is on a server that's bored stiff when it comes to CPU usage or I/O utilization (pure SSD). All that twiddling of thumbs up there is purely client based. Christian -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From danjde at msw.it Sat Feb 11 12:17:25 2017 From: danjde at msw.it (Davide Marchi) Date: Sat, 11 Feb 2017 13:17:25 +0100 Subject: Unable to use encrypted password for imap and pop3 Message-ID: <372581e1ed1bf2c79bc9b7b7537a4ebb@msw.it> Hi friends, Following the Workaround tutorial for Jessie (https://workaround.org/ispmail/jessie ) I've set up six month ago my first Postfix email server and all works very fine, except for the ability from the user to encrypt login password. I can use the encrypted login password with smtp (Postfix), but not with pop3 and imap (Dovecot). The certificates are from Letsencrypt. I've read "http://wiki2.dovecot.org/Authentication/Mechanisms#Non-plaintext_authentication" and set auth_mechanisms = plain login cram-md5 But with no luck, If I try to connect to Dovecot from openssl obtain an error: "Verify return code: 20 (unable to get local issuer certificate)" And now, if I enable on the client (Tunderbird) the encrypted password obtain a message to reinsert the password Here my Dovecot configuration: doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.5 ext4 auth_mechanisms = plain login cram-md5 apop mail_location = maildir:/var/vmail/%d/%n/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX.Junk { auto = subscribe special_use = \Junk } mailbox INBOX.Trash { auto = subscribe special_use = \Trash } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_after = /etc/dovecot/sieve-after sieve_dir = ~/sieve } protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = References: <6fed7b56fee93b911001ece16bace8ee@tesla.demon.nl> Message-ID: <4659b4f84f968e42b8e771b695aacf83@tesla.demon.nl> OK, I've figured it out: In the dovecot profile for apparmor the sieve directory is not confgured. I solved it this way: To configure only one directory in the apparmor profile, I placed the active-script link inside the .sieve directory. Keeping the scripts separate in a store subdirectory, like this: In /etc/dovecot/conf.d/90-sieve.conf : sieve = file:~/.sieve/store;active=~/.sieve/active.sieve Then dovecot is granted access by adding the .sieve directory in the apparmor profile. The dovecot file in the tunables directory seems to be a neat way to that: In /etc/apparmor.d/tunables/dovecot : @{DOVECOT_MAILSTORE}=@{HOME}/Maildir/ /var/spool/mail/ @{HOME}/.sieve/ Ofcourse the .sieve directory is not really a MAILSTORE. But this way, the configuration stays close to the defaults. I didn't find something like DOVECOT_SIEVESTORE, which would be more appropriate. After restart of apparmor and dovecot, it works! @Stephan: thanks for the advice - it did help to pinpoint the problem! Regards, Rogier From mauric at gmx.ch Sat Feb 11 14:45:19 2017 From: mauric at gmx.ch (Maurizio Caloro) Date: Sat, 11 Feb 2017 15:45:19 +0100 Subject: no Authnticationmethode from outlook to Imap Message-ID: <007201d28475$77a46bf0$66ed43d0$@gmx.ch> Hello Debian, with postfix and Dovecot Imap "2.2.13" running. If I want connect from Outlook or Thunderbird I will become this error: Plaintext disalowd on non secure SSL/TLS Connection. but I will that this run with secure connection, I don't have any intention to use Plaintext auth. Also I don't know why, dovecot will switch from IMAP --> POP3-Login. Or it's this Outlook that will ask for it. 10-ssl.conf SSL = yes 10-auth.conf disable_plaintext_auth = yes Mail.log pop3-login: Info: Disconnected (tried to use disallowed plaintext auth): user=<>, rip=IP, lip=ip, session= Regards From ml+dovecot at valo.at Sat Feb 11 15:24:27 2017 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sat, 11 Feb 2017 16:24:27 +0100 Subject: no Authnticationmethode from outlook to Imap In-Reply-To: <007201d28475$77a46bf0$66ed43d0$@gmx.ch> References: <007201d28475$77a46bf0$66ed43d0$@gmx.ch> Message-ID: On 2017-02-11 15:45, Maurizio Caloro wrote: > Hello > > Debian, with postfix and Dovecot Imap "2.2.13" running. > > > > If I want connect from Outlook or Thunderbird I will become this error: > Plaintext disalowd on non secure SSL/TLS Connection. > > but I will that this run with secure connection, I don't have any > intention > to use Plaintext auth. How is your Outlook account set up? Which port is configured in your Outlook account? Port 143 using starttls or port 993 ssl? > Also I don't know why, dovecot will switch from IMAP --> POP3-Login. Or > it's > this Outlook that will ask for it. Dovecot does not switch from imap to pop3. Your Outlook seems to be configured for pop3 access. > > 10-ssl.conf > > SSL = yes > > 10-auth.conf > > disable_plaintext_auth = yes > > Mail.log > > pop3-login: Info: Disconnected (tried to use disallowed plaintext > auth): > user=<>, rip=IP, lip=ip, session= Provide the output of doveconf -n for better help. > > > Regards -- Christian Kivalo From mauric at gmx.ch Sat Feb 11 16:08:55 2017 From: mauric at gmx.ch (Maurizio Caloro) Date: Sat, 11 Feb 2017 17:08:55 +0100 Subject: AW: no Authnticationmethode from outlook to Imap In-Reply-To: References: <007201d28475$77a46bf0$66ed43d0$@gmx.ch> Message-ID: <008101d28481$256a5960$703f0c20$@gmx.ch> Thanks for your feedback...... I use 143 and if using 993 tell the system the the Certificate are wrong ??? root at caloro:/etc/dovecot/conf.d# doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.7 auth_mechanisms = plain login debug_log_path = /var/log/mail.log info_log_path = /var/log/mail.log log_path = /var/log/mail.log mail_access_groups = vmail mail_location = mbox:~/mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = ~/sieve/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at caloro.ch protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-client { mode = 0660 user = mail } } ssl_cert = Hello > > Debian, with postfix and Dovecot Imap "2.2.13" running. > > > > If I want connect from Outlook or Thunderbird I will become this error: > Plaintext disalowd on non secure SSL/TLS Connection. > > but I will that this run with secure connection, I don't have any > intention to use Plaintext auth. How is your Outlook account set up? Which port is configured in your Outlook account? Port 143 using starttls or port 993 ssl? > Also I don't know why, dovecot will switch from IMAP --> POP3-Login. > Or it's this Outlook that will ask for it. Dovecot does not switch from imap to pop3. Your Outlook seems to be configured for pop3 access. --- > > 10-ssl.conf > > SSL = yes > > 10-auth.conf > > disable_plaintext_auth = yes > > Mail.log > > pop3-login: Info: Disconnected (tried to use disallowed plaintext > auth): > user=<>, rip=IP, lip=ip, session= Provide the output of doveconf -n for better help. > > > Regards -- Christian Kivalo From ml+dovecot at valo.at Sat Feb 11 19:07:22 2017 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sat, 11 Feb 2017 20:07:22 +0100 Subject: no Authnticationmethode from outlook to Imap In-Reply-To: <008101d28481$256a5960$703f0c20$@gmx.ch> References: <007201d28475$77a46bf0$66ed43d0$@gmx.ch> <008101d28481$256a5960$703f0c20$@gmx.ch> Message-ID: Am 11. Februar 2017 17:08:55 MEZ schrieb Maurizio Caloro : >Thanks for your feedback...... > >I use 143 and if using 993 tell the system the the Certificate are >wrong ??? You are using a self-signed certificate and the certificate. That is ok, you need to make outlook accept the certificate. I don't know outlook but maybe you get a message and can accept the certificate? For a secure connection on port 143 enable the use of starttls in your outlook account, this way encryption is enabled after the initial connect to dovecot. Connection to port 993 (imaps) is encrypted from the start and secure too. To get rid of the certificate warning you need a proper certificate from e.g. lets encrypt >root at caloro:/etc/dovecot/conf.d# doveconf -n ># 2.2.13: /etc/dovecot/dovecot.conf ># OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.7 >auth_mechanisms = plain login >debug_log_path = /var/log/mail.log >info_log_path = /var/log/mail.log >log_path = /var/log/mail.log >mail_access_groups = vmail >mail_location = mbox:~/mail >managesieve_notify_capability = mailto >managesieve_sieve_capability = fileinto reject envelope >encoded-character >vacation subaddress comparator-i;ascii-numeric relational regex >imap4flags >copy include variables body enotify environment mailbox date ihave >namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = >} >passdb { > args = scheme=CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file >} >plugin { > sieve = ~/sieve/.dovecot.sieve > sieve_dir = ~/sieve >} >postmaster_address = postmaster at caloro.ch >protocols = " imap sieve pop3" >service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-client { > mode = 0660 > user = mail > } >} >ssl_cert = ssl_key = userdb { > args = username_format=%u /etc/dovecot/users > driver = passwd-file >} >protocol lda { > mail_plugins = " quota sieve" >} >protocol imap { > mail_plugins = " quota imap_quota" >} >root at caloro:/etc/dovecot/conf.d# > > > >-----Urspr?ngliche Nachricht----- >Von: dovecot [mailto:dovecot-bounces at dovecot.org] Im Auftrag von >Christian >Kivalo >Gesendet: Samstag, 11. Februar 2017 16:24 >An: dovecot at dovecot.org >Betreff: Re: no Authnticationmethode from outlook to Imap > > > >On 2017-02-11 15:45, Maurizio Caloro wrote: >> Hello >> >> Debian, with postfix and Dovecot Imap "2.2.13" running. >> >> >> >> If I want connect from Outlook or Thunderbird I will become this >error: >> Plaintext disalowd on non secure SSL/TLS Connection. >> >> but I will that this run with secure connection, I don't have any >> intention to use Plaintext auth. > >How is your Outlook account set up? Which port is configured in your >Outlook >account? >Port 143 using starttls or port 993 ssl? > >> Also I don't know why, dovecot will switch from IMAP --> POP3-Login. >> Or it's this Outlook that will ask for it. > >Dovecot does not switch from imap to pop3. Your Outlook seems to be >configured for pop3 access. > >--- > >> >> 10-ssl.conf >> >> SSL = yes >> >> 10-auth.conf >> >> disable_plaintext_auth = yes >> >> Mail.log >> >> pop3-login: Info: Disconnected (tried to use disallowed plaintext >> auth): >> user=<>, rip=IP, lip=ip, session= > >Provide the output of doveconf -n for better help. > >> >> >> Regards > >-- > Christian Kivalo From chibi at gol.com Sun Feb 12 06:58:16 2017 From: chibi at gol.com (Christian Balzer) Date: Sun, 12 Feb 2017 15:58:16 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <307B1E3A-F63C-4A01-A9D7-13AD27E083CB@my.walr.us> References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> <20170210175858.0079da4d@batzmaru.gol.ad.jp> <20170211010714.5790a617@batzmaru.gol.ad.jp> <307B1E3A-F63C-4A01-A9D7-13AD27E083CB@my.walr.us> Message-ID: <20170212155816.50032285@batzmaru.gol.ad.jp> Hello, On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > 1. 256GB of real RAM, swap is for chums. > > Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > I'm sure about tax and death, not much else. But as a rule of thumb I'd avoid swapping out stuff on production servers, even if it were to SSDs. Incidentally the servers I'm talking about here have their OS and swap on Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC S3610s. Relying on the kernel to make swap decisions is likely to result in much reduced performance even with fast SWAP when you're overcommitting things on that scale. But read on. > Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > > Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. > See IMAP hibernation: https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html I'm going to deploy/test this in production in about 2 months from now, but if you look at the link and the consequent changelog entries you'll see that it has certain shortcomings and bug fixes in pretty much each release after it was introduced. But this is the correct way to tackle things, not SWAP. Alas I'm not expecting miracles and if more than 20% of the IMAP sessions here will be hibernated at any given time I'd be pleasantly surprised. Because between: 1. Finding a sensible imap_hibernate_timeout. 2. Having well behaved clients that keep idling instead of restarting the sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/) 3. Having lots of mobile clients who either get disconnected (invisible to Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts (a large mobile carrier here times out idle TCP sessions after 2 minutes, forcing people to use 1 minute IDLE renewals, making 1. up there a nightmare). 4. Having really broken clients (don't ask, I can't tell) which open IMAP sessions, don't put them into IDLE and thus having them expire after 30 minutes. the pool of eligible IDLE sessions isn't as big as it could be, in my case at least. > My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > Scaling up is nice and desirable from a cost (rack space, HW) perspective, but the scalability of things OTHER than Dovecot as I pointed out plus that little detail of failure domains (do you really want half of your eggs in one basket?) argue for scaling out after a certain density. I'm feeling my way there at this time, but expect more than 100k sessions per server to be tricky. Lastly, when I asked about 500k sessions per server here not so long ago, ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) Timo mentioned that he's not aware of anybody doing more than 50k per server, something I got licked already and definitely will go to 100k eventually. Regards, Christian > > On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > > > > On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > > > >>> 1500 IMAP sessions will eat up about 3GB alone. > >> > >> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > >> > > That depends on the IMAP session, read the mailbox size and index size, > > etc. > > Some are significantly larger: > > --- > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > > --- > > > > But yes, as somebody who has mailbox servers with 55k+ session the average > > is around 1.6MB. > > > >> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > >> > > You will want: > > > 2. Understanding how to tune Dovecot and more importantly the overall > > system to such a task (see that PID up there?). > > 3. Be willing to deal with stuff like top and ps taking ages to start/run > > and others like atop actually killing dovecot (performance wise, not > > literally) when doing their obviously flawed cleanup on exit. Some things > > clearly do NOT scale well. > > > > My current goal is to have 100k capable servers that work well, 200k in a > > failover scenario, but that won't be particular enjoyable. > > > > Christian > > > >>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > >>> > >>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > >>> > >>>> hello > >>>> > >>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > >>>> > >>> > >>> Be very precise here, you expect to see 1500 as the result of > >>> "doveadm who |grep pop3 |wc -l"? > >>> > >>> Because that implies an ungodly number of POP3 connects per second, given > >>> the typically short duration of these. > >>> > >>> 1500 IMAP connections (note that frequently a client will have more than > >>> the INBOX open and thus have more than one session and thus process on the > >>> server) are a much easier proposition, provided they are of the typical > >>> long lasting type. > >>> > >>> So can you put a number to your expected logins per second (both protocols)? > >>> > >>>> my server > >>>> > >>>> server configuration > >>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > >>>> gb hdd for data (No raid) > >>>> > >>> No RAID and no other replication like DRBD? > >>> Why would you even bother? > >>> > >>> How many users/mailboxes in total with what quota? > >>> > >>> 1500 IMAP sessions will eat up about 3GB alone. > >>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > >>> dentries) in RAM. > >>> > >>> If you really have several hundreds logins/s, you're facing several > >>> bottlenecks: > >>> 1. Login processes themselves (easily fixed by high performance mode) > >>> 2. Auth processes (that will depend on your backends, method mostly) > >>> 3. Dovecot master process (spawning mail processes) > >>> > >>> The later is a single-threaded process, so it will benefit from a faster > >>> CPU core. > >>> It can be dramatically improved by enabling process re-usage, see: > >>> http://wiki.dovecot.org/PerformanceTuning > >>> > >>> However that also means more memory usage. > >>> > >>> > >>> > >>> Christian > >>> > >>>> > >>>> thanks > >>>> rajesh > >>>> > >>> > >>> [snip] > >>> -- > >>> Christian Balzer Network/Systems Engineer > >>> chibi at gol.com Global OnLine Japan/Rakuten Communications > >>> http://www.gol.com/ > >> > > > > > > -- > > Christian Balzer Network/Systems Engineer > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > http://www.gol.com/ -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From stephan at rename-it.nl Sun Feb 12 11:12:39 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 12:12:39 +0100 Subject: Managesieve cannot access script store In-Reply-To: <6fed7b56fee93b911001ece16bace8ee@tesla.demon.nl> References: <6fed7b56fee93b911001ece16bace8ee@tesla.demon.nl> Message-ID: <7747648b-6e31-881b-1a2a-446fb518b795@rename-it.nl> Op 2/10/2017 om 5:05 PM schreef dovelist: > Hi Stephan, > >> Normally, Dovecot permission errors are more helpful than that. So, this >> error message in itself is a bit of a bug: > > I'm glad to h've been able to help with this beta-test ;-) > > >> About the cause of this error: keep in mind that the whole directory >> path needs read/execute permission, not only the leaf directory. > > Have checked. They are... > > >> You could try a command other than LISTSCRIPTS in your manual debugging >> efforts. That should take a different code path that provides a more >> detailed error. > > I tried: > > PUTSCRIPT "hutsefluts" {6+} > keep; > > Gives the same result: > > Feb 10 15:43:26 p150 dovecot[2042]: managesieve(rogier): Error: sieve: > file storage: save: > open(/home/rogier/sieve/tmp/hutsefluts_1486737806.M728733P6414.p150.sieve) > failed: Permission denied Ah, so there's more. Fixed that too: https://github.com/dovecot/pigeonhole/commit/34d44f7ad9e872dec6ffa62de2642cb91ad5f6fc Regards, Stephan. From stephan at rename-it.nl Sun Feb 12 11:29:44 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 12:29:44 +0100 Subject: Managesieve cannot access script store In-Reply-To: <4659b4f84f968e42b8e771b695aacf83@tesla.demon.nl> References: <6fed7b56fee93b911001ece16bace8ee@tesla.demon.nl> <4659b4f84f968e42b8e771b695aacf83@tesla.demon.nl> Message-ID: Op 2/11/2017 om 3:24 PM schreef dovelist: > OK, I've figured it out: > > In the dovecot profile for apparmor the sieve directory is not > confgured. I solved it this way: > > To configure only one directory in the apparmor profile, I placed the > active-script link inside the .sieve directory. Keeping the scripts > separate in a store subdirectory, like this: > In /etc/dovecot/conf.d/90-sieve.conf : > > sieve = file:~/.sieve/store;active=~/.sieve/active.sieve > > Then dovecot is granted access by adding the .sieve directory in the > apparmor profile. The dovecot file in the tunables directory seems to > be a neat way to that: > In /etc/apparmor.d/tunables/dovecot : > > @{DOVECOT_MAILSTORE}=@{HOME}/Maildir/ /var/spool/mail/ @{HOME}/.sieve/ > > Ofcourse the .sieve directory is not really a MAILSTORE. But this way, > the configuration stays close to the defaults. I didn't find something > like DOVECOT_SIEVESTORE, which would be more appropriate. > > After restart of apparmor and dovecot, it works! > > @Stephan: thanks for the advice - it did help to pinpoint the problem! I have no experience with AppArmor. I assume these profile configuration files are created by the packagers for your distribution. You could talk to them to get this fixed in general. Regards, Stephan. From stephan at rename-it.nl Sun Feb 12 11:32:56 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 12:32:56 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <8d6af978-a5bf-48d2-2b11-7035acfeef3d@rename-it.nl> Op 2/10/2017 om 2:20 PM schreef Alessio Cecchi: > Il 10/02/2017 09:06, Aki Tuomi ha scritto: >> Hi! >> Since antispam plugin is deprecated and we would really prefer people >> not to use it, we wrote instructions on how to replace it with >> IMAPSieve. Comments and suggestions are most welcome. >> >> https://wiki.dovecot.org/HowTo/AntispamWithSieve > Hi, > > imap_stats plugin is required? No, that is just part of the example. Regards, Stephan. From aki.tuomi at dovecot.fi Sun Feb 12 11:39:50 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sun, 12 Feb 2017 13:39:50 +0200 (EET) Subject: Replacement for antispam plugin In-Reply-To: <8d6af978-a5bf-48d2-2b11-7035acfeef3d@rename-it.nl> References: <8d6af978-a5bf-48d2-2b11-7035acfeef3d@rename-it.nl> Message-ID: <286488174.1385.1486899591764@appsuite-dev.open-xchange.com> > On February 12, 2017 at 1:32 PM Stephan Bosch wrote: > > > Op 2/10/2017 om 2:20 PM schreef Alessio Cecchi: > > Il 10/02/2017 09:06, Aki Tuomi ha scritto: > >> Hi! > >> Since antispam plugin is deprecated and we would really prefer people > >> not to use it, we wrote instructions on how to replace it with > >> IMAPSieve. Comments and suggestions are most welcome. > >> > >> https://wiki.dovecot.org/HowTo/AntispamWithSieve > > Hi, > > > > imap_stats plugin is required? > > No, that is just part of the example. > > Regards, > > Stephan. And removed from the example too, thank you for pointing this out. Aki From stephan at rename-it.nl Sun Feb 12 11:47:45 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 12:47:45 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> Message-ID: <54efa29f-cdc5-e6a5-115d-3bd8dc34d4c0@rename-it.nl> Op 2/10/2017 om 8:46 PM schreef David Mehler: > Hello, > > Chiming in on this with a question, and will be getting to it over the > weekend or later this evening time permitting. > > Does retraining a message as either spam or ham alter message headers > for example x-spam or the spamassassin-modified subject header? > > If not is it possible to do so after processing? For example, I have a > message inadvertently tagged as spam, from Spamassassin it gets an > x-spam header added as well as a modified subject. Retraining that > message as ham moving it to say any other folder but spam i'd like for > that x-spam header to be set to as it is not spam, and the > spamassassin subject to be removed. Is this doable? I think you could use the "editheader" extension in the report-ham.sieve script to do that. You'll need "variables" as well to modify the subject. Regards, Stephan. From idefix at fechner.net Sun Feb 12 12:15:08 2017 From: idefix at fechner.net (Matthias Fechner) Date: Sun, 12 Feb 2017 13:15:08 +0100 Subject: Plugin sieve vs. imap_sieve Message-ID: <18bde76f-1f64-391f-0a51-9b42344baa73@fechner.net> Dear all, I just saw that a "new" plugin imap_sieve is available: https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve I configured everything using the plugin sieve. Was the plugin sieve just renamed to imap_sieve or is it completely different? Thanks a lot for explaining what had been changed here, I cannot find anything in the wiki. Gru? Matthias -- "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook From urushkin at telros.ru Sun Feb 12 12:21:05 2017 From: urushkin at telros.ru (Sergey Urushkin) Date: Sun, 12 Feb 2017 15:21:05 +0300 Subject: antispam plugin pipe backend error when moving multiple emails In-Reply-To: <9C553CA2-F238-4F7B-A0C8-C8164AE50B14@vixns.com> References: <1B80B1F2-2FA3-4F42-A8F7-DD8AEDFA0CB9@vixns.com> <9C553CA2-F238-4F7B-A0C8-C8164AE50B14@vixns.com> Message-ID: <374dbcdd947af8e2bbe996df5481e775@telros.ru> Hello, wondering why it's still an issue with current git, while this old working solution exists (tested with dovecot 2.22 and rspamd 1.4) : https://www.dovecot.org/list/dovecot/2013-November/093810.html --- Best regards, Sergey Urushkin St?phane Cottin ????? 2016-04-18 15:44: > Hello, > > I'm bumping this because it still occurs with dovecot 2.2.22. > > my dovecot-antispam plugin configuration : > > antispam_allow_append_to_spam = no > antispam_backend = pipe > antispam_pipe_program = /usr/bin/rspamc > antispam_pipe_program_args = -h;127.0.0.1:11334;-P;******** > antispam_pipe_program_notspam_arg = learn_ham > antispam_pipe_program_spam_arg = learn_spam > antispam_pipe_tmpdir = /var/tmp > antispam_spam = Junk > antispam_trash = trash;Trash;Deleted Items;Deleted Messages > > zlib enabled: > zlib_save = gz > zlib_save_level = 9 > > > When moving 2 or more messages from inbox to the Junk folder: > > "J47 NO [CANNOT] Failed to copy to temporary file (0.000 + 0.000 > secs).?. Command attempted: ?J47 UID MOVE 106318:106319 Junk" > > or sometimes > > "J123 NO [CANNOT] Failed to read mail beginning (0.000 + 0.000 > secs).?. Command attempted: ?J123 UID MOVE 170789:170790 Junk" > > and still have the "Cached message size smaller..." in dovecot logs. > > It occurs at least when header lines of an email contains Non-ASCII > Text (rfc1342). > Batches of full ascii emails are not affected. > > I can easily reproduce this from/to the Junk folder, but had > unconfirmed reports of similar errors when batch moving mails across > regular folders. > > St?phane > > > On 8 Nov 2015, at 11:50, St?phane Cottin wrote: > >> Hi, >> >> I've got some trouble with the dovecot antispam plugin and the pipe >> backend. >> >> I'm using dovecot 2.2.18 with maildirs and zlib compression enabled. >> >> When moving 2 or more emails at once from the Junk folder to another >> one, I always have the following error : "Failed to copy to temporary >> file" >> >> In the server logs : >> >> imap(vvv at vvv.vvv): Error: >> read(zlib(/data/Maildir/.test/tmp/1446974366.M123890P936.vvv)) failed: >> Cached message size smaller than expected (13553 < 13562, box=test, >> UID=0) >> >> The same operation with one email at a time, on the same emails, works >> as expected. >> >> St?phane From stephan at rename-it.nl Sun Feb 12 12:25:24 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 13:25:24 +0100 Subject: Replacement for antispam plugin In-Reply-To: <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> Message-ID: <80b1a77b-873c-ecb3-c2b1-c4544997d1ba@rename-it.nl> Op 2/10/2017 om 8:59 PM schreef Ralph Seichter: > On 10.02.17 20:34, Michael Slusarz wrote: > >> When you move a message to a new mailbox, that is a "new message" >> event (a new UID in the target mailbox is created; the message count >> increases). So imap.mailbox is set to the name of the *target* mailbox. > My tests seem to indicate otherwise. Deleting a message currently > located in the Junk folder causes the report-ham.sieve script to be > invoked, which in my case contains the following: > > require ["vnd.dovecot.pipe", "copy", "environment"]; > if environment "imap.mailbox" "Trash" { > stop; > } elsif environment "imap.mailbox" "Junk" { > pipe :copy "debug-junk"; > } else { > pipe :copy "learn-ham"; > } > > I can see that "learn-ham" is always invoked when a message is deleted > from or moved out of the Junk folder, so my guess is that imap.mailbox > is neither "Trash" nor "Junk" ("debug-junk" is never called, according > to the Dovecot logs). Unfortunately I don't know how to debug this > further. > > I don't want "learn-ham" to be run when a message located in the Junk > folder is manually deleted or moved to Trash, and right now I don't know > how to accomplish this. The "imap.mailbox" environment is the empty string in this case. Why? Well, the Sieve interpreter does not know about it, since the "imapsieve" extension is not activated in the require line. You could debug this with the non-standard "vnd.dovecot.debug" extension. I debugged it like this: require ["imapsieve", "environment", "variables", "vnd.dovecot.debug"]; if environment :matches "imap.mailbox" "*" { set "mailbox" "${1}"; } debug_log "imap.mailbox = ${mailbox}"; if string "${mailbox}" "Trash" { stop; } debug_log "REPORT HAM"; This will log one or two info messages, depending on what the target mailbox is. I tested this with the example configuration and it works fine. I've verified this by invoking the imap service directly from the command line, so that it is immediately logged-in and spews al debug messages directly to stderr: $ sudo /usr/lib/dovecot/imap -u harrie imap(harrie)<>: Debug: Loading modules from directory: /usr/lib/dovecot/modules imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_replication_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so imap(harrie)<>: Debug: auth USER input: harrie uid=1000 gid=124 home=/ext/test/home/test/harrie Debug: Effective uid=1000, gid=124, home=/ext/test/home/test/harrie Debug: replication: No mail_replica setting - replication disabled Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Debug: maildir++: root=/ext/test/home/test/harrie/Maildir, index=, indexpvt=, control=, inbox=/ext/test/home/test/harrie/Maildir, alt= Debug: Namespace : type=private, prefix=virtual/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=virtual:~/Maildir/virtual Debug: fs: root=/ext/test/home/test/harrie/Maildir/virtual, index=, indexpvt=, control=, inbox=, alt= * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE LITERAL+ SEARCH=FUZZY NOTIFY URLAUTH URLAUTH=BINARY METADATA SPECIAL-USE] Logged in as harrie 23423 SELECT Spam imap(harrie): Debug: Namespace : Using permissions from /ext/test/home/test/harrie/Maildir: mode=0700 gid=default * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1485122806] UIDs valid * OK [UIDNEXT 2] Predicted next UID 23423 OK [READ-WRITE] Select completed (0.000 + 0.000 secs). This is what happens for moving from Spam to a non-Trash mailbox: 2343 MOVE 1 "Frop" imap(harrie): Debug: imapsieve: mailbox Frop: MOVE event imap(harrie): Debug: sieve: Pigeonhole version 0.5.devel (462a535) initializing imap(harrie): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. imap(harrie): Debug: sieve: Pigeonhole Sieve PGP Encrypt plugin version 0.2.devel loaded imap(harrie): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.devel (462a535) loaded imap(harrie): Debug: imapsieve: Static mailbox rule [1]: mailbox=`Spam' from=`*' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none) imap(harrie): Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Spam' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none) imap(harrie): Debug: imapsieve: Matched static mailbox rule [2] imap(harrie): Debug: sieve: file storage: Performing auto-detection imap(harrie): Debug: sieve: file storage: Root exists (/ext/test/home/test/harrie) imap(harrie): Debug: sieve: file storage: Storage path `/ext/test/home/test/harrie/sieve' not found imap(harrie): Debug: sieve: file storage: Active script path is unconfigured; using default (path=~/.dovecot.sieve) imap(harrie): Debug: sieve: file storage: Using Sieve script path: /ext/test/home/test/harrie/.dovecot.sieve imap(harrie): Debug: sieve: file storage: Storage path `/ext/test/home/test/harrie/.dovecot.sieve' not found imap(harrie): Debug: sieve: storage: No default script location configured imap(harrie): Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve imap(harrie): Debug: sieve: file script: Opened script `report-ham' from `/usr/lib/dovecot/sieve/report-ham.sieve' imap(harrie): Debug: sieve: Opening script 1 of 1 from `/usr/lib/dovecot/sieve/report-ham.sieve' imap(harrie): Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve imap(harrie): Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded imap(harrie): Debug: sieve: binary save: not saving binary /usr/lib/dovecot/sieve/report-ham.svbin, because it is already stored imap(harrie): Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-ham.svbin' imap(harrie): Info: sieve: report-ham: line 11: DEBUG: imap.mailbox = Frop imap(harrie): Info: sieve: report-ham: line 17: DEBUG: REPORT HAM imap(harrie): Info: copy from Spam: box=Frop, uid=2, msgid=<201701181731.v0IHVjvo001751 at localhost.localdomain>, flags=() imap(harrie): Info: expunge: box=Spam, uid=1, msgid=<201701181731.v0IHVjvo001751 at localhost.localdomain>, flags=(\Recent) * OK [COPYUID 1485122805 1 2] Moved UIDs. * 1 EXPUNGE * 0 RECENT 2343 OK Move completed (0.089 + 0.000 + 0.088 secs). Alternatively, towards Trash: 2343 MOVE 1 Trash imap(harrie): Debug: /ext/test/home/test/harrie/Maildir/.Trash/dovecot.index.cache: Compressed, file_seq changed 0 -> 1486901076, size=32, max_uid=0 imap(harrie): Debug: imapsieve: mailbox Trash: MOVE event imap(harrie): Debug: Mailbox Trash: Opened mail UID=0 because: header Message-ID (Cache file is empty, reset_id=1486901076) imap(harrie): Debug: imapsieve: Matched static mailbox rule [2] imap(harrie): Debug: sieve: file storage: Performing auto-detection imap(harrie): Debug: sieve: file storage: Root exists (/ext/test/home/test/harrie) imap(harrie): Debug: sieve: file storage: Storage path `/ext/test/home/test/harrie/sieve' not found imap(harrie): Debug: sieve: file storage: Active script path is unconfigured; using default (path=~/.dovecot.sieve) imap(harrie): Debug: sieve: file storage: Using Sieve script path: /ext/test/home/test/harrie/.dovecot.sieve imap(harrie): Debug: sieve: file storage: Storage path `/ext/test/home/test/harrie/.dovecot.sieve' not found imap(harrie): Debug: sieve: storage: No default script location configured imap(harrie): Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve imap(harrie): Debug: sieve: file script: Opened script `report-ham' from `/usr/lib/dovecot/sieve/report-ham.sieve' imap(harrie): Debug: sieve: Opening script 1 of 1 from `/usr/lib/dovecot/sieve/report-ham.sieve' imap(harrie): Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve imap(harrie): Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded imap(harrie): Debug: sieve: binary save: not saving binary /usr/lib/dovecot/sieve/report-ham.svbin, because it is already stored imap(harrie): Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-ham.svbin' imap(harrie): Info: sieve: report-ham: line 11: DEBUG: imap.mailbox = Trash imap(harrie): Info: copy from Spam: box=Trash, uid=1, msgid=<201701181731.v0IHVjvo001751 at localhost.localdomain>, flags=() imap(harrie): Info: expunge: box=Spam, uid=2, msgid=<201701181731.v0IHVjvo001751 at localhost.localdomain>, flags=(\Recent) * OK [COPYUID 1485122804 2 1] Moved UIDs. * 1 EXPUNGE * 0 RECENT 2343 OK Move completed (0.090 + 0.000 + 0.089 secs). If you want to debug this in even more detail, you can enable trace debugging for the targeted user (or the whole server if you really want to). The imapsieve plugins should obey this as well: https://github.com/dovecot/pigeonhole/blob/master/INSTALL#L551 This is apparently not currently documented in the wiki. Regards, Stephan. From stephan at rename-it.nl Sun Feb 12 12:36:00 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 13:36:00 +0100 Subject: Plugin sieve vs. imap_sieve In-Reply-To: <18bde76f-1f64-391f-0a51-9b42344baa73@fechner.net> References: <18bde76f-1f64-391f-0a51-9b42344baa73@fechner.net> Message-ID: <8ee78584-7af5-f4e5-ca04-9ec34902d1b6@rename-it.nl> Op 2/12/2017 om 1:15 PM schreef Matthias Fechner: > Dear all, > > > I just saw that a "new" plugin imap_sieve is available: > https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve > > I configured everything using the plugin sieve. > > Was the plugin sieve just renamed to imap_sieve or is it completely > different? It is completely different. The sieve plugin is a plugin for LDA and LMTP, which perform final delivery of a message. The imap_sieve plugin is for IMAP and invokes Sieve scripts for messages that are already stored. > Thanks a lot for explaining what had been changed here, I cannot find > anything in the wiki. For one, the wiki page you referenced should explain that pretty well. You could also read the referenced http://tools.ietf.org/html/rfc6785 to find out what features this plugin adds in detail. Regards, Stephan. From stephan at rename-it.nl Sun Feb 12 12:46:28 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 13:46:28 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> Message-ID: Op 2/10/2017 om 10:15 PM schreef George Kontostanos: > On Fri, Feb 10, 2017 at 9:59 PM, Ralph Seichter > wrote: > > > Same problem here. As a workaround I tried the following: > > # From Spam folder to Inbox > imapsieve_mailbox2_name = Inbox > imapsieve_mailbox2_from = Spam > imapsieve_mailbox2_causes = COPY > imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve > > In theory this should trigger the report-ham script, only if a mail is > moved from Spam to Inbox. However, it does not seem to work either. That is because you found a rather stupid, yet minor bug. The "INBOX" mailbox is rather special, since it is the only mailbox name that is treated case-insensitively. The normal form is "INBOX". Due to the bug, your "Inbox" will not match "INBOX". You can work around this for now by specifying "INBOX" instead. Well, you will not need this workaround for a workaround anyway if the fix I proposed in my earlier message also works for you. But for posterity... Regards, Stephan. From yacinechaouche at yahoo.com Sun Feb 12 13:18:56 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Sun, 12 Feb 2017 13:18:56 +0000 (UTC) Subject: Maildirsize not updated In-Reply-To: <589CBACD.2000908@carpenter.org> References: <824227861.1251135.1486644520643.ref@mail.yahoo.com> <824227861.1251135.1486644520643@mail.yahoo.com> <21b1ac9e-43fb-32e7-5ca8-e6267491eb38@dovecot.fi> <758119982.1220532.1486646410142@mail.yahoo.com> <589CBACD.2000908@carpenter.org> Message-ID: <2020836776.3079273.1486905536280@mail.yahoo.com> I am using dovecot lmtp root at messagerie[10.10.10.19] ~ # grep virtual_transport /etc/postfix/main.cf # transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf # virtual_transport = maildrop virtual_transport = lmtp:unix:private/dovecot-lmtp root at messagerie[10.10.10.19] ~ # On Thursday, February 9, 2017 7:54 PM, WJCarpenter wrote: Who delievers incoming mail, dovecot LDA or something else? This is what caused a similar problem for me: https://dovecot.org/list/dovecot/2016-April/104091.html From yacinechaouche at yahoo.com Sun Feb 12 13:24:03 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Sun, 12 Feb 2017 13:24:03 +0000 (UTC) Subject: Maildirsize not updated In-Reply-To: <2020836776.3079273.1486905536280@mail.yahoo.com> References: <824227861.1251135.1486644520643.ref@mail.yahoo.com> <824227861.1251135.1486644520643@mail.yahoo.com> <21b1ac9e-43fb-32e7-5ca8-e6267491eb38@dovecot.fi> <758119982.1220532.1486646410142@mail.yahoo.com> <589CBACD.2000908@carpenter.org> <2020836776.3079273.1486905536280@mail.yahoo.com> Message-ID: <1727387869.3027241.1486905843950@mail.yahoo.com> Now this is interesting : du, doveadm quota get and maildirsize have three different values for this particular user : Max Quota is : 1G du : 883M (86%) maildirsize : 1048M (102%) doveadm : 34402? (32%) Trace ----- root at messagerie[10.10.10.19] ~ # cd /var/vmail/domain.tld/m.stefan/ root at messagerie[10.10.10.19] ~ # alias dush alias dush='du -h --max-depth=1 | sort -h' root at messagerie[10.10.10.19] /var/vmail/domain.tld/m.stefan # dush 4.0K ./courierimapkeywords 4.0K ./new 4.0K ./tmp 8.0K ./courierimaphieracl 28K ./.INBOX.Altert 28K ./.INBOX.notifs 36K ./.BOITE_RECEPTION 36K ./.PIGE 80K ./dovecot 428K ./.Drafts 728K ./.Alert Process 1020K ./.Trash 1.3M ./.Junk 3.4M ./cur 3.6M ./.Alert BLANC 7.5M ./.Alert PUB 9.7M ./.IT SUPPORT 9.9M ./.GLPI 12M ./.REGIE 15M ./.LTO 27M ./.VDN 29M ./.Sent 43M ./.HD SUPPORT 105M ./.MAM 105M ./.PRTG 514M ./.INBOX 883M . root at messagerie[10.10.10.19] /var/vmail/domain.tld/m.stefan # cat maildirsize 1073741824S 1098898439 9380 -382366814 -439 382 1 27713 1 -383309268 -489 27433 1 4274 1 27740 1 31452 1 -665709 -153 5722 1 5797 1 27713 1 158840 1 27203 1 27744 1 26032 1 27717 1 1807 1 26989 1 27152 1 26239 1 3066 1 30846 1 4272 1 26020 1 27713 1 30040 1 26753 1 27152 1 27125 1 27744 1 26559 1 26166 1 29845 1 27121 1 27740 1 88624 1 32080 1 26672 1 27121 1 27152 1 26343 1 31456 1 27717 1 27152 1 26343 1 26032 1 29847 1 28309 1 29709 1 26241 1 26343 1 27152 1 31360 1 26016 1 27717 1 30857 1 27148 1 26343 1 28309 1 27152 1 26343 1 26168 1 28305 1 27822 1 27152 1 47273 1 26343 1 32056 1 26460 1 30523 1 28141 1 28152 1 28309 1 27148 1 26343 1 3999 1 1688 1 27990 1 28305 1 27152 1 26343 1 26032 1 27928 1 27479 1 90009 1 27366 1 27264 1 69705 1 27261 1 48383 1 28309 1 26241 1 27152 1 32079 1 30474 1 26016 1 30146 1 27713 1 29818 1 27152 1 30769 1 29194 1 27717 1 27740 1 1508 1 1536 1 1999 1 26146 1 1550 1 1587 1 27125 1 47289 1 27496 1 27744 1 32080 1 26464 1 32048 1 27713 1 27744 1 29845 1 27121 1 27744 1 26032 1 30146 1 27121 1 45454 1 26241 1 28332 1 32103 1 3859 1 26016 1 27121 1 28336 1 4272 1 29709 1 29688 1 27125 1 28336 1 1757 1 4631 1 54951 1 26170 1 1757 1 1975 1 29765 1 28882 1 1757 1 25683 1 71184 1 28332 1 32080 1 55040 1 26464 1 1757 1 4631 1 30706 1 4322 1 49607 1 1757 1 1757 1 46535 1 47378 1 27717 1 27744 1 21723 1 1759 1 1757 1 20218 1 21737 1 21724 1 539443 1 1892 1 27713 1 27744 1 26036 1 27713 1 27740 1 32079 1 12815596 1 26523 1 54511 1 26020 1 27125 1 29847 1 28336 1 30423 1 27009 1 27065 1 26510 1 27121 1 324880 1 27740 1 21128 1 213811 1 20186 1 20239 1 26146 1 21360 1 29481 1 25668 1 27826 1 28128 1 27125 1 31333 1 1820 1 28332 1 26488 1 27778 1 31127 1 4290 1 4290 1 4309 1 27121 1 27744 1 4290 1 8871 1 8871 1 4290 1 1773 1 30578 1 47705 1 10764 1 10764 1 30433 1 31066 1 4290 1 30461 1 31094 1 26680 1 29135 1 29141 1 26032 1 30146 1 30109 1 31054 1 29226 1 31483 1 26016 1 29949 1 30890 1 30095 1 29226 1 5880 1 29173 1 30422 1 root at messagerie[10.10.10.19] /var/vmail/domain.tld/m.stefan # doveadm quota get -u m.stefan at domain.tld Quota name Type Value Limit % User quota STORAGE 344402 1048576 32 User quota MESSAGE 8542 - 0 root at messagerie[10.10.10.19] /var/vmail/domain.tld/m.stefan # On Sunday, February 12, 2017 2:18 PM, chaouche yacine wrote: I am using dovecot lmtp root at messagerie[10.10.10.19] ~ # grep virtual_transport /etc/postfix/main.cf # transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf # virtual_transport = maildrop virtual_transport = lmtp:unix:private/dovecot-lmtp root at messagerie[10.10.10.19] ~ # On Thursday, February 9, 2017 7:54 PM, WJCarpenter wrote: Who delievers incoming mail, dovecot LDA or something else? This is what caused a similar problem for me: https://dovecot.org/list/dovecot/2016-April/104091.html From kevin at my.walr.us Sun Feb 12 13:27:21 2017 From: kevin at my.walr.us (KT Walrus) Date: Sun, 12 Feb 2017 08:27:21 -0500 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <20170212155816.50032285@batzmaru.gol.ad.jp> References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> <20170210175858.0079da4d@batzmaru.gol.ad.jp> <20170211010714.5790a617@batzmaru.gol.ad.jp> <307B1E3A-F63C-4A01-A9D7-13AD27E083CB@my.walr.us> <20170212155816.50032285@batzmaru.gol.ad.jp> Message-ID: <6F57185D-1F7D-4327-8714-02D152EAA2B8@my.walr.us> Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > On Feb 12, 2017, at 1:58 AM, Christian Balzer wrote: > > > Hello, > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > >>> 1. 256GB of real RAM, swap is for chums. >> >> Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? >> > > I'm sure about tax and death, not much else. > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > even if it were to SSDs. > Incidentally the servers I'm talking about here have their OS and swap on > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > S3610s. > > Relying on the kernel to make swap decisions is likely to result in much > reduced performance even with fast SWAP when you're overcommitting things > on that scale. > > > But read on. > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. >> >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. >> > See IMAP hibernation: > https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html > > I'm going to deploy/test this in production in about 2 months from now, > but if you look at the link and the consequent changelog entries you'll see > that it has certain shortcomings and bug fixes in pretty much each release > after it was introduced. > > But this is the correct way to tackle things, not SWAP. > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > here will be hibernated at any given time I'd be pleasantly surprised. > > Because between: > > 1. Finding a sensible imap_hibernate_timeout. > > 2. Having well behaved clients that keep idling instead of restarting the > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ ) > > 3. Having lots of mobile clients who either get disconnected (invisible to > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > forcing people to use 1 minute IDLE renewals, making 1. up there a > nightmare). > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > sessions, don't put them into IDLE and thus having them expire after 30 > minutes. > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > at least. > >> My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... >> > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > but the scalability of things OTHER than Dovecot as I pointed out plus > that little detail of failure domains (do you really want half of your > eggs in one basket?) argue for scaling out after a certain density. > > I'm feeling my way there at this time, but expect more than 100k sessions > per server to be tricky. > > Lastly, when I asked about 500k sessions per server here not so long ago, > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) > Timo mentioned that he's not aware of anybody doing more than 50k per > server, something I got licked already and definitely will go to 100k > eventually. > > Regards, > > Christian >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: >>> >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: >>> >>>>> 1500 IMAP sessions will eat up about 3GB alone. >>>> >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? >>>> >>> That depends on the IMAP session, read the mailbox size and index size, >>> etc. >>> Some are significantly larger: >>> --- >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap >>> --- >>> >>> But yes, as somebody who has mailbox servers with 55k+ session the average >>> is around 1.6MB. >>> >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? >>>> >>> You will want: >> >>> 2. Understanding how to tune Dovecot and more importantly the overall >>> system to such a task (see that PID up there?). >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run >>> and others like atop actually killing dovecot (performance wise, not >>> literally) when doing their obviously flawed cleanup on exit. Some things >>> clearly do NOT scale well. >>> >>> My current goal is to have 100k capable servers that work well, 200k in a >>> failover scenario, but that won't be particular enjoyable. >>> >>> Christian >>> >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: >>>>> >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: >>>>> >>>>>> hello >>>>>> >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. >>>>>> >>>>> >>>>> Be very precise here, you expect to see 1500 as the result of >>>>> "doveadm who |grep pop3 |wc -l"? >>>>> >>>>> Because that implies an ungodly number of POP3 connects per second, given >>>>> the typically short duration of these. >>>>> >>>>> 1500 IMAP connections (note that frequently a client will have more than >>>>> the INBOX open and thus have more than one session and thus process on the >>>>> server) are a much easier proposition, provided they are of the typical >>>>> long lasting type. >>>>> >>>>> So can you put a number to your expected logins per second (both protocols)? >>>>> >>>>>> my server >>>>>> >>>>>> server configuration >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 >>>>>> gb hdd for data (No raid) >>>>>> >>>>> No RAID and no other replication like DRBD? >>>>> Why would you even bother? >>>>> >>>>> How many users/mailboxes in total with what quota? >>>>> >>>>> 1500 IMAP sessions will eat up about 3GB alone. >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, >>>>> dentries) in RAM. >>>>> >>>>> If you really have several hundreds logins/s, you're facing several >>>>> bottlenecks: >>>>> 1. Login processes themselves (easily fixed by high performance mode) >>>>> 2. Auth processes (that will depend on your backends, method mostly) >>>>> 3. Dovecot master process (spawning mail processes) >>>>> >>>>> The later is a single-threaded process, so it will benefit from a faster >>>>> CPU core. >>>>> It can be dramatically improved by enabling process re-usage, see: >>>>> http://wiki.dovecot.org/PerformanceTuning >>>>> >>>>> However that also means more memory usage. >>>>> >>>>> >>>>> >>>>> Christian >>>>> >>>>>> >>>>>> thanks >>>>>> rajesh >>>>>> >>>>> >>>>> [snip] >>>>> -- >>>>> Christian Balzer Network/Systems Engineer >>>>> chibi at gol.com Global OnLine Japan/Rakuten Communications >>>>> http://www.gol.com/ >>>> >>> >>> >>> -- >>> Christian Balzer Network/Systems Engineer >>> chibi at gol.com > Global OnLine Japan/Rakuten Communications >>> http://www.gol.com/ > > > > -- > Christian Balzer Network/Systems Engineer > chibi at gol.com Global OnLine Japan/Rakuten Communications > http://www.gol.com/ From aki.tuomi at dovecot.fi Sun Feb 12 13:52:55 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sun, 12 Feb 2017 15:52:55 +0200 (EET) Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> > On February 10, 2017 at 10:06 AM Aki Tuomi wrote: > > > Hi! > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with > IMAPSieve. Comments and suggestions are most welcome. > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > > --- > Aki Tuomi > Dovecot oy Hi everyone, thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script. Aki From dovecot-ml at seichter.de Sun Feb 12 14:12:33 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Sun, 12 Feb 2017 15:12:33 +0100 Subject: Replacement for antispam plugin In-Reply-To: <80b1a77b-873c-ecb3-c2b1-c4544997d1ba@rename-it.nl> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> <80b1a77b-873c-ecb3-c2b1-c4544997d1ba@rename-it.nl> Message-ID: <288098f2-c89c-0c0c-7f5c-dcadb9411ddd@seichter.de> On 12.02.2017 13:25, Stephan Bosch wrote: > The "imap.mailbox" environment is the empty string in this case. Why? > Well, the Sieve interpreter does not know about it, since the > "imapsieve" extension is not activated in the require line. Now there's a facepalm moment. ;-) Thank you, with a modified 'require' statement things are working for me. I see that https://wiki.dovecot.org/HowTo/AntispamWithSieve has already been updated, that's nice. > You could debug this with the non-standard "vnd.dovecot.debug" > extension. Thanks again, I will keep this in mind for future debugging. -Ralph From gkontos.mail at gmail.com Sun Feb 12 16:36:45 2017 From: gkontos.mail at gmail.com (George Kontostanos) Date: Sun, 12 Feb 2017 18:36:45 +0200 Subject: Replacement for antispam plugin In-Reply-To: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: On Sun, Feb 12, 2017 at 3:52 PM, Aki Tuomi wrote: > > > On February 10, 2017 at 10:06 AM Aki Tuomi wrote: > > > > > > Hi! > > Since antispam plugin is deprecated and we would really prefer people > > not to use it, we wrote instructions on how to replace it with > > IMAPSieve. Comments and suggestions are most welcome. > > > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > > > > --- > > Aki Tuomi > > Dovecot oy > > Hi everyone, > > thank you all for your feedback, questions and comments. We have upgraded > the documentation based on this, including information how to exclude Trash > folder in ham script. > > Aki > Thank you all very much. I am now running into a very weird issue. Whenever an Junk email is ?seen? I get the following error: Debug: sieve: vnd.dovecot.execute extension: no bin or socket directory specified; extension is unconfigured (both sieve_execute_bin_dir and sieve_execute_socket_dir are not set) Feb 12 18:02:54 imap(user at domain): Debug: imapsieve: Static mailbox rule [1]: mailbox=`Junk' from=`*' causes=(COPY) => before=`file:/usr/local/lib/dovecot/sieve/report-spam.sieve' after=(none) Feb 12 18:02:54 imap(user at dmain): Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Junk' causes=(COPY) => before=`file:/usr/local/lib/dovecot/sieve/report-ham.sieve' after=(none) Also it automatically creates a .spamassassin/ folder in the user: -rw------- 1 vmail vmail 136 Feb 10 17:33 .dovecot.lda-dupes lrwx------ 1 vmail vmail 17 Mar 13 2016 .dovecot.sieve@ -> managesieve.sieve -rw------- 1 vmail vmail 322 Feb 11 03:02 .dovecot.svbin drwx------ 2 vmail vmail 512 Feb 12 18:04 .spamassassin/ drwx------ 5 vmail vmail 512 Feb 10 17:32 Drafts/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Junk/ drwx------ 5 vmail vmail 512 Feb 11 17:36 Saved/ drwx------ 5 vmail vmail 512 Jan 16 11:55 Sent/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Trash/ drwx------ 2 vmail vmail 1024 Feb 12 18:07 cur/ -rw------- 1 vmail vmail 21 Mar 13 2016 dovecot-acl-list -rw------- 1 vmail vmail 68 Oct 17 13:19 dovecot-keywords -rw------- 1 vmail vmail 245 Feb 12 18:07 dovecot-uidlist -rw------- 1 vmail vmail 8 Jun 9 2016 dovecot-uidvalidity -r--r--r-- 1 vmail vmail 0 Mar 12 2016 dovecot-uidvalidity.56e48129 -rw------- 1 vmail vmail 640 Feb 11 22:24 dovecot.index -rw------- 1 vmail vmail 27788 Feb 12 18:07 dovecot.index.cache -rw------- 1 vmail vmail 25996 Feb 12 18:07 dovecot.index.log -rw------- 1 vmail vmail 192 Feb 12 16:04 dovecot.mailbox.log -rw------- 1 vmail vmail 181 Feb 10 17:34 managesieve.sieve drwx------ 2 vmail vmail 512 Feb 12 17:46 new/ -rw------- 1 vmail vmail 39 Jun 9 2016 subscriptions drwx------ 2 vmail vmail 512 Feb 12 18:04 tmp/ Any ideas ? I am attaching my config. Thanks # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 10.3-RELEASE-p11 amd64 ufs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 2560 default_process_limit = 512 dict { acl = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } log_path = /var/log/dovecot.log mail_home = /usr/local/vhosts/mail/%d/%n mail_location = maildir:/usr/local/vhosts/mail/%d/%n:LAYOUT=fs mail_max_userip_connections = 20 mail_plugins = quota acl mail_privileged_group = vmail mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve mbox_write_locks = fcntl namespace { inbox = no list = children location = maildir:/usr/local/vhosts/mail/%%d/%%n:LAYOUT=fs:INDEX=/usr/local/vhosts/indexes/%d/%n/shared/%%u:INDEXPVT=/usr/local/vhosts/indexes/%d/%n/shared/%%u prefix = shared/%%d/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl imapsieve_mailbox1_before = file:/usr/local/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = /usr/local/vhosts/mail/%d/%n/.dovecot.sieve sieve_before = /usr/local/vhosts/sieve/before.d/ sieve_dir = /usr/local/vhosts/mail/%d/%n sieve_global_dir = /usr/local/vhosts/sieve/%d sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_global_path = /usr/local/vhosts/sieve/%d/default.sieve sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = imap lmtp sieve sieve service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl_cert = References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: On 12.02.2017 17:36, George Kontostanos wrote: > it automatically creates a .spamassassin/ folder in the user That happens because sa-learn is invoked as the user who is logged into IMAP. If you want all users to contribute to a global SpamAssassin database (like I do), you'll need to create your own learning mechanism instead of calling sa-learn directly. -Ralph From gkontos.mail at gmail.com Sun Feb 12 18:05:18 2017 From: gkontos.mail at gmail.com (George Kontostanos) Date: Sun, 12 Feb 2017 20:05:18 +0200 Subject: Replacement for antispam plugin In-Reply-To: References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: On Sun, Feb 12, 2017 at 7:52 PM, Ralph Seichter wrote: > On 12.02.2017 17:36, George Kontostanos wrote: > >> it automatically creates a .spamassassin/ folder in the user > > That happens because sa-learn is invoked as the user who is logged into > IMAP. If you want all users to contribute to a global SpamAssassin > database (like I do), you'll need to create your own learning mechanism > instead of calling sa-learn directly. > > -Ralph Actually I think that sa-learn is invoked as user vmail. But of course I might be wrong. sa-learn-ham.sh LOG='/var/log/sa-learn.log' exec /usr/local/bin/sa-learn --ham -D >> $LOG 2>&1 I had to give ownership to vmail to sa-learn.log otherwise it would refuse to run. Do you have any suggestions as per the way sa-learn should be executed? Thanks for your help. -- George Kontostanos --- From stephan at rename-it.nl Sun Feb 12 18:56:38 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 12 Feb 2017 19:56:38 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: Op 2/12/2017 om 5:36 PM schreef George Kontostanos: > On Sun, Feb 12, 2017 at 3:52 PM, Aki Tuomi wrote: > >>> On February 10, 2017 at 10:06 AM Aki Tuomi wrote: >>> >>> >>> Hi! >>> Since antispam plugin is deprecated and we would really prefer people >>> not to use it, we wrote instructions on how to replace it with >>> IMAPSieve. Comments and suggestions are most welcome. >>> >>> https://wiki.dovecot.org/HowTo/AntispamWithSieve >>> >>> --- >>> Aki Tuomi >>> Dovecot oy >> Hi everyone, >> >> thank you all for your feedback, questions and comments. We have upgraded >> the documentation based on this, including information how to exclude Trash >> folder in ham script. >> >> Aki >> > > Thank you all very much. I am now running into a very weird issue. Whenever > an Junk email is ?seen? I get the following error: > > Debug: sieve: vnd.dovecot.execute extension: no bin or socket directory > specified; extension is unconfigured (both sieve_execute_bin_dir and > sieve_execute_socket_dir are not set) > Feb 12 18:02:54 imap(user at domain): Debug: imapsieve: Static mailbox rule > [1]: mailbox=`Junk' from=`*' causes=(COPY) => > before=`file:/usr/local/lib/dovecot/sieve/report-spam.sieve' after=(none) > Feb 12 18:02:54 imap(user at dmain): Debug: imapsieve: Static mailbox rule > [2]: mailbox=`*' from=`Junk' causes=(COPY) => > before=`file:/usr/local/lib/dovecot/sieve/report-ham.sieve' after=(none) That is not an error. That is merely a debug message indicating that the vnd.dovecot.execute extension is unconfigured. Don't enable that extension if you're not using the "execute" command/test. Yes, it is in the example, but it has no function either. Only the vnd.dovecot.pipe extension is actually used ("pipe" command). > Also it automatically creates a .spamassassin/ folder in the user: > > -rw------- 1 vmail vmail 136 Feb 10 17:33 .dovecot.lda-dupes > lrwx------ 1 vmail vmail 17 Mar 13 2016 .dovecot.sieve@ -> > managesieve.sieve > -rw------- 1 vmail vmail 322 Feb 11 03:02 .dovecot.svbin > drwx------ 2 vmail vmail 512 Feb 12 18:04 .spamassassin/ > drwx------ 5 vmail vmail 512 Feb 10 17:32 Drafts/ > drwx------ 5 vmail vmail 512 Feb 12 18:09 Junk/ > drwx------ 5 vmail vmail 512 Feb 11 17:36 Saved/ > drwx------ 5 vmail vmail 512 Jan 16 11:55 Sent/ > drwx------ 5 vmail vmail 512 Feb 12 18:09 Trash/ > drwx------ 2 vmail vmail 1024 Feb 12 18:07 cur/ > -rw------- 1 vmail vmail 21 Mar 13 2016 dovecot-acl-list > -rw------- 1 vmail vmail 68 Oct 17 13:19 dovecot-keywords > -rw------- 1 vmail vmail 245 Feb 12 18:07 dovecot-uidlist > -rw------- 1 vmail vmail 8 Jun 9 2016 dovecot-uidvalidity > -r--r--r-- 1 vmail vmail 0 Mar 12 2016 dovecot-uidvalidity.56e48129 > -rw------- 1 vmail vmail 640 Feb 11 22:24 dovecot.index > -rw------- 1 vmail vmail 27788 Feb 12 18:07 dovecot.index.cache > -rw------- 1 vmail vmail 25996 Feb 12 18:07 dovecot.index.log > -rw------- 1 vmail vmail 192 Feb 12 16:04 dovecot.mailbox.log > -rw------- 1 vmail vmail 181 Feb 10 17:34 managesieve.sieve > drwx------ 2 vmail vmail 512 Feb 12 17:46 new/ > -rw------- 1 vmail vmail 39 Jun 9 2016 subscriptions > drwx------ 2 vmail vmail 512 Feb 12 18:04 tmp/ > > Any ideas ? I am attaching my config. The likely scenario is that the sa-learn tool is creating a hidden directory inside the $HOME directory of the user for user-specific state information. According to your mail_home and mail_location configuration, that is the same directory as the INBOX mailbox. That is why this may be interpreted as a mailbox by the maildir format. It is generally a bad idea to have those equal; you should put the mail location in a sub-directory of the home directory to prevent problems like this. https://wiki.dovecot.org/VirtualUsers/Home Regards, Stephan. From dovecot-ml at seichter.de Sun Feb 12 19:01:49 2017 From: dovecot-ml at seichter.de (Ralph Seichter) Date: Sun, 12 Feb 2017 20:01:49 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: On 12.02.17 19:05, George Kontostanos wrote: > Actually I think that sa-learn is invoked as user vmail. But of course > I might be wrong. It might depend on system configuration. On my servers, Sieve scripts are definitely executed as the OS user that matches the current IMAP user. > Do you have any suggestions as per the way sa-learn should be executed? Instead of calling sa-learn directly, I use a script to store the piped raw message in the file system, using separate directories for ham and spam. This way, there are no delays. A periodic Cron job, running as my global SpamAssassin user, later collects the files and invokes sa-learn, ensuring that sync operations only happen once per run. -Ralph From gkontos.mail at gmail.com Sun Feb 12 19:13:08 2017 From: gkontos.mail at gmail.com (George Kontostanos) Date: Sun, 12 Feb 2017 21:13:08 +0200 Subject: Replacement for antispam plugin In-Reply-To: References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: On Sun, Feb 12, 2017 at 8:56 PM, Stephan Bosch wrote: > Op 2/12/2017 om 5:36 PM schreef George Kontostanos: >> On Sun, Feb 12, 2017 at 3:52 PM, Aki Tuomi wrote: >> >>>> On February 10, 2017 at 10:06 AM Aki Tuomi wrote: >>>> >>>> >>>> Hi! >>>> Since antispam plugin is deprecated and we would really prefer people >>>> not to use it, we wrote instructions on how to replace it with >>>> IMAPSieve. Comments and suggestions are most welcome. >>>> >>>> https://wiki.dovecot.org/HowTo/AntispamWithSieve >>>> >>>> --- >>>> Aki Tuomi >>>> Dovecot oy >>> Hi everyone, >>> >>> thank you all for your feedback, questions and comments. We have upgraded >>> the documentation based on this, including information how to exclude Trash >>> folder in ham script. >>> >>> Aki >>> >> >> Thank you all very much. I am now running into a very weird issue. Whenever >> an Junk email is ?seen? I get the following error: >> >> Debug: sieve: vnd.dovecot.execute extension: no bin or socket directory >> specified; extension is unconfigured (both sieve_execute_bin_dir and >> sieve_execute_socket_dir are not set) >> Feb 12 18:02:54 imap(user at domain): Debug: imapsieve: Static mailbox rule >> [1]: mailbox=`Junk' from=`*' causes=(COPY) => >> before=`file:/usr/local/lib/dovecot/sieve/report-spam.sieve' after=(none) >> Feb 12 18:02:54 imap(user at dmain): Debug: imapsieve: Static mailbox rule >> [2]: mailbox=`*' from=`Junk' causes=(COPY) => >> before=`file:/usr/local/lib/dovecot/sieve/report-ham.sieve' after=(none) > > That is not an error. That is merely a debug message indicating that the > vnd.dovecot.execute extension is unconfigured. > > Don't enable that extension if you're not using the "execute" > command/test. Yes, it is in the example, but it has no function either. > Only the vnd.dovecot.pipe extension is actually used ("pipe" command). > >> Also it automatically creates a .spamassassin/ folder in the user: >> >> -rw------- 1 vmail vmail 136 Feb 10 17:33 .dovecot.lda-dupes >> lrwx------ 1 vmail vmail 17 Mar 13 2016 .dovecot.sieve@ -> >> managesieve.sieve >> -rw------- 1 vmail vmail 322 Feb 11 03:02 .dovecot.svbin >> drwx------ 2 vmail vmail 512 Feb 12 18:04 .spamassassin/ >> drwx------ 5 vmail vmail 512 Feb 10 17:32 Drafts/ >> drwx------ 5 vmail vmail 512 Feb 12 18:09 Junk/ >> drwx------ 5 vmail vmail 512 Feb 11 17:36 Saved/ >> drwx------ 5 vmail vmail 512 Jan 16 11:55 Sent/ >> drwx------ 5 vmail vmail 512 Feb 12 18:09 Trash/ >> drwx------ 2 vmail vmail 1024 Feb 12 18:07 cur/ >> -rw------- 1 vmail vmail 21 Mar 13 2016 dovecot-acl-list >> -rw------- 1 vmail vmail 68 Oct 17 13:19 dovecot-keywords >> -rw------- 1 vmail vmail 245 Feb 12 18:07 dovecot-uidlist >> -rw------- 1 vmail vmail 8 Jun 9 2016 dovecot-uidvalidity >> -r--r--r-- 1 vmail vmail 0 Mar 12 2016 dovecot-uidvalidity.56e48129 >> -rw------- 1 vmail vmail 640 Feb 11 22:24 dovecot.index >> -rw------- 1 vmail vmail 27788 Feb 12 18:07 dovecot.index.cache >> -rw------- 1 vmail vmail 25996 Feb 12 18:07 dovecot.index.log >> -rw------- 1 vmail vmail 192 Feb 12 16:04 dovecot.mailbox.log >> -rw------- 1 vmail vmail 181 Feb 10 17:34 managesieve.sieve >> drwx------ 2 vmail vmail 512 Feb 12 17:46 new/ >> -rw------- 1 vmail vmail 39 Jun 9 2016 subscriptions >> drwx------ 2 vmail vmail 512 Feb 12 18:04 tmp/ >> >> Any ideas ? I am attaching my config. > > The likely scenario is that the sa-learn tool is creating a hidden > directory inside the $HOME directory of the user for user-specific state > information. According to your mail_home and mail_location > configuration, that is the same directory as the INBOX mailbox. That is > why this may be interpreted as a mailbox by the maildir format. It is > generally a bad idea to have those equal; you should put the mail > location in a sub-directory of the home directory to prevent problems > like this. > > https://wiki.dovecot.org/VirtualUsers/Home > > Regards, > > Stephan. > > > That makes perfect sense. I might need to find a way to migrate now to the correct structure. Thanks -- George Kontostanos --- From ebroch at whitehorsetc.com Sun Feb 12 20:44:52 2017 From: ebroch at whitehorsetc.com (ebroch at whitehorsetc.com) Date: Sun, 12 Feb 2017 20:44:52 +0000 (UTC) Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> Any opinion on dspam's interoperability with this? On Fri, Feb 10, 2017 at 1:07 AM -0700, "Aki Tuomi" wrote: Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome. https://wiki.dovecot.org/HowTo/AntispamWithSieve --- Aki Tuomi Dovecot oy From hakon at alstadheim.priv.no Sun Feb 12 21:52:32 2017 From: hakon at alstadheim.priv.no (=?UTF-8?Q?H=c3=a5kon_Alstadheim?=) Date: Sun, 12 Feb 2017 22:52:32 +0100 Subject: Replacement for antispam plugin In-Reply-To: <288098f2-c89c-0c0c-7f5c-dcadb9411ddd@seichter.de> References: <081018fb-960e-4544-e34d-606e414088f2@seichter.de> <20170210150936.3ich5dkvrwhtbp7r@darac.org.uk> <1283894883.1074.1486748060346@appsuite-dev.open-xchange.com> <2d40dc0f-e013-be6e-18fe-d26052d692d3@seichter.de> <720467326.1194.1486755291036@appsuite-dev.open-xchange.com> <01775064-0cd6-65d0-5fe3-62e40c9b123a@seichter.de> <80b1a77b-873c-ecb3-c2b1-c4544997d1ba@rename-it.nl> <288098f2-c89c-0c0c-7f5c-dcadb9411ddd@seichter.de> Message-ID: <9b533fd6-11e3-8233-e7ae-2247163f91b5@alstadheim.priv.no> Den 12. feb. 2017 15:12, skrev Ralph Seichter: > On 12.02.2017 13:25, Stephan Bosch wrote: > >> The "imap.mailbox" environment is the empty string in this case. Why? >> Well, the Sieve interpreter does not know about it, since the >> "imapsieve" extension is not activated in the require line. > > Now there's a facepalm moment. ;-) Thank you, with a modified 'require' > statement things are working for me. > > I see that https://wiki.dovecot.org/HowTo/AntispamWithSieve has already > been updated, that's nice. Tried to add a small variation for dSpam, but there is some anti-spam functionality that I don't understand. Mind putting this in after the sa-learn-ham.sh: ? ----------- Or, if you are using dspam, (dropping 'sa-' as that would be misleading) learn-spam.sh {{{ #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam }}} learn-ham.sh {{{ #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --debug --source=error --class=innocent }}} ------------- > >> You could debug this with the non-standard "vnd.dovecot.debug" >> extension. > > Thanks again, I will keep this in mind for future debugging. > > -Ralph > From hakon at alstadheim.priv.no Sun Feb 12 22:00:49 2017 From: hakon at alstadheim.priv.no (=?UTF-8?Q?H=c3=a5kon_Alstadheim?=) Date: Sun, 12 Feb 2017 23:00:49 +0100 Subject: Replacement for antispam plugin In-Reply-To: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> References: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> Message-ID: Den 12. feb. 2017 21:44, skrev ebroch at whitehorsetc.com: > > > Any opinion on dspam's interoperability with this? > > Just follow the wiki and replace sa-learn scripts with calling dspam. Dspam direct pipe needs mail-line-endings (\r\n) translated into unix line-endings (\r). like so: ----- learn-spam.sh --------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam ----- learn-ham.sh ---------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent ------ Theese will be invoked with the owner of the mailbox as current uid, so that is all. From darix at opensu.se Sun Feb 12 22:56:52 2017 From: darix at opensu.se (Marcus Rueckert) Date: Sun, 12 Feb 2017 23:56:52 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> Message-ID: <20170212225652.cha5orgr2bbwl2rw@nordisch.org> On 2017-02-12 23:00:49 +0100, H?kon Alstadheim wrote: > Just follow the wiki and replace sa-learn scripts with calling dspam. > Dspam direct pipe needs mail-line-endings (\r\n) translated into unix > line-endings (\r). tbh ... what do you do about mails which just use \r as separator? s|\r\n|\n| is safer. darix > like so: > > ----- learn-spam.sh --------- > #!/bin/sh > sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam > > ----- learn-ham.sh ---------- > #!/bin/sh > sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent > > ------ > > Theese will be invoked with the owner of the mailbox as current uid, so > that is all. -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From stephan at rename-it.nl Mon Feb 13 00:28:52 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2017 01:28:52 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> Message-ID: <677646b9-26ad-bb18-3c34-f44374491635@rename-it.nl> Op 2/12/2017 om 11:00 PM schreef H?kon Alstadheim: > > Den 12. feb. 2017 21:44, skrev ebroch at whitehorsetc.com: >> >> Any opinion on dspam's interoperability with this? >> >> > Just follow the wiki and replace sa-learn scripts with calling dspam. > Dspam direct pipe needs mail-line-endings (\r\n) translated into unix > line-endings (\r). > > like so: > > ----- learn-spam.sh --------- > #!/bin/sh > sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam > > ----- learn-ham.sh ---------- > #!/bin/sh > sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent > > ------ > > Theese will be invoked with the owner of the mailbox as current uid, so > that is all. Actually, Pigeonhole should be able to do that too: https://github.com/dovecot/pigeonhole/blob/master/doc/plugins/sieve_extprograms.txt#L112 Yes, I need to update the wiki. Regards, Stephan. From hakon at alstadheim.priv.no Mon Feb 13 01:30:19 2017 From: hakon at alstadheim.priv.no (=?UTF-8?Q?H=c3=a5kon_Alstadheim?=) Date: Mon, 13 Feb 2017 02:30:19 +0100 Subject: Replacement for antispam plugin In-Reply-To: <20170212225652.cha5orgr2bbwl2rw@nordisch.org> References: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> <20170212225652.cha5orgr2bbwl2rw@nordisch.org> Message-ID: <996fbb9a-6175-3107-03bd-c64712bd48c6@alstadheim.priv.no> Den 12. feb. 2017 23:56, skrev Marcus Rueckert: > On 2017-02-12 23:00:49 +0100, H?kon Alstadheim wrote: >> Just follow the wiki and replace sa-learn scripts with calling dspam. >> Dspam direct pipe needs mail-line-endings (\r\n) translated into unix >> line-endings (\r). I had a typo here, (\r\n) gets replaced by just (\n). > > tbh ... what do you do about mails which just use \r as separator? > You are mistaken. Firstly: s/\r$// will remove \r from the end of any line, do nothing if there is no \r. Secondly: All line-endings as seen by transfer-agents is transferred with the same line endings (MTA-MTA: \r\n; locally: usually the same but may be different). "Line ending" here pertains to after each header, and between message-parts. If there are naked (\n) characters in the message body, that is of no concern. Point of filter is to make Dspam-signature parseable for dspam. If there is an (\r) at the end of the header, Dspam will not find a match, and be unable to reclassify the mail. > s|\r\n|\n| is safer. > > darix > >> like so: >> >> ----- learn-spam.sh --------- >> #!/bin/sh >> sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam >> >> ----- learn-ham.sh ---------- >> #!/bin/sh >> sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent >> >> ------ >> >> Theese will be invoked with the owner of the mailbox as current uid, so >> that is all. > From chibi at gol.com Mon Feb 13 01:46:13 2017 From: chibi at gol.com (Christian Balzer) Date: Mon, 13 Feb 2017 10:46:13 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <6F57185D-1F7D-4327-8714-02D152EAA2B8@my.walr.us> References: <6533315E7A7A468A98EE140E872D2F4C.MAI@ns1.24x7server.net> <20170210175858.0079da4d@batzmaru.gol.ad.jp> <20170211010714.5790a617@batzmaru.gol.ad.jp> <307B1E3A-F63C-4A01-A9D7-13AD27E083CB@my.walr.us> <20170212155816.50032285@batzmaru.gol.ad.jp> <6F57185D-1F7D-4327-8714-02D152EAA2B8@my.walr.us> Message-ID: <20170213104613.14908ffe@batzmaru.gol.ad.jp> Hello, On Sun, 12 Feb 2017 08:27:21 -0500 KT Walrus wrote: > Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. > Those servers would actually be the wrong ones to look at, as they are primarily accessed by the aforementioned broken client, so the numbers are somewhat skewed. However looking at other servers with a more "normal" user spread things aren't actually too different. The average number of sessions per user tends to be 2. The clear majority (over 50%) only has one session open (people with well behaved and configured clients watching the INBOX mostly). Another 30% has 2 sessions open, again the typical state of this would be clients watching another mailbox besides INBOX, typically SENT. The rest has 3 and more sessions open. The number of sessions could of course be drastically reduced if any client would support IMAP NOTIFY, alas none that I'm aware of do. Lastly no more than 60% of the session seem to be in IDLE at any given time, so my comments about RAM and IMAP hibernation effectiveness stand. > Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > Yes to Dovecot Proxying, of course. No idea about MySQL, with (Open)LDAP nothing is breaking a sweat at an average of 140 logins per second (IMAP and POP) on the 2 proxy servers. If you can fit your entire dataset into RAM it should be fine, my LDAP servers fall into that category and take about 10% of a slow (1.2GHz, 34%, power-save mode) core only to handle the that load (also 2 servers). And the rate of logins/s is what you need to worry about most and optimize for. The proxies will of course have to do the shuffling of data and SSL en/de-coding, but again they're not particular busy with that. The number of sessions comes into play when looking at the number of login processes on the proxies and their memory footprint. An IMAP login process on the proxies in performance mode with a client limit of 1000 will consume about 55MB at most. So assume at least 55KB RAM per session. Read Timo's mail I linked to about IMAP hibernation, AFAIK nothing has happened to make proxies more supportive for this though. Christian > > On Feb 12, 2017, at 1:58 AM, Christian Balzer wrote: > > > > > > Hello, > > > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > > >>> 1. 256GB of real RAM, swap is for chums. > >> > >> Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > >> > > > > I'm sure about tax and death, not much else. > > > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > > even if it were to SSDs. > > Incidentally the servers I'm talking about here have their OS and swap on > > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > > S3610s. > > > > Relying on the kernel to make swap decisions is likely to result in much > > reduced performance even with fast SWAP when you're overcommitting things > > on that scale. > > > > > > But read on. > > > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > >> > >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. > >> > > See IMAP hibernation: > > https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html > > > > I'm going to deploy/test this in production in about 2 months from now, > > but if you look at the link and the consequent changelog entries you'll see > > that it has certain shortcomings and bug fixes in pretty much each release > > after it was introduced. > > > > But this is the correct way to tackle things, not SWAP. > > > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > > here will be hibernated at any given time I'd be pleasantly surprised. > > > > Because between: > > > > 1. Finding a sensible imap_hibernate_timeout. > > > > 2. Having well behaved clients that keep idling instead of restarting the > > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ ) > > > > 3. Having lots of mobile clients who either get disconnected (invisible to > > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > > forcing people to use 1 minute IDLE renewals, making 1. up there a > > nightmare). > > > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > > sessions, don't put them into IDLE and thus having them expire after 30 > > minutes. > > > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > > at least. > > > >> My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > >> > > > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > > but the scalability of things OTHER than Dovecot as I pointed out plus > > that little detail of failure domains (do you really want half of your > > eggs in one basket?) argue for scaling out after a certain density. > > > > I'm feeling my way there at this time, but expect more than 100k sessions > > per server to be tricky. > > > > Lastly, when I asked about 500k sessions per server here not so long ago, > > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) > > Timo mentioned that he's not aware of anybody doing more than 50k per > > server, something I got licked already and definitely will go to 100k > > eventually. > > > > Regards, > > > > Christian > >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > >>> > >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > >>> > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > >>>> > >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > >>>> > >>> That depends on the IMAP session, read the mailbox size and index size, > >>> etc. > >>> Some are significantly larger: > >>> --- > >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > >>> --- > >>> > >>> But yes, as somebody who has mailbox servers with 55k+ session the average > >>> is around 1.6MB. > >>> > >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > >>>> > >>> You will want: > >> > >>> 2. Understanding how to tune Dovecot and more importantly the overall > >>> system to such a task (see that PID up there?). > >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run > >>> and others like atop actually killing dovecot (performance wise, not > >>> literally) when doing their obviously flawed cleanup on exit. Some things > >>> clearly do NOT scale well. > >>> > >>> My current goal is to have 100k capable servers that work well, 200k in a > >>> failover scenario, but that won't be particular enjoyable. > >>> > >>> Christian > >>> > >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > >>>>> > >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > >>>>> > >>>>>> hello > >>>>>> > >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > >>>>>> > >>>>> > >>>>> Be very precise here, you expect to see 1500 as the result of > >>>>> "doveadm who |grep pop3 |wc -l"? > >>>>> > >>>>> Because that implies an ungodly number of POP3 connects per second, given > >>>>> the typically short duration of these. > >>>>> > >>>>> 1500 IMAP connections (note that frequently a client will have more than > >>>>> the INBOX open and thus have more than one session and thus process on the > >>>>> server) are a much easier proposition, provided they are of the typical > >>>>> long lasting type. > >>>>> > >>>>> So can you put a number to your expected logins per second (both protocols)? > >>>>> > >>>>>> my server > >>>>>> > >>>>>> server configuration > >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > >>>>>> gb hdd for data (No raid) > >>>>>> > >>>>> No RAID and no other replication like DRBD? > >>>>> Why would you even bother? > >>>>> > >>>>> How many users/mailboxes in total with what quota? > >>>>> > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > >>>>> dentries) in RAM. > >>>>> > >>>>> If you really have several hundreds logins/s, you're facing several > >>>>> bottlenecks: > >>>>> 1. Login processes themselves (easily fixed by high performance mode) > >>>>> 2. Auth processes (that will depend on your backends, method mostly) > >>>>> 3. Dovecot master process (spawning mail processes) > >>>>> > >>>>> The later is a single-threaded process, so it will benefit from a faster > >>>>> CPU core. > >>>>> It can be dramatically improved by enabling process re-usage, see: > >>>>> http://wiki.dovecot.org/PerformanceTuning > >>>>> > >>>>> However that also means more memory usage. > >>>>> > >>>>> > >>>>> > >>>>> Christian > >>>>> > >>>>>> > >>>>>> thanks > >>>>>> rajesh > >>>>>> > >>>>> > >>>>> [snip] > >>>>> -- > >>>>> Christian Balzer Network/Systems Engineer > >>>>> chibi at gol.com Global OnLine Japan/Rakuten Communications > >>>>> http://www.gol.com/ > >>>> > >>> > >>> > >>> -- > >>> Christian Balzer Network/Systems Engineer > >>> chibi at gol.com > Global OnLine Japan/Rakuten Communications > >>> http://www.gol.com/ > > > > > > > -- > > Christian Balzer Network/Systems Engineer > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > http://www.gol.com/ > -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From ygrishin-lists at mail2.ca Mon Feb 13 02:27:36 2017 From: ygrishin-lists at mail2.ca (ygrishin-lists at mail2.ca) Date: Sun, 12 Feb 2017 19:27:36 -0700 Subject: Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22. In-Reply-To: References: <9df9b8cb09e1c47a6c5f72f5bfa1b696@mail2.ca> Message-ID: <9647ac34537ffa06b59474b960a79278@mail2.ca> On 2017-02-08 00:10, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Sun, 5 Feb 2017, ygrishin-lists at mail2.ca wrote: > >> service dict { >> unix_listener dict { >> mode = 0660 >> user = Debian-exim >> group = Debian-exim >> } >> } >> >> dovecot-lda-erros.log: >> ********************** >> Feb 04 14:23:33 lda(testuser at XXX): Error: read(/var/run/dovecot/dict) >> failed: Remote disconnected >> >> dovecot.log: >> ************ >> Feb 04 13:57:06 imap(YYY at XXX): Error: write(/var/run/dovecot/dict) >> failed: Broken pipe >> ... >> >> dovecot-debug.log: >> ****************** >> Feb 04 13:18:12 lda(YYY at XXX): Error: read(/var/run/dovecot/dict) >> failed: Remote disconnected >> Feb 04 13:18:12 lda(YYY at XXX): Error: dict quota: Quota update failed, >> it's now desynced >> Feb 04 13:57:07 lda(testuser at XXX): Error: write(/var/run/dovecot/dict) >> failed: Broken pipe > > Does a process listens on /var/run/dovecot/dict ? It certainly does: #lsof /var/run/dovecot/dict COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME dovecot 1140 root 42u unix 0xc6fe2300 0t0 15861 /var/run/dovecot/dict type=STREAM $ ls -l /var/run/dovecot/dict srw-rw---- 1 Debian-exim Debian-exim 0 Feb 12 03:53 /var/run/dovecot/dict > The socket is accessable by Debian-exim:Debian-exim only (0660). As > what user and group does the LDA and imap service run as? LDA works as dovecot:Debian-exim: ********************************* lda: driver = pipe ... group = Debian-exim ... (without specifying the user explicitly). Yuriy From lucabert at lucabert.de Mon Feb 13 09:26:28 2017 From: lucabert at lucabert.de (Luca Bertoncello) Date: Mon, 13 Feb 2017 09:26:28 +0000 Subject: Problem with Horde "Mailbox does not support mod-sequences" Message-ID: <20170213092628.Horde.Lmd0RKrTCFRKd7p3edRKhTk@horde.lucabert.de> Hi list! I already asked about this problem about two years ago, but I couldn't solve my problem... Now I have a new Server, with Debian 8 and Dovecot 2.2.13-12 (from Debian repositories) and Horde 5.2.13. When I delete an E-Mail, I always get the error "Mailbox does not support mod-sequences". It results in having the E-Mail not moved to Trash and I must update more times the folder to see the E-Mail moved to Trash... Could someone help me to add this support for mod-sequences? Thanks a lot! Luca Bertoncello (lucabert at lucabert.de) From doug at mail.sermon-archive.info Mon Feb 13 09:47:21 2017 From: doug at mail.sermon-archive.info (Doug Hardie) Date: Mon, 13 Feb 2017 01:47:21 -0800 Subject: Problem with Horde "Mailbox does not support mod-sequences" In-Reply-To: <20170213092628.Horde.Lmd0RKrTCFRKd7p3edRKhTk@horde.lucabert.de> References: <20170213092628.Horde.Lmd0RKrTCFRKd7p3edRKhTk@horde.lucabert.de> Message-ID: > On 13 February 2017, at 01:26, Luca Bertoncello wrote: > > Hi list! > > I already asked about this problem about two years ago, but I couldn't solve my problem... > > Now I have a new Server, with Debian 8 and Dovecot 2.2.13-12 (from Debian repositories) and Horde 5.2.13. > > When I delete an E-Mail, I always get the error "Mailbox does not support mod-sequences". > It results in having the E-Mail not moved to Trash and I must update more times the folder to see the E-Mail moved to Trash... > > Could someone help me to add this support for mod-sequences? > A quick search turned up: https://dovecot.org/list/dovecot/2015-February/099674.html Perhaps that will help. From lucabert at lucabert.de Mon Feb 13 09:59:31 2017 From: lucabert at lucabert.de (Luca Bertoncello) Date: Mon, 13 Feb 2017 09:59:31 +0000 Subject: Problem with Horde "Mailbox does not support mod-sequences" In-Reply-To: References: <20170213092628.Horde.Lmd0RKrTCFRKd7p3edRKhTk@horde.lucabert.de> Message-ID: <20170213095931.Horde.C8G-IUIEwGf9vb3V51k3UVf@horde.lucabert.de> Zitat von Doug Hardie : > A quick search turned up: > https://dovecot.org/list/dovecot/2015-February/099674.html > > Perhaps that will help. Hello Doug, did you read the post? :) It was __MY__ post where I said that the solution suggested by Michael Slusarz didn't help me and asked for other suggestion... Continuing this post: - how can I check if mod-sequences are enabled? - how can I check WHY they don't work for this account (at least two accounts have the problem) Thanks Luca Bertoncello (lucabert at lucabert.de) From skdovecot at smail.inf.fh-brs.de Mon Feb 13 14:25:57 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 13 Feb 2017 15:25:57 +0100 (CET) Subject: Unable to use encrypted password for imap and pop3 In-Reply-To: <372581e1ed1bf2c79bc9b7b7537a4ebb@msw.it> References: <372581e1ed1bf2c79bc9b7b7537a4ebb@msw.it> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 11 Feb 2017, Davide Marchi wrote: > ssl_cert = ssl_key = unsubscribe -- -- JF Pion /*Marisol Touraine, Ministre de la Sant? ? propos du virus Zika:*/ *"Quand on a un projet de grossesse, il faut avoir des relations sexuelles prot?g?es parce que le virus peut se transmettre par la voie sexuelle. "* Oui, mais?enfin, s?rieusement quoi ? ?? elle r?fl?chit parfois ?? --- L'absence de virus dans ce courrier ?lectronique a ?t? v?rifi?e par le logiciel antivirus Avast. https://www.avast.com/antivirus From luciano at vespaperitivo.it Mon Feb 13 15:45:21 2017 From: luciano at vespaperitivo.it (Luciano Mannucci) Date: Mon, 13 Feb 2017 16:45:21 +0100 Subject: pop3 login core dump Message-ID: <3vMVHj4FrnzRRrq@baobab.bilink.it> Hello! Sorry for my newbie question, I've notied some lines in my logfile like this: Feb 13 16:05:54 pop3-login: Fatal: master: service(pop3-login): child 10660 killed with signal 11 (core not dumped - add -D parameter to service pop3-login { executable } ... where am I supposed to add the -D option in order to get the core file dumped so I can examine it? Thanks to all, luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster at sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.lesassaie.IT/ From luciano at vespaperitivo.it Mon Feb 13 17:14:00 2017 From: luciano at vespaperitivo.it (Luciano Mannucci) Date: Mon, 13 Feb 2017 18:14:00 +0100 Subject: pop3 login core dump In-Reply-To: <3vMVHj4FrnzRRrq@baobab.bilink.it> References: <3vMVHj4FrnzRRrq@baobab.bilink.it> Message-ID: <3vMXG113rczRRrs@baobab.bilink.it> On Mon, 13 Feb 2017 16:45:21 +0100 Luciano Mannucci wrote: > where am I supposed to add the -D option in order to get the core file > dumped so I can examine it? Ops, forgotten to post my doveconf. Here is it: # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 3.0.101-105-pae i686 openSUSE 11.4 (i586) ext3 auth_cache_size = 3 k auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes default_client_limit = 1228 default_vsz_limit = 712 M disable_plaintext_auth = no first_valid_gid = 0 first_valid_uid = 100 info_log_path = /var/log/dovecot/logfile.info listen = * log_path = /var/log/dovecot/logfile login_greeting = Dovecot at Baobab ready. login_trusted_networks = 127.0.0.0/8 212.45.144.0/24 192.168.134.0/24 mail_location = mbox:/var/spool/mailboxes/%u:INBOX=/var/spool/mail/%u:DIRNAME=mbox:INDEX=/var/dovecot_indexes/%u maildir_copy_with_hardlinks = no managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { inbox = yes location = prefix = separator = . type = private } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = pop3 imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { inet_listener imaps { address = * } process_limit = 512 } service pop3-login { inet_listener pop3s { address = * } process_limit = 512 } ssl_ca = Hi, I am seeing the followin error in my logs (doveconf -n at the bottom of this mail): Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: cs3NOQ7moVj4sQXXXXX: sieve: sieve file backend: invalid option `active=~/.dovecot.sieve' Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: cs3NOQ7moVj4sQXXXXX: sieve: failed to access user's Sieve script file:~/sieve;active=~/.dovecot.sieve (temporary failure) Looking at http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration, the syntax "sieve = file:~/sieve;active=~/.dovecot.sieve" in my config is correct ? Is this a false error that is only appearing because this is a newly created user with no sieve file ? If this is the case, how do I tell dovecot not to error out and allow the mail ? Thanks ! # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.6.1.el7.x86_64 x86_64 CentOS Linux release 7.3.1611 (Core) auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1 first_valid_uid = 1000 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body environment mailbox date ihave enotify mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = its_virtmail mode = 0660 user = its_virtmail } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = its_virtmail } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieves { address = port = 5190 ssl = yes } } ssl = required ssl_cert = was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 20 } From 24x7server at 24x7server.net Mon Feb 13 17:23:23 2017 From: 24x7server at 24x7server.net (Rajesh M) Date: Mon, 13 Feb 2017 22:53:23 +0530 Subject: dovecot config for 1500 simultaneous connection Message-ID: <224C0082B2F940C2861338B315BDA802.MAI@ns1.24x7server.net> thanks for your help happy to say that the performance dramatically improved after i use the high performance settings from here http://wiki.dovecot.org/LoginProcess grep Login: /var/log/mail.log.1 |wc -l with the mail.log being of a typical, busy day. 412992 i also picked up the imap and pop3 connections during peak hours [root at ns1 domains]# doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' username # proto (pids) (ips) max figures i got was 535 for imap and 97 for pop Question 1 i wish to improve the performance further by caching the logins. current the same is kept disable because when user's change passwords then they are not able to immediately login with the new password for some time. How to solve this issue. further to the above. i am planning to put up a large mail server with one of the following options option 1 OS and mailqueue : 2 X 600 gb 15k rpm raid 1 data : 4 X 2000 gb raid 10 for data backup : 2 X 2000 gb raid 10 for backup 32 gb ram option 2 os and mail queue : 4 X 600 gb 15k rpm raid 10 data : 4 X 2000 gb raid 10 for data 32 gb ram i am having the OS and mail queue on primary drives i understand that Raid10 give a faster write and read whereas Raid 1 will have slow writes and fast reads the question is will there be huge performance difference between Raid1 and Raid 10 when it comes small files like mail queue ? thanks rajesh ----- Original Message ----- From: Christian Balzer [mailto:chibi at gol.com] To: dovecot at dovecot.org Cc: kevin at my.walr.us Sent: Mon, 13 Feb 2017 10:46:13 +0900 Subject: Hello, On Sun, 12 Feb 2017 08:27:21 -0500 KT Walrus wrote: > Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. > Those servers would actually be the wrong ones to look at, as they are primarily accessed by the aforementioned broken client, so the numbers are somewhat skewed. However looking at other servers with a more "normal" user spread things aren't actually too different. The average number of sessions per user tends to be 2. The clear majority (over 50%) only has one session open (people with well behaved and configured clients watching the INBOX mostly). Another 30% has 2 sessions open, again the typical state of this would be clients watching another mailbox besides INBOX, typically SENT. The rest has 3 and more sessions open. The number of sessions could of course be drastically reduced if any client would support IMAP NOTIFY, alas none that I'm aware of do. Lastly no more than 60% of the session seem to be in IDLE at any given time, so my comments about RAM and IMAP hibernation effectiveness stand. > Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > Yes to Dovecot Proxying, of course. No idea about MySQL, with (Open)LDAP nothing is breaking a sweat at an average of 140 logins per second (IMAP and POP) on the 2 proxy servers. If you can fit your entire dataset into RAM it should be fine, my LDAP servers fall into that category and take about 10% of a slow (1.2GHz, 34%, power-save mode) core only to handle the that load (also 2 servers). And the rate of logins/s is what you need to worry about most and optimize for. The proxies will of course have to do the shuffling of data and SSL en/de-coding, but again they're not particular busy with that. The number of sessions comes into play when looking at the number of login processes on the proxies and their memory footprint. An IMAP login process on the proxies in performance mode with a client limit of 1000 will consume about 55MB at most. So assume at least 55KB RAM per session. Read Timo's mail I linked to about IMAP hibernation, AFAIK nothing has happened to make proxies more supportive for this though. Christian > > On Feb 12, 2017, at 1:58 AM, Christian Balzer wrote: > > > > > > Hello, > > > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > > >>> 1. 256GB of real RAM, swap is for chums. > >> > >> Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > >> > > > > I'm sure about tax and death, not much else. > > > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > > even if it were to SSDs. > > Incidentally the servers I'm talking about here have their OS and swap on > > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > > S3610s. > > > > Relying on the kernel to make swap decisions is likely to result in much > > reduced performance even with fast SWAP when you're overcommitting things > > on that scale. > > > > > > But read on. > > > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > >> > >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. > >> > > See IMAP hibernation: > > https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html > > > > I'm going to deploy/test this in production in about 2 months from now, > > but if you look at the link and the consequent changelog entries you'll see > > that it has certain shortcomings and bug fixes in pretty much each release > > after it was introduced. > > > > But this is the correct way to tackle things, not SWAP. > > > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > > here will be hibernated at any given time I'd be pleasantly surprised. > > > > Because between: > > > > 1. Finding a sensible imap_hibernate_timeout. > > > > 2. Having well behaved clients that keep idling instead of restarting the > > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ ) > > > > 3. Having lots of mobile clients who either get disconnected (invisible to > > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > > forcing people to use 1 minute IDLE renewals, making 1. up there a > > nightmare). > > > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > > sessions, don't put them into IDLE and thus having them expire after 30 > > minutes. > > > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > > at least. > > > >> My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > >> > > > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > > but the scalability of things OTHER than Dovecot as I pointed out plus > > that little detail of failure domains (do you really want half of your > > eggs in one basket?) argue for scaling out after a certain density. > > > > I'm feeling my way there at this time, but expect more than 100k sessions > > per server to be tricky. > > > > Lastly, when I asked about 500k sessions per server here not so long ago, > > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) > > Timo mentioned that he's not aware of anybody doing more than 50k per > > server, something I got licked already and definitely will go to 100k > > eventually. > > > > Regards, > > > > Christian > >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > >>> > >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > >>> > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > >>>> > >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > >>>> > >>> That depends on the IMAP session, read the mailbox size and index size, > >>> etc. > >>> Some are significantly larger: > >>> --- > >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > >>> --- > >>> > >>> But yes, as somebody who has mailbox servers with 55k+ session the average > >>> is around 1.6MB. > >>> > >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > >>>> > >>> You will want: > >> > >>> 2. Understanding how to tune Dovecot and more importantly the overall > >>> system to such a task (see that PID up there?). > >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run > >>> and others like atop actually killing dovecot (performance wise, not > >>> literally) when doing their obviously flawed cleanup on exit. Some things > >>> clearly do NOT scale well. > >>> > >>> My current goal is to have 100k capable servers that work well, 200k in a > >>> failover scenario, but that won't be particular enjoyable. > >>> > >>> Christian > >>> > >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > >>>>> > >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > >>>>> > >>>>>> hello > >>>>>> > >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > >>>>>> > >>>>> > >>>>> Be very precise here, you expect to see 1500 as the result of > >>>>> "doveadm who |grep pop3 |wc -l"? > >>>>> > >>>>> Because that implies an ungodly number of POP3 connects per second, given > >>>>> the typically short duration of these. > >>>>> > >>>>> 1500 IMAP connections (note that frequently a client will have more than > >>>>> the INBOX open and thus have more than one session and thus process on the > >>>>> server) are a much easier proposition, provided they are of the typical > >>>>> long lasting type. > >>>>> > >>>>> So can you put a number to your expected logins per second (both protocols)? > >>>>> > >>>>>> my server > >>>>>> > >>>>>> server configuration > >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > >>>>>> gb hdd for data (No raid) > >>>>>> > >>>>> No RAID and no other replication like DRBD? > >>>>> Why would you even bother? > >>>>> > >>>>> How many users/mailboxes in total with what quota? > >>>>> > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > >>>>> dentries) in RAM. > >>>>> > >>>>> If you really have several hundreds logins/s, you're facing several > >>>>> bottlenecks: > >>>>> 1. Login processes themselves (easily fixed by high performance mode) > >>>>> 2. Auth processes (that will depend on your backends, method mostly) > >>>>> 3. Dovecot master process (spawning mail processes) > >>>>> > >>>>> The later is a single-threaded process, so it will benefit from a faster > >>>>> CPU core. > >>>>> It can be dramatically improved by enabling process re-usage, see: > >>>>> http://wiki.dovecot.org/PerformanceTuning > >>>>> > >>>>> However that also means more memory usage. > >>>>> > >>>>> > >>>>> > >>>>> Christian > >>>>> > >>>>>> > >>>>>> thanks > >>>>>> rajesh > >>>>>> > >>>>> > >>>>> [snip] > >>>>> -- > >>>>> Christian Balzer Network/Systems Engineer > >>>>> chibi at gol.com Global OnLine Japan/Rakuten Communications > >>>>> http://www.gol.com/ > >>>> > >>> > >>> > >>> -- > >>> Christian Balzer Network/Systems Engineer > >>> chibi at gol.com > Global OnLine Japan/Rakuten Communications > >>> http://www.gol.com/ > > > > > > > -- > > Christian Balzer Network/Systems Engineer > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > http://www.gol.com/ > -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From Burak.Seydioglu at servicenow.com Mon Feb 13 22:44:57 2017 From: Burak.Seydioglu at servicenow.com (Burak Seydioglu) Date: Mon, 13 Feb 2017 22:44:57 +0000 Subject: pop3 and dsync master-master replication issue causing duplicate delivery (resolved) Message-ID: I had opened a thread a while ago in regards to dsync duplicate delivery issues. Here is an update in case anybody else is suffering from this: https://dovecot.org/list/dovecot/2016-April/103973.html We are using an LDAP backend for userdb and passdb lookups and this is causing multiple replication users to be created. Here is the broken setup: ############################################### userdb ############################################### hosts = localhost dn = cn=dovecot,dc... dnpass = PASSWORD ldap_version = 3 base = ou=instances,dc... deref = never scope = subtree user_attrs = \ =home=/mail/spool/hash-%0.2M{ldap:uid}/%{ldap:uid}, \ =mail=maildir:/mail/spool/hash-%0.2M{ldap:uid}/%{ldap:maildrop} user_filter = (&(objectClass=mailUser)(uid=%n)) iterate_attrs = uid=user iterate_filter = (objectClass=mailUser) ############################################### passdsb ############################################### hosts = localhost dn = cn=dovecot,dc=dc... dnpass = PASSWORD ldap_version = 3 base = ou=instances,dc=dc... deref = never scope = subtree pass_attrs = uid=user,userPassword=password, \ =userdb_home=/mail/spool/hash-%0.2M{ldap:uid}/%{ldap:uid}, \ =userdb_mail=maildir:/mail/spool/hash-%0.2M{ldap:uid}/%{ldap:maildrop} pass_filter = (&(objectClass=mailUser)(uid=%n)) ############################################### If you don't specify a user attribute in the userdb configuration, dovecot uses the destination email address to create the replication user in addition to users based on the definitions from iterate_attr and pass_attr entries. At the end, multiple replication users are created for the same mailbox causing deleted messages to re-appear and get delivered multiple times. ############################################### username priority fast sync full sync failed buraktest1 none 02:07:02 02:07:02 - buraktest1 at buraktest1.domain.com none 02:57:13 02:57:13 - ############################################### One thing to note here is that the behavior does not manifest itself during the first delivery/retrieval/delete cycle. You need to repeat the cycle again to expose the issue. The fix is to define and override the "user" attribute in the userdb configuration: ############################################### userdb ############################################### hosts = localhost dn = cn=dovecot,dc... dnpass = PASSWORD ldap_version = 3 base = ou=instances,dc... deref = never scope = subtree user_attrs = \ =user=%{ldap:uid}, \ =home=/mail/spool/hash-%0.2M{ldap:uid}/%{ldap:uid}, \ =mail=maildir:/mail/spool/hash-%0.2M{ldap:uid}/%{ldap:maildrop} user_filter = (&(objectClass=mailUser)(uid=%n)) iterate_attrs = uid=user iterate_filter = (objectClass=mailUser) ############################################### From chibi at gol.com Tue Feb 14 07:40:01 2017 From: chibi at gol.com (Christian Balzer) Date: Tue, 14 Feb 2017 16:40:01 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <224C0082B2F940C2861338B315BDA802.MAI@ns1.24x7server.net> References: <224C0082B2F940C2861338B315BDA802.MAI@ns1.24x7server.net> Message-ID: <20170214164001.361a41ee@batzmaru.gol.ad.jp> Hello, On Mon, 13 Feb 2017 22:53:23 +0530 Rajesh M wrote: > thanks for your help > > happy to say that the performance dramatically improved after i use the high performance settings from here > http://wiki.dovecot.org/LoginProcess > That's why that page is there. > grep Login: /var/log/mail.log.1 |wc -l > with the mail.log being of a typical, busy day. > 412992 > So about 5 login/s, not that busy, but that of course also depends on your HW and authentication backend. > i also picked up the imap and pop3 connections during peak hours > [root at ns1 domains]# doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' > username # proto (pids) (ips) > > max figures i got was 535 for imap and 97 for pop > > > Question 1 > i wish to improve the performance further by caching the logins. current the same is kept disable because when user's change passwords then they are not able to immediately login with the new password for some time. How to solve this issue. > There is no solution, caching auth data is by default fraught with that sort of peril. And how did you determine that caching auth data would actually improve things? What is your authentication backend? With 4000 users and 5 logins/s I can't imagine anything that wouldn't be able to: a) put all of the relevant data into RAM and b) thus handle auth requests in very timely manner. The example I gave earlier about my servers with 140 logins/s authenticates against LDAP and isn't very busy doing so while also fitting all 2.7GB of data for about 1 million users into RAM. > > > further to the above. > i am planning to put up a large mail server with one of the following options > Don't plan 1, always plan a pair and then use whatever you're comfortable with to turn them into a HA cluster, like Pacemaker with DRBD. > option 1 > OS and mailqueue : 2 X 600 gb 15k rpm raid 1 What kind of RAID, HW or MD? > data : 4 X 2000 gb raid 10 for data > backup : 2 X 2000 gb raid 10 for backup While with Linux MD a 2 disk RAID10 would be possible, it wouldn't be particular helpful (slower than a RAID 1 with writes). > 32 gb ram > > > option 2 > os and mail queue : 4 X 600 gb 15k rpm raid 10 Those 15k HDDs tend to be expensive. Estimate your expected writes per day (iostat on an existing server can give you some hints) and get 2 SSDs instead, DC S3520s if 1 DWPD is sufficient, DC S3610s if you need 3 DWPD (doubtful), S3710 if you need 10 DWPD (very doubtful). Similar for Samsung DC level SSDs, check their specs. For example, my busiest mailbox servers only write about 250KB/s averaged, which mean about 50GB/day and thus would be a fit even for a small S3520. I'm still using 200GB S3710s because I can afford them, like the speed and never ever want to worry about wearout. > data : 4 X 2000 gb raid 10 for data > 32 gb ram > > i am having the OS and mail queue on primary drives > > i understand that Raid10 give a faster write and read whereas Raid 1 will have slow writes and fast reads > While that is true, you're missing the real difference here, the number of HDDs. With 4 of them in the RAID10 they will be twice as fast as the 2 disk RAID1. This can be tweaked even further with the various RAID 10 layout options, "man md". > the question is will there be huge performance difference between Raid1 and Raid 10 when it comes small files like mail queue ? > Again as explained above, double the IOPS, so yes. Christian > > thanks > rajesh > ----- Original Message ----- > From: Christian Balzer [mailto:chibi at gol.com] > To: dovecot at dovecot.org > Cc: kevin at my.walr.us > Sent: Mon, 13 Feb 2017 10:46:13 +0900 > Subject: > > > Hello, > > On Sun, 12 Feb 2017 08:27:21 -0500 KT Walrus wrote: > > > Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. > > > > Those servers would actually be the wrong ones to look at, as they are > primarily accessed by the aforementioned broken client, so the numbers are > somewhat skewed. > However looking at other servers with a more "normal" user spread things > aren't actually too different. > > The average number of sessions per user tends to be 2. > The clear majority (over 50%) only has one session open (people with well > behaved and configured clients watching the INBOX mostly). > Another 30% has 2 sessions open, again the typical state of this would be > clients watching another mailbox besides INBOX, typically SENT. > The rest has 3 and more sessions open. > > The number of sessions could of course be drastically reduced if any > client would support IMAP NOTIFY, alas none that I'm aware of do. > > Lastly no more than 60% of the session seem to be in IDLE at any given > time, so my comments about RAM and IMAP hibernation effectiveness stand. > > > Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > > > > Yes to Dovecot Proxying, of course. > > No idea about MySQL, with (Open)LDAP nothing is breaking a sweat at an > average of 140 logins per second (IMAP and POP) on the 2 proxy servers. > If you can fit your entire dataset into RAM it should be fine, my LDAP > servers fall into that category and take about 10% of a slow (1.2GHz, 34%, > power-save mode) core only to handle the that load (also 2 servers). > And the rate of logins/s is what you need to worry about most and optimize > for. > > The proxies will of course have to do the shuffling of data and SSL > en/de-coding, but again they're not particular busy with that. > > The number of sessions comes into play when looking at the number of login > processes on the proxies and their memory footprint. > An IMAP login process on the proxies in performance mode with a client > limit of 1000 will consume about 55MB at most. > So assume at least 55KB RAM per session. > > Read Timo's mail I linked to about IMAP hibernation, AFAIK nothing has > happened to make proxies more supportive for this though. > > Christian > > > On Feb 12, 2017, at 1:58 AM, Christian Balzer wrote: > > > > > > > > > Hello, > > > > > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > > > > >>> 1. 256GB of real RAM, swap is for chums. > > >> > > >> Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > > >> > > > > > > I'm sure about tax and death, not much else. > > > > > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > > > even if it were to SSDs. > > > Incidentally the servers I'm talking about here have their OS and swap on > > > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > > > S3610s. > > > > > > Relying on the kernel to make swap decisions is likely to result in much > > > reduced performance even with fast SWAP when you're overcommitting things > > > on that scale. > > > > > > > > > But read on. > > > > > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > > >> > > >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. > > >> > > > See IMAP hibernation: > > > https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html > > > > > > I'm going to deploy/test this in production in about 2 months from now, > > > but if you look at the link and the consequent changelog entries you'll see > > > that it has certain shortcomings and bug fixes in pretty much each release > > > after it was introduced. > > > > > > But this is the correct way to tackle things, not SWAP. > > > > > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > > > here will be hibernated at any given time I'd be pleasantly surprised. > > > > > > Because between: > > > > > > 1. Finding a sensible imap_hibernate_timeout. > > > > > > 2. Having well behaved clients that keep idling instead of restarting the > > > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ ) > > > > > > 3. Having lots of mobile clients who either get disconnected (invisible to > > > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > > > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > > > forcing people to use 1 minute IDLE renewals, making 1. up there a > > > nightmare). > > > > > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > > > sessions, don't put them into IDLE and thus having them expire after 30 > > > minutes. > > > > > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > > > at least. > > > > > >> My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > > >> > > > > > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > > > but the scalability of things OTHER than Dovecot as I pointed out plus > > > that little detail of failure domains (do you really want half of your > > > eggs in one basket?) argue for scaling out after a certain density. > > > > > > I'm feeling my way there at this time, but expect more than 100k sessions > > > per server to be tricky. > > > > > > Lastly, when I asked about 500k sessions per server here not so long ago, > > > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) > > > Timo mentioned that he's not aware of anybody doing more than 50k per > > > server, something I got licked already and definitely will go to 100k > > > eventually. > > > > > > Regards, > > > > > > Christian > > >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > > >>> > > >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > > >>> > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > >>>> > > >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > > >>>> > > >>> That depends on the IMAP session, read the mailbox size and index size, > > >>> etc. > > >>> Some are significantly larger: > > >>> --- > > >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > > >>> --- > > >>> > > >>> But yes, as somebody who has mailbox servers with 55k+ session the average > > >>> is around 1.6MB. > > >>> > > >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > > >>>> > > >>> You will want: > > >> > > >>> 2. Understanding how to tune Dovecot and more importantly the overall > > >>> system to such a task (see that PID up there?). > > >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run > > >>> and others like atop actually killing dovecot (performance wise, not > > >>> literally) when doing their obviously flawed cleanup on exit. Some things > > >>> clearly do NOT scale well. > > >>> > > >>> My current goal is to have 100k capable servers that work well, 200k in a > > >>> failover scenario, but that won't be particular enjoyable. > > >>> > > >>> Christian > > >>> > > >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > > >>>>> > > >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > > >>>>> > > >>>>>> hello > > >>>>>> > > >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > > >>>>>> > > >>>>> > > >>>>> Be very precise here, you expect to see 1500 as the result of > > >>>>> "doveadm who |grep pop3 |wc -l"? > > >>>>> > > >>>>> Because that implies an ungodly number of POP3 connects per second, given > > >>>>> the typically short duration of these. > > >>>>> > > >>>>> 1500 IMAP connections (note that frequently a client will have more than > > >>>>> the INBOX open and thus have more than one session and thus process on the > > >>>>> server) are a much easier proposition, provided they are of the typical > > >>>>> long lasting type. > > >>>>> > > >>>>> So can you put a number to your expected logins per second (both protocols)? > > >>>>> > > >>>>>> my server > > >>>>>> > > >>>>>> server configuration > > >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > > >>>>>> gb hdd for data (No raid) > > >>>>>> > > >>>>> No RAID and no other replication like DRBD? > > >>>>> Why would you even bother? > > >>>>> > > >>>>> How many users/mailboxes in total with what quota? > > >>>>> > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > > >>>>> dentries) in RAM. > > >>>>> > > >>>>> If you really have several hundreds logins/s, you're facing several > > >>>>> bottlenecks: > > >>>>> 1. Login processes themselves (easily fixed by high performance mode) > > >>>>> 2. Auth processes (that will depend on your backends, method mostly) > > >>>>> 3. Dovecot master process (spawning mail processes) > > >>>>> > > >>>>> The later is a single-threaded process, so it will benefit from a faster > > >>>>> CPU core. > > >>>>> It can be dramatically improved by enabling process re-usage, see: > > >>>>> http://wiki.dovecot.org/PerformanceTuning > > >>>>> > > >>>>> However that also means more memory usage. > > >>>>> > > >>>>> > > >>>>> > > >>>>> Christian > > >>>>> > > >>>>>> > > >>>>>> thanks > > >>>>>> rajesh > > >>>>>> > > >>>>> > > >>>>> [snip] > > >>>>> -- > > >>>>> Christian Balzer Network/Systems Engineer > > >>>>> chibi at gol.com Global OnLine Japan/Rakuten Communications > > >>>>> http://www.gol.com/ > > >>>> > > >>> > > >>> > > >>> -- > > >>> Christian Balzer Network/Systems Engineer > > >>> chibi at gol.com > Global OnLine Japan/Rakuten Communications > > >>> http://www.gol.com/ > > > > > > > > > > -- > > > Christian Balzer Network/Systems Engineer > > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > > http://www.gol.com/ > > > > -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From dovecot-user at tributh.net Tue Feb 14 11:01:50 2017 From: dovecot-user at tributh.net (Tributh) Date: Tue, 14 Feb 2017 12:01:50 +0100 Subject: openssl 1.1.0d breaks Android7 TLS connects Message-ID: <1cdarn-0004NN-Qz@tributh.net> Hi, the actual OpenSSL version detection in dovecot is insufficient. The implementation only checks for SSL_CTRL_SET_ECDH_AUTO. That was effective for OpenSSL 1.0.2, but in 1.1.0 it is removed. Thats the code part: #ifdef SSL_CTRL_SET_ECDH_AUTO /* OpenSSL >= 1.0.2 automatically handles ECDH temporary key parameter selection. */ SSL_CTX_set_ecdh_auto(ssl_ctx, 1); #else /* For OpenSSL < 1.0.2, ECDH temporary key parameter selection must be performed manually. Attempt to select the same curve as that used in the server's private EC key file. Otherwise fall back to the NIST P-384 (secp384r1) curve to be compliant with RFC 6460 when AES-256 TLS cipher suites are in use. This fall back option does however make Dovecot non-compliant with RFC 6460 which requires curve NIST P-256 (prime256v1) be used when AES-128 TLS cipher suites are in use. At least the non-compliance is in the form of providing too much security rather than too little. */ nid = ssl_proxy_ctx_get_pkey_ec_curve_name(set); ecdh = EC_KEY_new_by_curve_name(nid); if (ecdh == NULL) { /* Fall back option */ nid = NID_secp384r1; ecdh = EC_KEY_new_by_curve_name(nid); } if ((curve_name = OBJ_nid2sn(nid)) != NULL && set->verbose_ssl) i_debug("SSL: elliptic curve %s will be used for ECDH and" " ECDHE key exchanges", curve_name); if (ecdh != NULL) { SSL_CTX_set_tmp_ecdh(ssl_ctx, ecdh); EC_KEY_free(ecdh); } #endif The OpenSSL CHANGES file says for version 1.1.0: Changes between 1.0.2h and 1.1.0 [25 Aug 2016] ... ... *) SSL_{CTX_}set_ecdh_auto() has been removed and ECDH is support is always enabled now. If you want to disable the support you should exclude it using the list of supported ciphers. This also means that the "-no_ecdhe" option has been removed from s_server. [Kurt Roeckx] So when the check for OpenSSL 1.1.0 fails, the curve selection will be forced to use secp384r1 like it would be on older versions. This curve change during negotiation breaks the connect for Android7 devices. They are not able to negotiate any ECDHE cipher. The dovecot log shows: ...SSL_accept() failed: error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher... but here it is not a cipher problem. Instead it is a curve problem. This is most relevant if the server is suited with an ECDSA-Certificate. Than no TLS negotiation is possible. There should be added a more sufficient check for the OpenSSL version. If using OpenSSL 1.1.0*, the easiest way the have auto curve selection again is to remove the whole check for testing purpose which results in a working auto curve offering installation and allows also Android7 devices again to connect with TLS & ECDHE ciphers. Regards Torsten From sven at cs-ware.de Tue Feb 14 11:44:44 2017 From: sven at cs-ware.de (Sven Strickroth) Date: Tue, 14 Feb 2017 12:44:44 +0100 Subject: Mailbox suddenly inaccessible Message-ID: Hi, out of the sudden I've an inaccessible folder with Dovecot 2.2.27 (c0f36b0) from Debian Jessy backports. $ doveadm mailbox list -u mail at example.com INBOX/Erledigt INBOX/Erledigt/Erledigt INBOX/Versicherung gesendet Sent INBOX However, it is not accessible any more, my IMAP client tells me: "STATUS: Mailbox doesn't exist: INBOX/Erledigt" Also: $ doveadm search -u mail at example.com mailbox INBOX/Erledigt doveadm(mail at example.com): Error: Couldn't get mailbox 'INBOX/Erledigt' GUID: Mailbox doesn't exist: INBOX/Erledigt Other mailboxes (especially INBOX/Erledigt/Erledigt) are accessible: $ doveadm search -u mail at example.com mailbox INBOX/Erledigt/Erledigt 60af0f1964d68d574a480000f94cf17a 346 60af0f1964d68d574a480000f94cf17a 367 ... $ doveadm -f table mailbox status -u mail at example.com "messages vsize" "INBOX*" mailbox messages vsize INBOX/Erledigt2 15 25868676 INBOX/Erledigt/Erledigt 4344 6107238270 INBOX 24 36081472 I'm using mdbox and see no errors in syslog from dovecot. Any ideas on what's going on and how I can recover the mails in that folder? -- Best regards, Sven Strickroth PGP key id F5A9D4C4 @ any key-server From aki.tuomi at dovecot.fi Tue Feb 14 11:44:58 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 14 Feb 2017 13:44:58 +0200 Subject: Mailbox suddenly inaccessible In-Reply-To: References: Message-ID: <659972c0-1005-85f5-dda2-5cd47bc76174@dovecot.fi> On 14.02.2017 13:44, Sven Strickroth wrote: > Hi, > > out of the sudden I've an inaccessible folder with Dovecot 2.2.27 > (c0f36b0) from Debian Jessy backports. > > $ doveadm mailbox list -u mail at example.com > INBOX/Erledigt > INBOX/Erledigt/Erledigt > INBOX/Versicherung gesendet > Sent > INBOX > > However, it is not accessible any more, my IMAP client tells me: > "STATUS: Mailbox doesn't exist: INBOX/Erledigt" > > Also: > $ doveadm search -u mail at example.com mailbox INBOX/Erledigt > doveadm(mail at example.com): Error: Couldn't get mailbox 'INBOX/Erledigt' > GUID: Mailbox doesn't exist: INBOX/Erledigt > > Other mailboxes (especially INBOX/Erledigt/Erledigt) are accessible: > $ doveadm search -u mail at example.com mailbox INBOX/Erledigt/Erledigt > 60af0f1964d68d574a480000f94cf17a 346 > 60af0f1964d68d574a480000f94cf17a 367 > ... > > $ doveadm -f table mailbox status -u mail at example.com "messages vsize" > "INBOX*" > mailbox messages vsize > INBOX/Erledigt2 15 25868676 > INBOX/Erledigt/Erledigt 4344 6107238270 > INBOX 24 36081472 > > I'm using mdbox and see no errors in syslog from dovecot. > > Any ideas on what's going on and how I can recover the mails in that folder? > Hi! Is the Erledigt2 folder somehow related to this? Aki From sven at cs-ware.de Tue Feb 14 12:42:56 2017 From: sven at cs-ware.de (Sven Strickroth) Date: Tue, 14 Feb 2017 13:42:56 +0100 Subject: Mailbox suddenly inaccessible In-Reply-To: <659972c0-1005-85f5-dda2-5cd47bc76174@dovecot.fi> References: <659972c0-1005-85f5-dda2-5cd47bc76174@dovecot.fi> Message-ID: <82e4c29d-4422-2bef-8d7e-c98c47dc0855@cs-ware.de> Am 14.02.2017 um 12:44 schrieb Aki Tuomi: > Is the Erledigt2 folder somehow related to this? Oh, the user just created Erledigt2 as a workaroud as long as Erledigt is not accessible while I executed the command for my previous mail. What concerns me is that on the filesystem Erledigt2, Erledigt/Erledigt and all other mailboxes have a subfolder named "dbox-Mails", Erledigt does not have this subfolder. How can this happen? Btw. It seems as if the INBOX/Erledigt/Erledigt folder contains all the mails from the now inaccessible INBOX/Erledigt folder. -- Best regards, Sven Strickroth PGP key id F5A9D4C4 @ any key-server From stephan at rename-it.nl Tue Feb 14 14:44:13 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 14 Feb 2017 15:44:13 +0100 Subject: sieve file backend: invalid option `active=~/.dovecot.sieve' In-Reply-To: <0ed0da6f-2a6a-d2ce-6458-3e164d608031@list-subs.com> References: <0ed0da6f-2a6a-d2ce-6458-3e164d608031@list-subs.com> Message-ID: <16ddcba3-d7f9-361a-b1b0-7effc67ad984@rename-it.nl> Op 13-2-2017 om 18:15 schreef Ben: > Hi, > > I am seeing the followin error in my logs (doveconf -n at the bottom > of this mail): > > Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: > cs3NOQ7moVj4sQXXXXX: sieve: sieve file backend: invalid option > `active=~/.dovecot.sieve' > Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: > cs3NOQ7moVj4sQXXXXX: sieve: failed to access user's Sieve script > file:~/sieve;active=~/.dovecot.sieve (temporary failure) > > > Looking at http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration, > the syntax "sieve = file:~/sieve;active=~/.dovecot.sieve" in my config > is correct ? > > Is this a false error that is only appearing because this is a newly > created user with no sieve file ? If this is the case, how do I tell > dovecot not to error out and allow the mail ? Don't specify the "sieve_dir" setting when you're using the new location syntax for the "sieve" setting. That setting is deprecated and causes the "sieve" setting to be interpreted differently for backwards compatibility. > > Thanks ! > > > > # 2.2.10: /etc/dovecot/dovecot.conf > # OS: Linux 3.10.0-514.6.1.el7.x86_64 x86_64 CentOS Linux release > 7.3.1611 (Core) > auth_mechanisms = plain login > auth_verbose = yes > auth_verbose_passwords = sha1 > first_valid_uid = 1000 > mail_location = maildir:~/Maildir > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body environment > mailbox date ihave enotify > mbox_write_locks = fcntl > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > driver = pam > } > passdb { > driver = pam > } > passdb { > args = scheme=CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file > } > plugin { > sieve = file:~/sieve;active=~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap lmtp > service auth { > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = its_virtmail > mode = 0660 > user = its_virtmail > } > } > service imap-login { > process_min_avail = 3 > } > service lmtp { > process_min_avail = 5 > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > user = its_virtmail > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > inet_listener sieves { > address = > port = 5190 > ssl = yes > } > } > ssl = required > ssl_cert = ssl_cipher_list = REMOVED > ssl_dh_parameters_length = 2048 > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSlv2 !SSLv3 > userdb { > driver = passwd > } > userdb { > args = username_format=%u /etc/dovecot/users > driver = passwd-file > } > protocol lmtp { > mail_plugins = sieve > postmaster_address = me at example.com > } > protocol lda { > deliver_log_format = msgid=%m: %$ > mail_plugins = sieve > postmaster_address = postmaster > quota_full_tempfail = yes > rejection_reason = Your message to <%t> was automatically rejected:%n%r > } > protocol imap { > imap_client_workarounds = delay-newmail > mail_max_userip_connections = 20 > } From luciano at vespaperitivo.it Tue Feb 14 14:54:47 2017 From: luciano at vespaperitivo.it (Luciano Mannucci) Date: Tue, 14 Feb 2017 15:54:47 +0100 Subject: pop3 login core dump In-Reply-To: <3vMVHj4FrnzRRrq@baobab.bilink.it> References: <3vMVHj4FrnzRRrq@baobab.bilink.it> Message-ID: <3vN56w0jtGzRRqQ@baobab.bilink.it> On Mon, 13 Feb 2017 16:45:21 +0100 Luciano Mannucci wrote: > where am I supposed to add the -D option in order to get the core file > dumped so I can examine it? I think I've got it. I've set my config to: service pop3-login { executable = pop3-login -D inet_listener pop3s { address = * } process_limit = 512 } Now I have to wait till next SIGSEV... Luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster at sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.lesassaie.IT/ From ben+dovecot at list-subs.com Tue Feb 14 17:46:48 2017 From: ben+dovecot at list-subs.com (Ben) Date: Tue, 14 Feb 2017 17:46:48 +0000 Subject: sieve file backend: invalid option `active=~/.dovecot.sieve' In-Reply-To: <16ddcba3-d7f9-361a-b1b0-7effc67ad984@rename-it.nl> References: <0ed0da6f-2a6a-d2ce-6458-3e164d608031@list-subs.com> <16ddcba3-d7f9-361a-b1b0-7effc67ad984@rename-it.nl> Message-ID: On 14/02/2017 14:44, Stephan Bosch wrote: > Don't specify the "sieve_dir" setting when you're using the new location > syntax for the "sieve" setting. That setting is deprecated and causes > the "sieve" setting to be interpreted differently for backwards > compatibility. > Thanks for the pointer, although I don't think I am explicitly setting it (any changes I'm making to the default install I'm puting in local.conf) Guess I'll just have to track down which default file sieve_dir is being set in and comment it out... From ben+dovecot at list-subs.com Tue Feb 14 19:45:49 2017 From: ben+dovecot at list-subs.com (Ben) Date: Tue, 14 Feb 2017 19:45:49 +0000 Subject: sieve file backend: invalid option `active=~/.dovecot.sieve' In-Reply-To: <16ddcba3-d7f9-361a-b1b0-7effc67ad984@rename-it.nl> References: <0ed0da6f-2a6a-d2ce-6458-3e164d608031@list-subs.com> <16ddcba3-d7f9-361a-b1b0-7effc67ad984@rename-it.nl> Message-ID: On 14/02/2017 14:44, Stephan Bosch wrote: > > > Op 13-2-2017 om 18:15 schreef Ben: >> Hi, >> >> I am seeing the followin error in my logs (doveconf -n at the bottom >> of this mail): >> >> Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: >> cs3NOQ7moVj4sQXXXXX: sieve: sieve file backend: invalid option >> `active=~/.dovecot.sieve' >> Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: >> cs3NOQ7moVj4sQXXXXX: sieve: failed to access user's Sieve script >> file:~/sieve;active=~/.dovecot.sieve (temporary failure) >> >> >> Looking at http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration, >> the syntax "sieve = file:~/sieve;active=~/.dovecot.sieve" in my config >> is correct ? >> >> Is this a false error that is only appearing because this is a newly >> created user with no sieve file ? If this is the case, how do I tell >> dovecot not to error out and allow the mail ? > > Don't specify the "sieve_dir" setting when you're using the new location > syntax for the "sieve" setting. That setting is deprecated and causes > the "sieve" setting to be interpreted differently for backwards > compatibility. > > I've now commented out sieve_dir and its no longer appearing in doveconf -n, however the problem originally described still exists. Any other ideas ? From jtam.home at gmail.com Tue Feb 14 22:50:21 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 14 Feb 2017 14:50:21 -0800 (PST) Subject: dovecot config for 1500 simultaneous connection In-Reply-To: References: Message-ID: Rajesh M wrote: > i wish to improve the performance further by caching the logins. > current the same is kept disable because when user's change passwords > then they are not able to immediately login with the new password for > some time. How to solve this issue. Dovecot shouldn't be doing that. According to https://wiki2.dovecot.org/Authentication/Caching Data is used from the cache if it's not expired (auth_cache_ttl setting) - If authentication fails this time, but it didn't fail last time, it's assumed that the password has changed and a database lookup is done. As I read it, an authentication failure will cause an update of cache credentials. The problem I encountered is the converse -- a user changes their password, but forgets to update their mail reader's or web browser's stored password, which continues to work until the TTL expires. Another related security situation I've encountered is when a fraudster has phished a user's password. A user/admin changes the password, but forgets to invalidate dovecot's cached entry, allowing the fraudster contunuing access to the mail account until the TTL expires or user logs in with new credentials. I've been burnt by this one. Joseph Tam From kevin at my.walr.us Tue Feb 14 23:02:22 2017 From: kevin at my.walr.us (KT Walrus) Date: Tue, 14 Feb 2017 18:02:22 -0500 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: References: Message-ID: <376701DA-2D8C-483F-B463-FE21E58062B5@my.walr.us> > On Feb 14, 2017, at 5:50 PM, Joseph Tam wrote: > > Another related security situation I've encountered is when a fraudster > has phished a user's password. A user/admin changes the password, > but forgets to invalidate dovecot's cached entry, allowing the fraudster > contunuing access to the mail account until the TTL expires or user logs > in with new credentials. I've been burnt by this one. I?m no expert, but should the code that updates the password hash in the database also immediately try to log into dovecot for the user with a fake password? Authentication should fail but the cache would be updated? Or, doesn?t Dovecot expire the cache?d entry on failed authentication? From jtam.home at gmail.com Tue Feb 14 23:12:15 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 14 Feb 2017 15:12:15 -0800 (PST) Subject: dovecot config for 1500 simultaneous connection In-Reply-To: <376701DA-2D8C-483F-B463-FE21E58062B5@my.walr.us> References: <376701DA-2D8C-483F-B463-FE21E58062B5@my.walr.us> Message-ID: On Tue, 14 Feb 2017, KT Walrus wrote: >> Another related security situation I've encountered is when a fraudster >> has phished a user's password. A user/admin changes the password, >> but forgets to invalidate dovecot's cached entry, allowing the fraudster >> contunuing access to the mail account until the TTL expires or user logs >> in with new credentials. I've been burnt by this one. > > I?m no expert, but should the code that updates the password hash in the > database also immediately try to log into dovecot for the user with a fake > password? > > Authentication should fail but the cache would be updated? I guess you could write a hook for that, but that's not a scalable solution if your auth database is used by many different services that do their own credential caching. I do this manually i.e. if I have to invalidate mail system cache, I log in with wrong credentials. > Or, doesn?t Dovecot expire the cache?d entry on failed authentication? Dovecot does have a auth_cache_negative_ttl setting. I think the best proactive approach is to keep auth_cache_ttl modest (a few minutes) rather than hours or days. Joseph Tam From 24x7server at 24x7server.net Wed Feb 15 03:13:23 2017 From: 24x7server at 24x7server.net (Rajesh M) Date: Wed, 15 Feb 2017 08:43:23 +0530 Subject: dovecot config for 1500 simultaneous connection Message-ID: christian the servers i currently own are dell servers. The servers i plan to buy are Dell R530, 2U rack servers with 8 x 3.5 inch drives, with 64 gb ram each, Hardware raid. I am thinking of 2 X 300 gb ssds raid1 and 6 x 2 tb drives in raid10 for data. I do not have any experience in setting up drdb (that would be my next step) ... primarily using standalone servers with hardware level redundancy. For backup-ups i rsync data to backup servers, also use qmail taps for backing up sent/recd emails on a per user basis. i am using mysql authentication backend my mail server is qmailtoaster, ie with vpopmail, spamdyke, simscan. you had mentioned as follows ############## With 4000 users and 5 logins/s I can't imagine anything that wouldn't be able to: a) put all of the relevant data into RAM and b) thus handle auth requests in very timely manner. The example I gave earlier about my servers with 140 logins/s authenticates against LDAP and isn't very busy doing so while also fitting all 2.7GB of data for about 1 million users into RAM. ############## >>>>>>>>>>>>>>>>>>how do i put all relevant data into the RAM ? thanks rajesh ----- Original Message ----- From: Christian Balzer [mailto:chibi at gol.com] To: dovecot at dovecot.org Sent: Tue, 14 Feb 2017 16:40:01 +0900 Subject: Hello, On Mon, 13 Feb 2017 22:53:23 +0530 Rajesh M wrote: > thanks for your help > > happy to say that the performance dramatically improved after i use the high performance settings from here > http://wiki.dovecot.org/LoginProcess > That's why that page is there. > grep Login: /var/log/mail.log.1 |wc -l > with the mail.log being of a typical, busy day. > 412992 > So about 5 login/s, not that busy, but that of course also depends on your HW and authentication backend. > i also picked up the imap and pop3 connections during peak hours > [root at ns1 domains]# doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' > username # proto (pids) (ips) > > max figures i got was 535 for imap and 97 for pop > > > Question 1 > i wish to improve the performance further by caching the logins. current the same is kept disable because when user's change passwords then they are not able to immediately login with the new password for some time. How to solve this issue. > There is no solution, caching auth data is by default fraught with that sort of peril. And how did you determine that caching auth data would actually improve things? What is your authentication backend? With 4000 users and 5 logins/s I can't imagine anything that wouldn't be able to: a) put all of the relevant data into RAM and b) thus handle auth requests in very timely manner. The example I gave earlier about my servers with 140 logins/s authenticates against LDAP and isn't very busy doing so while also fitting all 2.7GB of data for about 1 million users into RAM. > > > further to the above. > i am planning to put up a large mail server with one of the following options > Don't plan 1, always plan a pair and then use whatever you're comfortable with to turn them into a HA cluster, like Pacemaker with DRBD. > option 1 > OS and mailqueue : 2 X 600 gb 15k rpm raid 1 What kind of RAID, HW or MD? > data : 4 X 2000 gb raid 10 for data > backup : 2 X 2000 gb raid 10 for backup While with Linux MD a 2 disk RAID10 would be possible, it wouldn't be particular helpful (slower than a RAID 1 with writes). > 32 gb ram > > > option 2 > os and mail queue : 4 X 600 gb 15k rpm raid 10 Those 15k HDDs tend to be expensive. Estimate your expected writes per day (iostat on an existing server can give you some hints) and get 2 SSDs instead, DC S3520s if 1 DWPD is sufficient, DC S3610s if you need 3 DWPD (doubtful), S3710 if you need 10 DWPD (very doubtful). Similar for Samsung DC level SSDs, check their specs. For example, my busiest mailbox servers only write about 250KB/s averaged, which mean about 50GB/day and thus would be a fit even for a small S3520. I'm still using 200GB S3710s because I can afford them, like the speed and never ever want to worry about wearout. > data : 4 X 2000 gb raid 10 for data > 32 gb ram > > i am having the OS and mail queue on primary drives > > i understand that Raid10 give a faster write and read whereas Raid 1 will have slow writes and fast reads > While that is true, you're missing the real difference here, the number of HDDs. With 4 of them in the RAID10 they will be twice as fast as the 2 disk RAID1. This can be tweaked even further with the various RAID 10 layout options, "man md". > the question is will there be huge performance difference between Raid1 and Raid 10 when it comes small files like mail queue ? > Again as explained above, double the IOPS, so yes. Christian > > thanks > rajesh > ----- Original Message ----- > From: Christian Balzer [mailto:chibi at gol.com] > To: dovecot at dovecot.org > Cc: kevin at my.walr.us > Sent: Mon, 13 Feb 2017 10:46:13 +0900 > Subject: > > > Hello, > > On Sun, 12 Feb 2017 08:27:21 -0500 KT Walrus wrote: > > > Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. > > > > Those servers would actually be the wrong ones to look at, as they are > primarily accessed by the aforementioned broken client, so the numbers are > somewhat skewed. > However looking at other servers with a more "normal" user spread things > aren't actually too different. > > The average number of sessions per user tends to be 2. > The clear majority (over 50%) only has one session open (people with well > behaved and configured clients watching the INBOX mostly). > Another 30% has 2 sessions open, again the typical state of this would be > clients watching another mailbox besides INBOX, typically SENT. > The rest has 3 and more sessions open. > > The number of sessions could of course be drastically reduced if any > client would support IMAP NOTIFY, alas none that I'm aware of do. > > Lastly no more than 60% of the session seem to be in IDLE at any given > time, so my comments about RAM and IMAP hibernation effectiveness stand. > > > Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > > > > Yes to Dovecot Proxying, of course. > > No idea about MySQL, with (Open)LDAP nothing is breaking a sweat at an > average of 140 logins per second (IMAP and POP) on the 2 proxy servers. > If you can fit your entire dataset into RAM it should be fine, my LDAP > servers fall into that category and take about 10% of a slow (1.2GHz, 34%, > power-save mode) core only to handle the that load (also 2 servers). > And the rate of logins/s is what you need to worry about most and optimize > for. > > The proxies will of course have to do the shuffling of data and SSL > en/de-coding, but again they're not particular busy with that. > > The number of sessions comes into play when looking at the number of login > processes on the proxies and their memory footprint. > An IMAP login process on the proxies in performance mode with a client > limit of 1000 will consume about 55MB at most. > So assume at least 55KB RAM per session. > > Read Timo's mail I linked to about IMAP hibernation, AFAIK nothing has > happened to make proxies more supportive for this though. > > Christian > > > On Feb 12, 2017, at 1:58 AM, Christian Balzer wrote: > > > > > > > > > Hello, > > > > > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > > > > >>> 1. 256GB of real RAM, swap is for chums. > > >> > > >> Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > > >> > > > > > > I'm sure about tax and death, not much else. > > > > > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > > > even if it were to SSDs. > > > Incidentally the servers I'm talking about here have their OS and swap on > > > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > > > S3610s. > > > > > > Relying on the kernel to make swap decisions is likely to result in much > > > reduced performance even with fast SWAP when you're overcommitting things > > > on that scale. > > > > > > > > > But read on. > > > > > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > > >> > > >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. > > >> > > > See IMAP hibernation: > > > https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html > > > > > > I'm going to deploy/test this in production in about 2 months from now, > > > but if you look at the link and the consequent changelog entries you'll see > > > that it has certain shortcomings and bug fixes in pretty much each release > > > after it was introduced. > > > > > > But this is the correct way to tackle things, not SWAP. > > > > > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > > > here will be hibernated at any given time I'd be pleasantly surprised. > > > > > > Because between: > > > > > > 1. Finding a sensible imap_hibernate_timeout. > > > > > > 2. Having well behaved clients that keep idling instead of restarting the > > > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ ) > > > > > > 3. Having lots of mobile clients who either get disconnected (invisible to > > > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > > > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > > > forcing people to use 1 minute IDLE renewals, making 1. up there a > > > nightmare). > > > > > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > > > sessions, don't put them into IDLE and thus having them expire after 30 > > > minutes. > > > > > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > > > at least. > > > > > >> My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > > >> > > > > > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > > > but the scalability of things OTHER than Dovecot as I pointed out plus > > > that little detail of failure domains (do you really want half of your > > > eggs in one basket?) argue for scaling out after a certain density. > > > > > > I'm feeling my way there at this time, but expect more than 100k sessions > > > per server to be tricky. > > > > > > Lastly, when I asked about 500k sessions per server here not so long ago, > > > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) > > > Timo mentioned that he's not aware of anybody doing more than 50k per > > > server, something I got licked already and definitely will go to 100k > > > eventually. > > > > > > Regards, > > > > > > Christian > > >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > > >>> > > >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > > >>> > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > >>>> > > >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > > >>>> > > >>> That depends on the IMAP session, read the mailbox size and index size, > > >>> etc. > > >>> Some are significantly larger: > > >>> --- > > >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > > >>> --- > > >>> > > >>> But yes, as somebody who has mailbox servers with 55k+ session the average > > >>> is around 1.6MB. > > >>> > > >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > > >>>> > > >>> You will want: > > >> > > >>> 2. Understanding how to tune Dovecot and more importantly the overall > > >>> system to such a task (see that PID up there?). > > >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run > > >>> and others like atop actually killing dovecot (performance wise, not > > >>> literally) when doing their obviously flawed cleanup on exit. Some things > > >>> clearly do NOT scale well. > > >>> > > >>> My current goal is to have 100k capable servers that work well, 200k in a > > >>> failover scenario, but that won't be particular enjoyable. > > >>> > > >>> Christian > > >>> > > >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > > >>>>> > > >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > > >>>>> > > >>>>>> hello > > >>>>>> > > >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > > >>>>>> > > >>>>> > > >>>>> Be very precise here, you expect to see 1500 as the result of > > >>>>> "doveadm who |grep pop3 |wc -l"? > > >>>>> > > >>>>> Because that implies an ungodly number of POP3 connects per second, given > > >>>>> the typically short duration of these. > > >>>>> > > >>>>> 1500 IMAP connections (note that frequently a client will have more than > > >>>>> the INBOX open and thus have more than one session and thus process on the > > >>>>> server) are a much easier proposition, provided they are of the typical > > >>>>> long lasting type. > > >>>>> > > >>>>> So can you put a number to your expected logins per second (both protocols)? > > >>>>> > > >>>>>> my server > > >>>>>> > > >>>>>> server configuration > > >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > > >>>>>> gb hdd for data (No raid) > > >>>>>> > > >>>>> No RAID and no other replication like DRBD? > > >>>>> Why would you even bother? > > >>>>> > > >>>>> How many users/mailboxes in total with what quota? > > >>>>> > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > > >>>>> dentries) in RAM. > > >>>>> > > >>>>> If you really have several hundreds logins/s, you're facing several > > >>>>> bottlenecks: > > >>>>> 1. Login processes themselves (easily fixed by high performance mode) > > >>>>> 2. Auth processes (that will depend on your backends, method mostly) > > >>>>> 3. Dovecot master process (spawning mail processes) > > >>>>> > > >>>>> The later is a single-threaded process, so it will benefit from a faster > > >>>>> CPU core. > > >>>>> It can be dramatically improved by enabling process re-usage, see: > > >>>>> http://wiki.dovecot.org/PerformanceTuning > > >>>>> > > >>>>> However that also means more memory usage. > > >>>>> > > >>>>> > > >>>>> > > >>>>> Christian > > >>>>> > > >>>>>> > > >>>>>> thanks > > >>>>>> rajesh > > >>>>>> > > >>>>> > > >>>>> [snip] > > >>>>> -- > > >>>>> Christian Balzer Network/Systems Engineer > > >>>>> chibi at gol.com Global OnLine Japan/Rakuten Communications > > >>>>> http://www.gol.com/ > > >>>> > > >>> > > >>> > > >>> -- > > >>> Christian Balzer Network/Systems Engineer > > >>> chibi at gol.com > Global OnLine Japan/Rakuten Communications > > >>> http://www.gol.com/ > > > > > > > > > > -- > > > Christian Balzer Network/Systems Engineer > > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > > http://www.gol.com/ > > > > -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From chibi at gol.com Wed Feb 15 05:41:24 2017 From: chibi at gol.com (Christian Balzer) Date: Wed, 15 Feb 2017 14:41:24 +0900 Subject: dovecot config for 1500 simultaneous connection In-Reply-To: References: Message-ID: <20170215144124.24e139ef@batzmaru.gol.ad.jp> On Wed, 15 Feb 2017 08:43:23 +0530 Rajesh M wrote: > christian > > the servers i currently own are dell servers. The servers i plan to buy are Dell R530, 2U rack servers with 8 x 3.5 inch drives, with 64 gb ram each, Hardware raid. I am thinking of 2 X 300 gb ssds raid1 and 6 x 2 tb drives in raid10 for data. I do not have any experience in setting up drdb (that would be my next step) ... primarily using standalone servers with hardware level redundancy. For backup-ups i rsync data to backup servers, also use qmail taps for backing up sent/recd emails on a per user basis. > > i am using mysql authentication backend > my mail server is qmailtoaster, ie with vpopmail, spamdyke, simscan. > > you had mentioned as follows > ############## > With 4000 users and 5 logins/s I can't imagine anything that wouldn't be > able to: > a) put all of the relevant data into RAM and > b) thus handle auth requests in very timely manner. > The example I gave earlier about my servers with 140 logins/s > authenticates against LDAP and isn't very busy doing so while also > fitting all 2.7GB of data for about 1 million users into RAM. > ############## > > >>>>>>>>>>>>>>>>>>how do i put all relevant data into the RAM ? > By configuring MySQL accordingly of course, for example: --- my.cnf: # Set buffer pool size to 50-80% of your computer's memory, # but make sure on Linux x86 total memory usage is < 2GB innodb_buffer_pool_size=8G innodb_additional_mem_pool_size=32M --- --- top: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 39045 mysql 20 0 9170m 784m 10m S 0 2.4 3:38.69 mysqld ---- --- tuning-primer: INNODB STATUS Current InnoDB index space = 49 M Current InnoDB data space = 109 M Current InnoDB buffer pool free = 98 % Current innodb_buffer_pool_size = 8.00 G Depending on how much space your innodb indexes take up it may be safe to increase this value to up to 2 / 3 of total system memory --- Christian > thanks > rajesh > > > ----- Original Message ----- > From: Christian Balzer [mailto:chibi at gol.com] > To: dovecot at dovecot.org > Sent: Tue, 14 Feb 2017 16:40:01 +0900 > Subject: > > > Hello, > > On Mon, 13 Feb 2017 22:53:23 +0530 Rajesh M wrote: > > > thanks for your help > > > > happy to say that the performance dramatically improved after i use the high performance settings from here > > http://wiki.dovecot.org/LoginProcess > > > That's why that page is there. > > > grep Login: /var/log/mail.log.1 |wc -l > > with the mail.log being of a typical, busy day. > > 412992 > > > So about 5 login/s, not that busy, but that of course also depends on your > HW and authentication backend. > > > i also picked up the imap and pop3 connections during peak hours > > [root at ns1 domains]# doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' > > username # proto (pids) (ips) > > > > max figures i got was 535 for imap and 97 for pop > > > > > > Question 1 > > i wish to improve the performance further by caching the logins. current the same is kept disable because when user's change passwords then they are not able to immediately login with the new password for some time. How to solve this issue. > > > There is no solution, caching auth data is by default fraught with that > sort of peril. > > And how did you determine that caching auth data would actually improve > things? > > What is your authentication backend? > > With 4000 users and 5 logins/s I can't imagine anything that wouldn't be > able to: > a) put all of the relevant data into RAM and > b) thus handle auth requests in very timely manner. > > The example I gave earlier about my servers with 140 logins/s > authenticates against LDAP and isn't very busy doing so while also > fitting all 2.7GB of data for about 1 million users into RAM. > > > > > > > further to the above. > > i am planning to put up a large mail server with one of the following options > > > Don't plan 1, always plan a pair and then use whatever you're comfortable > with to turn them into a HA cluster, like Pacemaker with DRBD. > > > option 1 > > OS and mailqueue : 2 X 600 gb 15k rpm raid 1 > What kind of RAID, HW or MD? > > > data : 4 X 2000 gb raid 10 for data > > backup : 2 X 2000 gb raid 10 for backup > > While with Linux MD a 2 disk RAID10 would be possible, it wouldn't be > particular helpful (slower than a RAID 1 with writes). > > > 32 gb ram > > > > > > option 2 > > os and mail queue : 4 X 600 gb 15k rpm raid 10 > Those 15k HDDs tend to be expensive. > Estimate your expected writes per day (iostat on an existing server can > give you some hints) and get 2 SSDs instead, DC S3520s if 1 DWPD is > sufficient, DC S3610s if you need 3 DWPD (doubtful), S3710 if you need 10 > DWPD (very doubtful). > Similar for Samsung DC level SSDs, check their specs. > > For example, my busiest mailbox servers only write about 250KB/s averaged, > which mean about 50GB/day and thus would be a fit even for a small S3520. > I'm still using 200GB S3710s because I can afford them, like the speed and > never ever want to worry about wearout. > > > data : 4 X 2000 gb raid 10 for data > > 32 gb ram > > > > i am having the OS and mail queue on primary drives > > > > i understand that Raid10 give a faster write and read whereas Raid 1 will have slow writes and fast reads > > > While that is true, you're missing the real difference here, the number of > HDDs. With 4 of them in the RAID10 they will be twice as fast as the 2 > disk RAID1. > This can be tweaked even further with the various RAID 10 layout options, > "man md". > > > the question is will there be huge performance difference between Raid1 and Raid 10 when it comes small files like mail queue ? > > > Again as explained above, double the IOPS, so yes. > > Christian > > > > thanks > > rajesh > > ----- Original Message ----- > > From: Christian Balzer [mailto:chibi at gol.com] > > To: dovecot at dovecot.org > > Cc: kevin at my.walr.us > > Sent: Mon, 13 Feb 2017 10:46:13 +0900 > > Subject: > > > > > > Hello, > > > > On Sun, 12 Feb 2017 08:27:21 -0500 KT Walrus wrote: > > > > > Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I?d like to hear about some real-world numbers that could be used for budgeting a new project like mine. > > > > > > > Those servers would actually be the wrong ones to look at, as they are > > primarily accessed by the aforementioned broken client, so the numbers are > > somewhat skewed. > > However looking at other servers with a more "normal" user spread things > > aren't actually too different. > > > > The average number of sessions per user tends to be 2. > > The clear majority (over 50%) only has one session open (people with well > > behaved and configured clients watching the INBOX mostly). > > Another 30% has 2 sessions open, again the typical state of this would be > > clients watching another mailbox besides INBOX, typically SENT. > > The rest has 3 and more sessions open. > > > > The number of sessions could of course be drastically reduced if any > > client would support IMAP NOTIFY, alas none that I'm aware of do. > > > > Lastly no more than 60% of the session seem to be in IDLE at any given > > time, so my comments about RAM and IMAP hibernation effectiveness stand. > > > > > Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > > > > > > > Yes to Dovecot Proxying, of course. > > > > No idea about MySQL, with (Open)LDAP nothing is breaking a sweat at an > > average of 140 logins per second (IMAP and POP) on the 2 proxy servers. > > If you can fit your entire dataset into RAM it should be fine, my LDAP > > servers fall into that category and take about 10% of a slow (1.2GHz, 34%, > > power-save mode) core only to handle the that load (also 2 servers). > > And the rate of logins/s is what you need to worry about most and optimize > > for. > > > > The proxies will of course have to do the shuffling of data and SSL > > en/de-coding, but again they're not particular busy with that. > > > > The number of sessions comes into play when looking at the number of login > > processes on the proxies and their memory footprint. > > An IMAP login process on the proxies in performance mode with a client > > limit of 1000 will consume about 55MB at most. > > So assume at least 55KB RAM per session. > > > > Read Timo's mail I linked to about IMAP hibernation, AFAIK nothing has > > happened to make proxies more supportive for this though. > > > > Christian > > > > On Feb 12, 2017, at 1:58 AM, Christian Balzer wrote: > > > > > > > > > > > > Hello, > > > > > > > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > > > > > > >>> 1. 256GB of real RAM, swap is for chums. > > > >> > > > >> Are you sure that 100,000 IMAP sessions wouldn?t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > > > >> > > > > > > > > I'm sure about tax and death, not much else. > > > > > > > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > > > > even if it were to SSDs. > > > > Incidentally the servers I'm talking about here have their OS and swap on > > > > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > > > > S3610s. > > > > > > > > Relying on the kernel to make swap decisions is likely to result in much > > > > reduced performance even with fast SWAP when you're overcommitting things > > > > on that scale. > > > > > > > > > > > > But read on. > > > > > > > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don?t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > > > >> > > > >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren?t doing much most of the time and shouldn?t really average 2MB of active data per session that needs to be resident in main memory at all times. > > > >> > > > > See IMAP hibernation: > > > > https://www.mail-archive.com/dovecot at dovecot.org/msg63429.html > > > > > > > > I'm going to deploy/test this in production in about 2 months from now, > > > > but if you look at the link and the consequent changelog entries you'll see > > > > that it has certain shortcomings and bug fixes in pretty much each release > > > > after it was introduced. > > > > > > > > But this is the correct way to tackle things, not SWAP. > > > > > > > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > > > > here will be hibernated at any given time I'd be pleasantly surprised. > > > > > > > > Because between: > > > > > > > > 1. Finding a sensible imap_hibernate_timeout. > > > > > > > > 2. Having well behaved clients that keep idling instead of restarting the > > > > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ ) > > > > > > > > 3. Having lots of mobile clients who either get disconnected (invisible to > > > > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > > > > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > > > > forcing people to use 1 minute IDLE renewals, making 1. up there a > > > > nightmare). > > > > > > > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > > > > sessions, don't put them into IDLE and thus having them expire after 30 > > > > minutes. > > > > > > > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > > > > at least. > > > > > > > >> My mail server isn?t that large yet as I haven?t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn?t limited to 50,000 IMAP sessions on a server... > > > >> > > > > > > > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > > > > but the scalability of things OTHER than Dovecot as I pointed out plus > > > > that little detail of failure domains (do you really want half of your > > > > eggs in one basket?) argue for scaling out after a certain density. > > > > > > > > I'm feeling my way there at this time, but expect more than 100k sessions > > > > per server to be tricky. > > > > > > > > Lastly, when I asked about 500k sessions per server here not so long ago, > > > > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html ) > > > > Timo mentioned that he's not aware of anybody doing more than 50k per > > > > server, something I got licked already and definitely will go to 100k > > > > eventually. > > > > > > > > Regards, > > > > > > > > Christian > > > >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer wrote: > > > >>> > > > >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > > > >>> > > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > > >>>> > > > >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > > > >>>> > > > >>> That depends on the IMAP session, read the mailbox size and index size, > > > >>> etc. > > > >>> Some are significantly larger: > > > >>> --- > > > >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > > > >>> --- > > > >>> > > > >>> But yes, as somebody who has mailbox servers with 55k+ session the average > > > >>> is around 1.6MB. > > > >>> > > > >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > > > >>>> > > > >>> You will want: > > > >> > > > >>> 2. Understanding how to tune Dovecot and more importantly the overall > > > >>> system to such a task (see that PID up there?). > > > >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run > > > >>> and others like atop actually killing dovecot (performance wise, not > > > >>> literally) when doing their obviously flawed cleanup on exit. Some things > > > >>> clearly do NOT scale well. > > > >>> > > > >>> My current goal is to have 100k capable servers that work well, 200k in a > > > >>> failover scenario, but that won't be particular enjoyable. > > > >>> > > > >>> Christian > > > >>> > > > >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer wrote: > > > >>>>> > > > >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > > > >>>>> > > > >>>>>> hello > > > >>>>>> > > > >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > > > >>>>>> > > > >>>>> > > > >>>>> Be very precise here, you expect to see 1500 as the result of > > > >>>>> "doveadm who |grep pop3 |wc -l"? > > > >>>>> > > > >>>>> Because that implies an ungodly number of POP3 connects per second, given > > > >>>>> the typically short duration of these. > > > >>>>> > > > >>>>> 1500 IMAP connections (note that frequently a client will have more than > > > >>>>> the INBOX open and thus have more than one session and thus process on the > > > >>>>> server) are a much easier proposition, provided they are of the typical > > > >>>>> long lasting type. > > > >>>>> > > > >>>>> So can you put a number to your expected logins per second (both protocols)? > > > >>>>> > > > >>>>>> my server > > > >>>>>> > > > >>>>>> server configuration > > > >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > > > >>>>>> gb hdd for data (No raid) > > > >>>>>> > > > >>>>> No RAID and no other replication like DRBD? > > > >>>>> Why would you even bother? > > > >>>>> > > > >>>>> How many users/mailboxes in total with what quota? > > > >>>>> > > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > > >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > > > >>>>> dentries) in RAM. > > > >>>>> > > > >>>>> If you really have several hundreds logins/s, you're facing several > > > >>>>> bottlenecks: > > > >>>>> 1. Login processes themselves (easily fixed by high performance mode) > > > >>>>> 2. Auth processes (that will depend on your backends, method mostly) > > > >>>>> 3. Dovecot master process (spawning mail processes) > > > >>>>> > > > >>>>> The later is a single-threaded process, so it will benefit from a faster > > > >>>>> CPU core. > > > >>>>> It can be dramatically improved by enabling process re-usage, see: > > > >>>>> http://wiki.dovecot.org/PerformanceTuning > > > >>>>> > > > >>>>> However that also means more memory usage. > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> Christian > > > >>>>> > > > >>>>>> > > > >>>>>> thanks > > > >>>>>> rajesh > > > >>>>>> > > > >>>>> > > > >>>>> [snip] > > > >>>>> -- > > > >>>>> Christian Balzer Network/Systems Engineer > > > >>>>> chibi at gol.com Global OnLine Japan/Rakuten Communications > > > >>>>> http://www.gol.com/ > > > >>>> > > > >>> > > > >>> > > > >>> -- > > > >>> Christian Balzer Network/Systems Engineer > > > >>> chibi at gol.com > Global OnLine Japan/Rakuten Communications > > > >>> http://www.gol.com/ > > > > > > > > > > > > > -- > > > > Christian Balzer Network/Systems Engineer > > > > chibi at gol.com Global OnLine Japan/Rakuten Communications > > > > http://www.gol.com/ > > > > > > > > > -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From gandalf.corvotempesta at gmail.com Wed Feb 15 09:47:30 2017 From: gandalf.corvotempesta at gmail.com (Gandalf Corvotempesta) Date: Wed, 15 Feb 2017 10:47:30 +0100 Subject: Upgrade from 1.2 to 2.2 Message-ID: Hi, I have a production server running Debian Squeeze with Dovecot 1.2 I would like to upgrade everything to Jessie, running 2.2 Last time I did something similiar, but from Lenny to Squeeze, the whole dovecot installation broke. Any suggestion on how to upgrade everything ? Can I test our current configuration with a newer dovecot version to be sure that everything would be converted properly with "doveconf -n -c /etc/dovecot/dovecot.conf > dovecot-2.conf" ? Thank you From k0ste at k0ste.ru Wed Feb 15 10:13:31 2017 From: k0ste at k0ste.ru (Konstantin Shalygin) Date: Wed, 15 Feb 2017 17:13:31 +0700 Subject: [Sieve] Is the way to run external script to get attachment? Message-ID: Hello. Read all about Extprograms, but have a question. Use case: Every day we have couple messages with attachment (from one sender, this is one-way communication). This attachment is uploads to ownCloud. By hand... need some automation. As I see, pipe cat execute scripts with text data USER/FROM/SUBJECT - and I think with variables it can be any text data, how about attachment? How I see this: script executes and as arg receives from sieve attachment file, or message file and we can parse it for attachment. When we have attachment - upload to ownCloud via RESTapi and purge message. Found some realization for encrypt messages https://github.com/EtiennePerot/gpgit/blob/master/encmaildir.sh but for me more easy just connect to IMAP and do what I want (but I love sieve). Thanks. From aki.tuomi at dovecot.fi Wed Feb 15 12:27:23 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 15 Feb 2017 14:27:23 +0200 Subject: Upgrade from 1.2 to 2.2 In-Reply-To: References: Message-ID: <40423105-f818-f0c9-b2be-9b554f14d426@dovecot.fi> On 15.02.2017 11:47, Gandalf Corvotempesta wrote: > Hi, > I have a production server running Debian Squeeze with Dovecot 1.2 > I would like to upgrade everything to Jessie, running 2.2 > > Last time I did something similiar, but from Lenny to Squeeze, the > whole dovecot installation broke. > > Any suggestion on how to upgrade everything ? Can I test our current > configuration with a newer dovecot version to be sure that everything > would be converted properly with > "doveconf -n -c /etc/dovecot/dovecot.conf > dovecot-2.conf" ? > > Thank you Hi! For good pointers, see http://wiki.dovecot.org/Upgrading it's not complete, but it should give you some idea. Aki From gandalf.corvotempesta at gmail.com Wed Feb 15 12:31:04 2017 From: gandalf.corvotempesta at gmail.com (Gandalf Corvotempesta) Date: Wed, 15 Feb 2017 13:31:04 +0100 Subject: Upgrade from 1.2 to 2.2 In-Reply-To: <40423105-f818-f0c9-b2be-9b554f14d426@dovecot.fi> References: <40423105-f818-f0c9-b2be-9b554f14d426@dovecot.fi> Message-ID: 2017-02-15 13:27 GMT+01:00 Aki Tuomi : > For good pointers, see http://wiki.dovecot.org/Upgrading > > it's not complete, but it should give you some idea. I've already read that, and as wrote previously, everything broke down. dovecont -n wasn't able to convert the configuration file and dovecot wasn't started properly. The only way to fix was to downgrade. As this is a production server, I would like to avoid this kind of issue..... From eduardo at kalinowski.com.br Wed Feb 15 12:38:26 2017 From: eduardo at kalinowski.com.br (Eduardo M KALINOWSKI) Date: Wed, 15 Feb 2017 12:38:26 +0000 Subject: Upgrade from 1.2 to 2.2 In-Reply-To: References: <40423105-f818-f0c9-b2be-9b554f14d426@dovecot.fi> Message-ID: <20170215123826.Horde.cELfEIOoaB0pgb3VFgiCCUH@mail.kalinowski.com.br> On Qua, 15 Fev 2017, Gandalf Corvotempesta wrote: > 2017-02-15 13:27 GMT+01:00 Aki Tuomi : >> For good pointers, see http://wiki.dovecot.org/Upgrading >> >> it's not complete, but it should give you some idea. > > I've already read that, and as wrote previously, everything broke down. > dovecont -n wasn't able to convert the configuration file and dovecot > wasn't started properly. > The only way to fix was to downgrade. > > As this is a production server, I would like to avoid this kind of issue..... Set up a server replicating exactly the configuration you have at present. Doesn't need to be anything fancy, a virtual machine is enough. Actually, a virtual machine is a great tool if you can make snapshots and then rollback to known states if any attempt at a change fails. Do your upgrade work in this new server. Take note of the problems and the solutions. If you have concrete issues, you can ask help in this mailing list. Once you get your test server working, you can then upgrade the real server using what you've learned from the test server. -- Eduardo M KALINOWSKI eduardo at kalinowski.com.br From skdovecot at smail.inf.fh-brs.de Wed Feb 15 12:39:59 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 15 Feb 2017 13:39:59 +0100 (CET) Subject: Upgrade from 1.2 to 2.2 In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 15 Feb 2017, Gandalf Corvotempesta wrote: > Any suggestion on how to upgrade everything ? Can I test our current > configuration with a newer dovecot version to be sure that everything > would be converted properly with > "doveconf -n -c /etc/dovecot/dovecot.conf > dovecot-2.conf" ? actually, you answer your question yourself: "Can I test" yes: you can and *should* test, but it's a major upgrade. I did it so: + checked out the automated conversation, + each change I made to the conf of Dovecot v1, I read, what to do in v2 and applied the change to the default configuration manually with hints from the step above + once all in place, I started a test system with a bunch of accounts that used all features: sieve, sharing, ... . - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWKRMH3z1H7kL/d9rAQIbBgf/YUDU2IQbb8Uhif9q52y2Rk0eJWZJ0j6X LYEEEyHBFgxQA8YRRrgRwkShRPLXzUKIEmMGGsdyCcxwqZvnufcjWuS76EObs1xw Mg5NWbiwgtx0QrCLGoPt2KAC8h2a27AEETgoiF+WzlNteG/uiI7Yz2nKNXMU2Cc0 8rB9BGs16PAukBE0Zkgs2NLMaFt9JqiRPbHxtGaMrecKPDxx8BusK6zpxq69zUDQ Rzdw21+t7eZhSuSbSbf9N/aZRYtJ8b/2RC4FXNuNDTV6vvVngRkxLJSwwfSMB3hu GheseYUKmq9vQWy+lWL1R9tT5gZgTO1s2kmg4uAhQ0KsQ3k0+aDMEw== =rXtr -----END PGP SIGNATURE----- From aki.tuomi at dovecot.fi Wed Feb 15 12:41:30 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 15 Feb 2017 14:41:30 +0200 Subject: Upgrade from 1.2 to 2.2 In-Reply-To: References: Message-ID: <485dfb4a-87c5-2a79-bf44-03f2251a8237@dovecot.fi> On 15.02.2017 14:39, Steffen Kaiser wrote: > On Wed, 15 Feb 2017, Gandalf Corvotempesta wrote: > > > Any suggestion on how to upgrade everything ? Can I test our current > > configuration with a newer dovecot version to be sure that everything > > would be converted properly with > > "doveconf -n -c /etc/dovecot/dovecot.conf > dovecot-2.conf" ? > > actually, you answer your question yourself: > > "Can I test" > > yes: you can and *should* test, but it's a major upgrade. > > I did it so: > > + checked out the automated conversation, > + each change I made to the conf of Dovecot v1, I read, what to do in > v2 and applied the change to the default configuration manually with > hints from the step above > + once all in place, I started a test system with a bunch of accounts > that used all features: sieve, sharing, ... . > > -- Steffen Kaiser Btw, I think the most safe option would be to make *new* server (advantage to upgrade the platform here too) and migrate users to the new server instead of upgrading. Just my 2c. Aki From gandalf.corvotempesta at gmail.com Wed Feb 15 13:11:34 2017 From: gandalf.corvotempesta at gmail.com (Gandalf Corvotempesta) Date: Wed, 15 Feb 2017 14:11:34 +0100 Subject: Upgrade from 1.2 to 2.2 In-Reply-To: <485dfb4a-87c5-2a79-bf44-03f2251a8237@dovecot.fi> References: <485dfb4a-87c5-2a79-bf44-03f2251a8237@dovecot.fi> Message-ID: 2017-02-15 13:41 GMT+01:00 Aki Tuomi : > Btw, I think the most safe option would be to make *new* server > (advantage to upgrade the platform here too) and migrate users to the > new server instead of upgrading. Just my 2c. Probably, yes. From florent at coppint.com Wed Feb 15 15:46:05 2017 From: florent at coppint.com (florent at coppint.com) Date: Wed, 15 Feb 2017 16:46:05 +0100 Subject: doveadm backup and quota Message-ID: <595ed655e4dae0675983552bba694059@coppint.com> Hi, I'm looking to convert all mailboxes from Maildir format to mdbox. When I try my procedure on a user having ~45GB of messages, its quota is growing very fast and is now up to 66GB. I run this command : doveadm -v backup -u user at domain.com mdbox:/mnt/maildata1/domain.com/user/mdbox I expected this command to make a copy of user mailbox, it does not make sense to count the backup in its quota. Maybe quota is simply recalculated ? But running a "du -sh" on its home directory is showing a 46G disk usage ! (backup is not stored in home dir) Can someone tell me what happens on my setup ? I use Dovecot 2.2.24. Thank you. Florent From florent at coppint.com Wed Feb 15 15:54:32 2017 From: florent at coppint.com (florent at coppint.com) Date: Wed, 15 Feb 2017 16:54:32 +0100 Subject: doveadm backup and quota In-Reply-To: <595ed655e4dae0675983552bba694059@coppint.com> References: <595ed655e4dae0675983552bba694059@coppint.com> Message-ID: <5b0ec01ccb9c971272d5f85427ad5c27@coppint.com> Sorry I didn't search enough before posting. I found this : http://www.dovecot.org/list/dovecot/2012-February/063585.html I will try the workaround. From jtam.home at gmail.com Thu Feb 16 00:01:16 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 15 Feb 2017 16:01:16 -0800 (PST) Subject: Upgrade from 1.2 to 2.2 In-Reply-To: References: Message-ID: Eduardo M KALINOWSKI writes in reesponse to Gandalf Corvotempesta post: >> I've already read that, and as wrote previously, everything broke down. >> dovecont -n wasn't able to convert the configuration file and dovecot >> wasn't started properly. >> The only way to fix was to downgrade. >> >> As this is a production server, I would like to avoid this kind of issue..... > > Set up a server replicating exactly the configuration you have at > present. Doesn't need to be anything fancy, a virtual machine is > enough. Actually, a virtual machine is a great tool if you can make > snapshots and then rollback to known states if any attempt at a change > fails. If you absolutely can't test on a VM or another test host, you can run the new version along side your current version in another installation directory. Configure your new installation to run on alternate ports. You can work off copies of mail data as well if you really want to isolate your test. Then you can gradually phase in your new version by swapping alternate port/data with production port/data. Joseph Tam From flatworm at users.sourceforge.net Thu Feb 16 09:19:00 2017 From: flatworm at users.sourceforge.net (Konstantin Khomoutov) Date: Thu, 16 Feb 2017 12:19:00 +0300 Subject: [Sieve] Is the way to run external script to get attachment? In-Reply-To: References: Message-ID: <20170216121900.15484f1ff86e6e9d0cb53781@domain007.com> On Wed, 15 Feb 2017 17:13:31 +0700 Konstantin Shalygin wrote: > Read all about Extprograms, but have a question. > Use case: > Every day we have couple messages with attachment (from one sender, > this is one-way communication). This attachment is uploads to > ownCloud. By hand... need some automation. > As I see, pipe cat execute scripts with text data USER/FROM/SUBJECT - > and I think with variables it can be any text data, how about > attachment? > > How I see this: script executes and as arg receives from sieve > attachment file, or message file and we can parse it for attachment. > When we have attachment - upload to ownCloud via RESTapi and purge > message. > > Found some realization for encrypt messages > https://github.com/EtiennePerot/gpgit/blob/master/encmaildir.sh > but for me more easy just connect to IMAP and do what I want (but I > love sieve). Do you really need Sieve for this? IMO such things are best handled in the SMTP server: 1) Set up an alias for the mail address receiving these mails. 2) Make that alias expand both to its real final destination (an address, mails to which would be delivered to the IMAP folder as before) and to a special "program" entry which looks like "| /path/to/the/external/program". See `man aliases` [4] for more info. The program is supposed to receive the mail message to its standard input stream and do whatever it wishes with them. For instance, this could be a shell script calling something like `ripmime` [1] on the input, saving the attachments and then calling into whatever would upload them. Note that if the program completed its task OK, it should return with the zero result code. Otherwise it's advised to use one of result codes defined by Sendmail, which are described in [2] and whose exact numeric values could be googled (for instance, see [3]). If your program return a "known" exit code on error, you'll get better diagnostics reported for that failure by your SMTP server. 1. http://www.pldaniels.com/ripmime/ 2. http://docstore.mik.ua/orelly/networking/sendmail/ch36_05.htm 3. https://gist.github.com/bojanrajkovic/831993 4. http://www.postfix.org/aliases.5.html From stephan at rename-it.nl Fri Feb 17 10:45:24 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 17 Feb 2017 11:45:24 +0100 Subject: fts_solr and connection via https:// In-Reply-To: References: <3e405043-9e5b-39a5-6ff1-0c462a7bb8cb@jan-von.de> <836bf373-79be-a2cf-3012-45b238014b1d@rename-it.nl> <4c177121-f649-11e5-7f34-f91ea20ceb79@jan-von.de> <69269fb5-65a9-8cfa-ea06-4e6a3cf0232d@rename-it.nl> Message-ID: <71c06e27-7028-a457-a519-5566b00cd42f@rename-it.nl> Op 8-2-2017 om 21:07 schreef Jan Vonde: > Am 07.02.2017 um 12:29 schrieb Stephan Bosch: >> >> Op 31-1-2017 om 6:33 schreef Jan Vonde: >>> Am 31.01.2017 um 00:04 schrieb Stephan Bosch: >>>> Op 1/22/2017 om 12:01 PM schreef Stephan Bosch: >>>>> Op 1/22/2017 om 10:01 AM schreef Jan Vonde: >>>>>> I tried adding the following settings but that didn't help: >>>>>> ssl_ca = < /etc/ssl/certs/ca-certificates.crt >>>>>> ssl_client_ca_dir = /etc/ssl/certs >>>>>> >>>>>> Can you give me a hint how I can get the ssl certificate accepted? >>>>> That should normally have done the trick. However, the sources tell me >>>>> that no ssl_client settings are propagated to the http_client used by >>>>> fts-solr, so SSL is not currently supported it seems. >>>>> >>>>> I'll check how easy it is to add that. >>>> Just to keep you informed: I created a patch, but it is still being >>>> tested. >>>> >>> Thanks for the update Stephan! Awesome! Looking forward to test it >>> myself :-) >> https://github.com/dovecot/core/commit/526631052ca3175357302af8fa7dcbf763b40c53 >> > Thank you. I am using now the following version: > 2.3.0.alpha0 (2eeea57) [XI:2:2.3.0~alpha0-1~auto+650] > > The error messages I am getting now are like this: > > doveadm(user at host): Info: Received invalid SSL certificate: unable to > get local issuer certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt > Authority X3 > doveadm(user at host): Error: fts_solr: Lookup failed: 9002 SSL handshaking > with 5.45.106.248:443 failed: read(SSL 5.45.106.248:443) failed: > Received invalid SSL certificate: unable to get local issuer > certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 > > > You can connect to 5.45.106.248:443 and IMHO everything is correct with > the chain. > > > I am no SSL expert, but I am reading it as "doveadm and its ssl part > cannot verify the Let's Encrypt certificate". It would need the DST Root > CA X3 and this is in the local trust store (ssl_client_ca_dir...) > > > Do you have another hint maybe? We seem to have found another issue there. More on this will follow. Regards, Stephan. From stephan at rename-it.nl Fri Feb 17 13:34:45 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 17 Feb 2017 14:34:45 +0100 Subject: Sieve removeflag Action In-Reply-To: <20170119094251.GA32291@nihlus.leuxner.net> References: <20170113081514.GA60507@nihlus.leuxner.net> <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> Message-ID: <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> Op 19-1-2017 om 10:43 schreef Thomas Leuxner: > * Stephan Bosch 2017.01.19 10:32: > >> Could you provide a more detailed example? > Sure. Personal script v > > /var/vmail/domains/leuxner.net/tlx/.dovecot.sieve: > > require ["include","copy","fileinto","imap4flags","vacation"]; > include :global "global"; > > -- > > Global script referenced v > > /var/vmail/conf.d/leuxner.net/sieve/global.sieve: > > require ["fileinto","imap4flags","duplicate"]; > > #Newsletters > > if header :contains "List-Id" "debian-security-announce.lists.debian.org" > { > removeflag "\\Flagged $MailFlagBit1"; > fileinto ":public/Newsletters/Debian/Security"; > addflag "\\Flagged $MailFlagBit1"; > keep; > } > > -- > Basically it is reproducible with the same stanza we used before by putting this in the included script: > > #Test > if address :is "From" "user at example.com" > { > removeflag "\\Flagged $MailFlagBit1"; > fileinto "Trash"; > addflag "\\Flagged $MailFlagBit1"; > keep; > } Couldn't reproduce this with v2.3.devel yesterday (i.e. no flags set for the Security mailbox and all flags set for the message in INBOX), but I will try later with some older version. Regards, Stephan. From ben+dovecot at list-subs.com Fri Feb 17 16:24:25 2017 From: ben+dovecot at list-subs.com (Ben) Date: Fri, 17 Feb 2017 16:24:25 +0000 Subject: Sieve not filtering Message-ID: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> Hi, I have copied accross a known-good sieve file from a working server and its not filtering. Everything just gets chucked into INBOX. doveconf-n at the bottom of this mail Feb 17 16:05:20 server postfix/smtpd[51562]: 7FA5E12CBBC: client=unknown[192.168.167.57] Feb 17 16:05:23 server postfix/cleanup[51565]: 7FA5E12CBBC: message-id=<> Feb 17 16:05:23 server postfix/qmgr[45471]: 7FA5E12CBBC: from=, size=182, nrcpt=1 (queue active) Feb 17 16:05:23 server dovecot: lmtp(51467): Connect from local Feb 17 16:05:23 server dovecot: auth-worker(51568): passwd(recipient at example.com): unknown user Feb 17 16:05:23 server dovecot: lmtp(51467, recipient at example.com): 1JK2B0Mfp1gLyQAAHLpRfg: sieve: msgid=unspecified: stored mail into mailbox 'INBOX' Feb 17 16:05:23 server dovecot: lmtp(51467): Disconnect from local: Successful quit Feb 17 16:05:23 server postfix/lmtp[51566]: 7FA5E12CBBC: to=, orig_to=, relay=server.example.com[private/dovecot-lmtp], delay=4.9, delays=4.9/0.01/0/0.05, dsn=2.0.0, status=sent (250 2.0.0 1JK2B0Mfp1gLyQAAHLpRfg Saved) Feb 17 16:05:23 server postfix/qmgr[45471]: 7FA5E12CBBC: removed This is the syntax I'm using in my sieve file: require ["fileinto","envelope"]; if anyof (address :is :all "to" ["recipient+something at example.com?,?mailing_list at example.com?], header :contains ["Cc", "Delivered-To"] ["recipient+something at example.com?,?mailing_list at example.com?]) { fileinto ?THIS_FOLDER?; stop; } # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.6.1.el7.x86_64 x86_64 CentOS Linux release 7.3.1611 (Core) auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1 first_valid_uid = 1000 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body environment mailbox date ihave enotify mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = ~/.dovecot.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = its_virtmail mode = 0660 user = its_virtmail } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = its_virtmail } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieves { address = port = 5190 ssl = yes } } ssl = required ssl_cert = was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 20 } From mail at jan-von.de Fri Feb 17 16:27:43 2017 From: mail at jan-von.de (Jan Vonde) Date: Fri, 17 Feb 2017 17:27:43 +0100 Subject: fts_solr and connection via https:// In-Reply-To: <71c06e27-7028-a457-a519-5566b00cd42f@rename-it.nl> References: <3e405043-9e5b-39a5-6ff1-0c462a7bb8cb@jan-von.de> <836bf373-79be-a2cf-3012-45b238014b1d@rename-it.nl> <4c177121-f649-11e5-7f34-f91ea20ceb79@jan-von.de> <69269fb5-65a9-8cfa-ea06-4e6a3cf0232d@rename-it.nl> <71c06e27-7028-a457-a519-5566b00cd42f@rename-it.nl> Message-ID: <62f756c9-27b7-163c-34ed-6fa7b8de0e7c@jan-von.de> Am 17.02.2017 um 11:45 schrieb Stephan Bosch: > Op 8-2-2017 om 21:07 schreef Jan Vonde: >> Am 07.02.2017 um 12:29 schrieb Stephan Bosch: >>> Op 31-1-2017 om 6:33 schreef Jan Vonde: >>>> Am 31.01.2017 um 00:04 schrieb Stephan Bosch: >>>>> Op 1/22/2017 om 12:01 PM schreef Stephan Bosch: >>>>>> Op 1/22/2017 om 10:01 AM schreef Jan Vonde: >>>>>>> I tried adding the following settings but that didn't help: >>>>>>> ssl_ca = < /etc/ssl/certs/ca-certificates.crt >>>>>>> ssl_client_ca_dir = /etc/ssl/certs >>>>>>> >>>>>>> Can you give me a hint how I can get the ssl certificate accepted? >>>>>> That should normally have done the trick. However, the sources >>>>>> tell me >>>>>> that no ssl_client settings are propagated to the http_client used by >>>>>> fts-solr, so SSL is not currently supported it seems. >>>>>> >>>>>> I'll check how easy it is to add that. >>>>> Just to keep you informed: I created a patch, but it is still being >>>>> tested. >>>>> >>>> Thanks for the update Stephan! Awesome! Looking forward to test it >>>> myself :-) >>> https://github.com/dovecot/core/commit/526631052ca3175357302af8fa7dcbf763b40c53 >>> >>> >> Thank you. I am using now the following version: >> 2.3.0.alpha0 (2eeea57) [XI:2:2.3.0~alpha0-1~auto+650] >> >> The error messages I am getting now are like this: >> >> doveadm(user at host): Info: Received invalid SSL certificate: unable to >> get local issuer certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt >> Authority X3 >> doveadm(user at host): Error: fts_solr: Lookup failed: 9002 SSL handshaking >> with 5.45.106.248:443 failed: read(SSL 5.45.106.248:443) failed: >> Received invalid SSL certificate: unable to get local issuer >> certificate: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 >> >> >> You can connect to 5.45.106.248:443 and IMHO everything is correct with >> the chain. >> >> >> I am no SSL expert, but I am reading it as "doveadm and its ssl part >> cannot verify the Let's Encrypt certificate". It would need the DST Root >> CA X3 and this is in the local trust store (ssl_client_ca_dir...) >> >> >> Do you have another hint maybe? > > We seem to have found another issue there. More on this will follow. > Thanks for the update and have a nice weekend, Jan :-) From b.sebode at linet-services.de Fri Feb 17 16:58:08 2017 From: b.sebode at linet-services.de (Bastian Sebode) Date: Fri, 17 Feb 2017 17:58:08 +0100 Subject: Problem with Let's Encrypt Certificate Message-ID: Hello Folks, my StartCom SSL-Certificate expires soon and so I wanted to switch to Let's Encrypt Certificates instead. Unfortunatelly Thunderbird seems not to like it, although all -tested- other Clients work without any problems. When I connect with Thunderbird it sends an "Encrypted Alert" directly after the TLS handshake although Dovecot wants to continue the session. In the Dovecot Log it says: Feb 17 17:27:17 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [82.100.242.26] Feb 17 17:27:17 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [82.100.242.26] Feb 17 17:27:17 imap-login: Warning: SSL alert: where=0x4004, ret=554: fatal bad certificate [82.100.242.26] But the certificate is okay, cause it works with other Mailclients and openssl also says so. What certificate is Thunderbird complaining about? Thunderbird says something like "There's no supported authentication method". I don't use any Certificates for Client Authentication, neither in Dovecot nor in Thunderbird. When I do, it fails the same way. Weirdly my friend uses the same Dovecot Version with Let's Encrypt on his Server and it works with Thunderbird without any flaws. Mine fails the same way in his Thunderbird and also in a fresh installation. After two weeks of investigating I still have no clue why it behaves like this. I uploaded two Wireshark tracefiles, further logs and dovecot -n, may be someone sees any possible reasons for this weird behavior or has any further tips on solving this issue. https://sebode-online.de/dovecot-letsencrypt/ Every hint is highly appreciated! Best Regards Bastian -- Bastian Sebode Fachinformatiker Systemintegration LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de LINET in den sozialen Netzwerken: www.twitter.com/linetservices | www.facebook.com/linetservices Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ Gesch?ftsf?hrung: Timo Springmann, Mirko Savic und Moritz Bunkus HR B 9170 Amtsgericht Braunschweig USt-IdNr. DE 259 526 516 From lists at tigertech.com Fri Feb 17 18:28:28 2017 From: lists at tigertech.com (Robert L Mathews) Date: Fri, 17 Feb 2017 10:28:28 -0800 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: On 2/17/17 8:58 AM, Bastian Sebode wrote: > I uploaded two Wireshark tracefiles, further logs and dovecot -n Looking at your dovecot -n, you're using two different files here: ssl_cert = Hi Dovecot Users, I?ve configured dovecot dsync replication and I see troubles in the logs and get user complaints which I can?t explain. I found similar threads on this mailinglist, but I couldn?t find a solution anywhere. Does anybody have dsync running without problems on a high volume mailserver? I see the following logs, examples given: Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: /var/mail/zoechi/dovecot-uidlist: Duplicate file entry at line 10395: 1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377 (uid 41092 -> 41093) - retrying by re-reading from beginning with this one I?m not sure - it might be that this is completely OK because due to replication UIDs clash. Maybe that?s OK, but I couldn?t find a confirmation. Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: Maildir /var/mail/zoechi: Expunged message reappeared, giving a new UID (old uid=41092, file=1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377) This one is definitely a problem, deleted messages re-appear in the mailbox. I made sure that the 2 hosts doing replication have different hostnames. I run 2.2.27 (from debian-jessie backports). Config below. Thanks for hints (or pointers to other example configs where dsync works without problems) # dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.8.14 x86_64 Debian 8.7 ext4 auth_verbose = yes debug_log_path = /var/log/dovecot.debug doveadm_password = # hidden, use -P to show it first_valid_gid = 106 first_valid_uid = 104 hostname = localhost last_valid_gid = 106 last_valid_uid = 104 mail_gid = dovecot mail_location = maildir:/var/mail/%n mail_plugins = quota fts fts_lucene virtual notify replication mail_temp_dir = /var/lib/dovecot/tmp mail_uid = dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader namespace { list = children location = virtual:/var/mail/%n/virtual prefix = virtual. separator = . } namespace inbox { inbox = yes list = yes location = mailbox "Deleted Messages" { auto = subscribe special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { default_language = de fts = lucene fts_lucene = whitespace_chars=@. mail_replica = tcp:172.16.1.1:12345 quota = maildir:User quota quota_rule = *:storage=5G quota_rule2 = Trash:storage=+200M quota_rule3 = Spam:ignore quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /etc/sieve/%n.sieve sieve_default = /etc/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +editheader } pop3_deleted_flag = $POP3Deleted postmaster_address = postmaster at wogri.at protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { user = dovecot } unix_listener replication-notify { user = dovecot } } service doveadm { inet_listener { port = 12345 } } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { port = 2003 } unix_listener lmtp { user = dovecot } user = dovecot } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/sbin/quota-warning.sh unix_listener quota-warning { user = dovecot } user = dovecot } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = required ssl_cert = References: Message-ID: Hey Robert, thanks for your reply. Am 17.02.2017 um 19:28 schrieb Robert L Mathews: > Looking at your dovecot -n, you're using two different files here: > > ssl_cert = ssl_key = > Are you sure these two files match, and contain the right things in the > right order? > Yes, unfortunately I'm sure that everything has the right order. As you can see in the trace, both certificates (mine and the intermediate) get transferred to the client on connection. > We use a single PEM file as input for both of these parameters, and that > PEM file contains, in this order: > > -----BEGIN RSA PRIVATE KEY----- > ... > -----BEGIN CERTIFICATE----- > ... > -----BEGIN CERTIFICATE----- > > ... where the first BEGIN CERTIFICATE is the specific hostname one, and > the second BEGIN CERTIFICATE is the Let's Encrypt X3 intermediate > certificate that ends with "DNFu0Qg==". > Tried that, but without success. But your usage doesn't seem right to me. The parameters are not called ssl_cert and ssl_key for nothing. ;-) Normally you don't want your private key to have any other permissions than 600. > You're also manually specifying these non-default parameters: > > ssl_cipher_list = ... > ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 > > For testing, I would simplify. Does it work without any of those three > things set? > Tried this before. I set all SSL specific settings exactly like my friend where it works without a problem. But it doesn't work for me. Thanks anyway for your effort! Bastian -- Bastian Sebode Fachinformatiker Systemintegration LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de LINET in den sozialen Netzwerken: www.twitter.com/linetservices | www.facebook.com/linetservices Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ Gesch?ftsf?hrung: Timo Springmann, Mirko Savic und Moritz Bunkus HR B 9170 Amtsgericht Braunschweig USt-IdNr. DE 259 526 516 From listeem at ksb.id.lv Fri Feb 17 20:57:34 2017 From: listeem at ksb.id.lv (KSB) Date: Fri, 17 Feb 2017 22:57:34 +0200 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: On 2017.02.17. 22:31, Bastian Sebode wrote: > Hey Robert, > > thanks for your reply. > > Am 17.02.2017 um 19:28 schrieb Robert L Mathews: >> Looking at your dovecot -n, you're using two different files here: >> >> ssl_cert = > ssl_key = > Are You sure, chain.pem contains your cert + immediate? By default certbot in chain.pem includes only itermediate cert's and if you wan't everything, it's included in fullchain. -- KSB From aki.tuomi at dovecot.fi Fri Feb 17 20:58:31 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 17 Feb 2017 22:58:31 +0200 (EET) Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: <67651323.902.1487365112484@appsuite-dev.open-xchange.com> Usually with LE, the filename is fullchain.pem, not chain.pem. Can you please doublecheck this? Also, try openssl s_client -connect hostname:143 -starttls imap Aki > On February 17, 2017 at 10:31 PM Bastian Sebode wrote: > > > Hey Robert, > > thanks for your reply. > > Am 17.02.2017 um 19:28 schrieb Robert L Mathews: > > Looking at your dovecot -n, you're using two different files here: > > > > ssl_cert = > ssl_key = > > > Are you sure these two files match, and contain the right things in the > > right order? > > > Yes, unfortunately I'm sure that everything has the right order. As you > can see in the trace, both certificates (mine and the intermediate) get > transferred to the client on connection. > > > We use a single PEM file as input for both of these parameters, and that > > PEM file contains, in this order: > > > > -----BEGIN RSA PRIVATE KEY----- > > ... > > -----BEGIN CERTIFICATE----- > > ... > > -----BEGIN CERTIFICATE----- > > > > ... where the first BEGIN CERTIFICATE is the specific hostname one, and > > the second BEGIN CERTIFICATE is the Let's Encrypt X3 intermediate > > certificate that ends with "DNFu0Qg==". > > > Tried that, but without success. But your usage doesn't seem right to > me. The parameters are not called ssl_cert and ssl_key for nothing. ;-) > Normally you don't want your private key to have any other permissions > than 600. > > > You're also manually specifying these non-default parameters: > > > > ssl_cipher_list = ... > > ssl_prefer_server_ciphers = yes > > ssl_protocols = !SSLv2 !SSLv3 > > > > For testing, I would simplify. Does it work without any of those three > > things set? > > > Tried this before. I set all SSL specific settings exactly like my > friend where it works without a problem. But it doesn't work for me. > > Thanks anyway for your effort! > Bastian > -- > Bastian Sebode > Fachinformatiker Systemintegration > > LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig > Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de > > LINET in den sozialen Netzwerken: > www.twitter.com/linetservices | www.facebook.com/linetservices > Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ > > Gesch?ftsf?hrung: Timo Springmann, Mirko Savic und Moritz Bunkus > HR B 9170 Amtsgericht Braunschweig > > USt-IdNr. DE 259 526 516 From yacinechaouche at yahoo.com Fri Feb 17 21:38:29 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Fri, 17 Feb 2017 21:38:29 +0000 (UTC) Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: <1006495667.1617258.1487367509816@mail.yahoo.com> Seems wrong to me too, Robert. If you put your private key inside your certificate, won't it be sent to the client along with it ? Bastian, are you using an old version of thunderbird ? googling for "SSL alert number 42" gave me two results indicating a bug in thunderbird versions 31,32 and 33. You can check these links if you wish : * http://www.dovecot.org/list/dovecot/2014-July/097133.html * http://unix.stackexchange.com/questions/123367/thunderbird-fails-to-connect-to-dovecot-and-postfix -- Yassine On Friday, February 17, 2017 7:29 PM, Robert L Mathews wrote: On 2/17/17 8:58 AM, Bastian Sebode wrote: > I uploaded two Wireshark tracefiles, further logs and dovecot -n Looking at your dovecot -n, you're using two different files here: ssl_cert = References: <1006495667.1617258.1487367509816@mail.yahoo.com> Message-ID: <84454389ce257e901d6f61ccd8e4803a@valo.at> On 2017-02-17 22:38, chaouche yacine wrote: > Seems wrong to me too, Robert. If you put your private key inside your > certificate, won't it be sent to the client along with it ? This is one way of supplying cert + key to a daemon and no, the key is not sent to the client. While it is normaly true that one doesn't want the key to have access rights other than 0600, with dovecot as the file owner of the key+cert+intermediate .pem file the access rights can be set to 0600. -- Christian Kivalo From b.sebode at linet-services.de Fri Feb 17 21:57:27 2017 From: b.sebode at linet-services.de (Bastian Sebode) Date: Fri, 17 Feb 2017 22:57:27 +0100 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <67651323.902.1487365112484@appsuite-dev.open-xchange.com> References: <67651323.902.1487365112484@appsuite-dev.open-xchange.com> Message-ID: <64ead80c-795c-5674-5576-cb15de9f8ac8@linet-services.de> Hey. Thanks again for your help. I took the "dovecot -n" while the StartSSL Certificate was active, so the chain.pem was correct. Finally I found the issue! :-) But I still have no idea why the problem happens with Thunderbird. I used dehydrated to fetch the certificates from Let's Encrypt and as I said, it works for most clients pretty well. (Tried: Mulberry, Claws Mail, Outlook 2010, Android (HTC), iPhone, ...) Also it works perfectly with all my HTTPS-Services Whatever, Thunderbird didn't like that cert saying "bad certificate" (SSL Alert 42). Now I fetched the cert with Certbot and it works. Really strange though! I checked for any obvious differences between the certificates and private keys, but couldn't find any. So my solution will be to use certbot instead of dehydrated... :-/ Worst thing is, that a Microsoft Blog article (https://blogs.msdn.microsoft.com/kaushal/2012/10/05/ssltls-alert-protocol-the-alert-codes/) led me to the right direction.... ;-) -- 42 bad_certificate "There is a problem with the certificate, for example, a certificate is corrupt, or a certificate contains signatures that cannot be verified." -- Peace Bastian Am 17.02.2017 um 21:58 schrieb Aki Tuomi: > Usually with LE, the filename is fullchain.pem, not chain.pem. > > Can you please doublecheck this? > > Also, try > > openssl s_client -connect hostname:143 -starttls imap > > Aki > >> On February 17, 2017 at 10:31 PM Bastian Sebode wrote: >> >> >> Hey Robert, >> >> thanks for your reply. >> >> Am 17.02.2017 um 19:28 schrieb Robert L Mathews: >>> Looking at your dovecot -n, you're using two different files here: >>> >>> ssl_cert = >> ssl_key = >> >>> Are you sure these two files match, and contain the right things in the >>> right order? >>> >> Yes, unfortunately I'm sure that everything has the right order. As you >> can see in the trace, both certificates (mine and the intermediate) get >> transferred to the client on connection. >> >>> We use a single PEM file as input for both of these parameters, and that >>> PEM file contains, in this order: >>> >>> -----BEGIN RSA PRIVATE KEY----- >>> ... >>> -----BEGIN CERTIFICATE----- >>> ... >>> -----BEGIN CERTIFICATE----- >>> >>> ... where the first BEGIN CERTIFICATE is the specific hostname one, and >>> the second BEGIN CERTIFICATE is the Let's Encrypt X3 intermediate >>> certificate that ends with "DNFu0Qg==". >>> >> Tried that, but without success. But your usage doesn't seem right to >> me. The parameters are not called ssl_cert and ssl_key for nothing. ;-) >> Normally you don't want your private key to have any other permissions >> than 600. >> >>> You're also manually specifying these non-default parameters: >>> >>> ssl_cipher_list = ... >>> ssl_prefer_server_ciphers = yes >>> ssl_protocols = !SSLv2 !SSLv3 >>> >>> For testing, I would simplify. Does it work without any of those three >>> things set? >>> >> Tried this before. I set all SSL specific settings exactly like my >> friend where it works without a problem. But it doesn't work for me. >> >> Thanks anyway for your effort! >> Bastian >> -- >> Bastian Sebode >> Fachinformatiker Systemintegration >> >> LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig >> Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de >> >> LINET in den sozialen Netzwerken: >> www.twitter.com/linetservices | www.facebook.com/linetservices >> Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ >> >> Gesch?ftsf?hrung: Timo Springmann, Mirko Savic und Moritz Bunkus >> HR B 9170 Amtsgericht Braunschweig >> >> USt-IdNr. DE 259 526 516 -- Bastian Sebode Fachinformatiker Systemintegration LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de LINET in den sozialen Netzwerken: www.twitter.com/linetservices | www.facebook.com/linetservices Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ Gesch?ftsf?hrung: Timo Springmann, Mirko Savic und Moritz Bunkus HR B 9170 Amtsgericht Braunschweig USt-IdNr. DE 259 526 516 From elyograg at elyograg.org Fri Feb 17 22:26:38 2017 From: elyograg at elyograg.org (Shawn Heisey) Date: Fri, 17 Feb 2017 15:26:38 -0700 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1006495667.1617258.1487367509816@mail.yahoo.com> References: <1006495667.1617258.1487367509816@mail.yahoo.com> Message-ID: <6b158203-8f7a-1ad0-b419-62672278ceff@elyograg.org> On 2/17/2017 2:38 PM, chaouche yacine wrote: > Seems wrong to me too, Robert. If you put your private key inside your certificate, won't it be sent to the client along with it ? The private key should not be sent to the connecting client, even if it is contained in the same place as the certificate(s). If that data *is* sent to the client, that's a bug, probably in the SSL library (usually openssl). I am not using letsencrypt for my personal install, but my certificate provider does use one intermediate, just like letsencrypt does. I have the server certificate, the intermediate certificate, and the private key all in the same file, and my dovecot config contains these lines, both referring to that file: ssl_cert_file = /etc/ssl/certs/local/imap.REDACTED.com.pem ssl_key_file = /etc/ssl/certs/local/imap.REDACTED.com.pem This file is owned by root and has 600 permissions. Because root permissions are required in order to bind to port numbers below 1024, dovecot typically will initially start as root, then drop permissions as required. hostname:/etc/ssl/certs/local# ls -al imap.REDACTED.com.pem -rw------- 1 root root 6266 Jan 6 20:47 imap.REDACTED.com.pem Thanks, Shawn From doug at mail.sermon-archive.info Fri Feb 17 22:42:22 2017 From: doug at mail.sermon-archive.info (Doug Hardie) Date: Fri, 17 Feb 2017 14:42:22 -0800 Subject: Sieve not filtering In-Reply-To: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> References: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> Message-ID: <47B7C984-DB29-4371-899E-8D32BEF57990@mail.sermon-archive.info> > On 17 February 2017, at 08:24, Ben wrote: > > Hi, > > I have copied accross a known-good sieve file from a working server and its not filtering. Everything just gets chucked into INBOX. What I did when encountering a similar issue was to take one of the messages from INBOX that should have been moved elsewhere and use sieve-test on it: sieve-test -Tlevel=matching That generates a lot of output as it goes through every line of the sieve file and shows the actual values that are used for the tests. However, it pointed out my problem quite clearly. From yacinechaouche at yahoo.com Fri Feb 17 23:33:24 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Fri, 17 Feb 2017 23:33:24 +0000 (UTC) Subject: Problem with Let's Encrypt Certificate In-Reply-To: <6b158203-8f7a-1ad0-b419-62672278ceff@elyograg.org> References: <1006495667.1617258.1487367509816@mail.yahoo.com> <6b158203-8f7a-1ad0-b419-62672278ceff@elyograg.org> Message-ID: <673735249.9594.1487374404617@mail.yahoo.com> Interesting. Is there any particular benefit in having only one file for both certificate and private key ? I find that putting private key in a separate file feels more secure. Bastian, how could two identical certificates be processed differently by Thunderbid ? how did you check the differences between the two ? did you use "diff" ? did you compare the output of openssl x509 commands ? what method did you choose ? -- Yassine. From antoine.sirven at riseup.net Sat Feb 18 16:37:55 2017 From: antoine.sirven at riseup.net (Antoine Sirven) Date: Sat, 18 Feb 2017 17:37:55 +0100 Subject: Issue connecting to dovecot from remote machine Message-ID: Hi, I've set up a postfix +dovecot configuration on my debian jessie. But I have a connection issue. When I try to connect from thunderbird it doesn not work. When I check out my debug logs I get : auth-worker(22252): Info: pam(myuser,hostIP): pam_authenticate() failed: Authentication failure (password mismatch?) (given password: correctPassword) Running doveadm auth test tells me I can authenticate with the same password. So I tried connecting via openssl. When I connect from my local host, everything goes fine : * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. a login myUser myPassword a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in But when I run the same thing from my remote laptop, I never get any reply from the imap server. It's as if the request wasn't reaching the server. Although I do get a first reply from the server, I just can't log in. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. a login myUser myPassword (... and then nothing...) The port is properly open when I run nmap. I really don't get what's wrong. Any of you has any idea? Here are my versions & conf : # dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab120.16 x86_64 Debian 8.7 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = yes disable_plaintext_auth = no info_log_path = /var/log/maildebug.log mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = failure_show_msg=yes session=yes dovecot driver = pam } passdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } ssl = required ssl_cert = References: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> <47B7C984-DB29-4371-899E-8D32BEF57990@mail.sermon-archive.info> Message-ID: > What I did when encountering a similar issue was to take one of the messages from INBOX that should have been moved elsewhere and use sieve-test on it: > > sieve-test -Tlevel=matching > > That generates a lot of output as it goes through every line of the sieve file and shows the actual values that are used for the tests. However, it pointed out my problem quite clearly. > Thank you for this. Actually, after many hours of head-bashing, I discovered the problem. sieve doesn't work when you're just using telnet port 25 ! I was doing : ehlo test mail from:sender at example.com rcpt to:recip at example.com data Subject: hello world Hello World ! . With the above, sieve was simply sending everything to INBOX When I changed my methodology : ehlo test mail from:sender at example.com rcpt to:recip at example.com data From: To: Subject: hello world Hello World ! . It worked as expected. From ben+dovecot at list-subs.com Sat Feb 18 16:49:25 2017 From: ben+dovecot at list-subs.com (Ben) Date: Sat, 18 Feb 2017 16:49:25 +0000 Subject: doveadm: Fatal: All your namespaces have a location setting Message-ID: <5997b5c9-ab94-3259-777f-17de4ee10418@list-subs.com> Hi, I am trying to migrate mail from an old server and am receiving the following error : doveadm(user at example.com): Fatal: All your namespaces have a location setting. Only namespaces with empty location settings are converted. (One namespace should default to mail_location setting) I found an old thread (http://www.dovecot.org/list/dovecot/2012-September/068269.html) that referred to "location" being set. However this is not the case with me, my config reads: namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } My dsync conf looks as follows : imapc_host = old-server.example.com imapc_user = %u imapc_features = rfc822.size imapc_features = $imapc_features fetch-headers mail_prefetch_count = 20 imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = yes The command I'm calling is: sudo -u my_dovecot_user doveadm -c /etc/dovecot/dsync_config.conf -o mail_fsync=never backup -R -u user at example.com imapc: From kremels at kreme.com Sat Feb 18 18:01:54 2017 From: kremels at kreme.com (@lbutlr) Date: Sat, 18 Feb 2017 11:01:54 -0700 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: On 2017-02-17 (09:58 MST), Bastian Sebode wrote: > > Weirdly my friend uses the same Dovecot Version with Let's Encrypt on > his Server and it works with Thunderbird without any flaws. Mine fails > the same way in his Thunderbird and also in a fresh installation. Well, at least you?ve narrowed the fault down to Thrunderbird. Are you using TB through a proxy? Do you have a corporate LAN or an anti-virus that is behaving as a man-in-the-middle (Anything that claims to protect your web-browsing)? Have you tried from a different connection? Maybe on a different machine with ?identical? settings? Usually errors like these indicate you are not getting the secured connection you think you are. -- Apple broke AppleScripting signatures in Mail.app, so no random signatures. From kremels at kreme.com Sat Feb 18 18:06:43 2017 From: kremels at kreme.com (@lbutlr) Date: Sat, 18 Feb 2017 11:06:43 -0700 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: <5E80CBFC-3BB3-4596-AE9A-F4D4844BB51D@kreme.com> On 2017-02-17 (11:28 MST), Robert L Mathews wrote: > > ssl_cert = ssl_key = You're also manually specifying these non-default parameters: > > ssl_cipher_list = ... > ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 > > For testing, I would simplify. Does it work without any of those three > things set? ssl_protocols = !SSLv2 !SSLv3 is a sensible setting (and should be the default) a no one should still be supporting SSLv2 or SSLv3. I do not have the other settings. -- Apple broke AppleScripting signatures in Mail.app, so no random signatures. From lists at tigertech.com Sun Feb 19 06:24:14 2017 From: lists at tigertech.com (Robert L Mathews) Date: Sat, 18 Feb 2017 22:24:14 -0800 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1006495667.1617258.1487367509816@mail.yahoo.com> References: <1006495667.1617258.1487367509816@mail.yahoo.com> Message-ID: On 2/17/17 1:38 PM, chaouche yacine wrote: > Seems wrong to me too, Robert. If you put your private key inside > your certificate, won't it be sent to the client along with it ? No; any SSL software that uses the file will extract the parts it needs from it and convert them to its internal format for future use. It never literally sends the file contents anywhere. It's common and often recommended for a PEM file to contain everything needed; see, for example, the bottom section of: https://www.digicert.com/ssl-support/pem-ssl-creation.htm Doing this avoids the key and certificate files getting out of sync later. -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ From mpeters at domblogger.net Sun Feb 19 07:00:17 2017 From: mpeters at domblogger.net (Michael A. Peters) Date: Sat, 18 Feb 2017 23:00:17 -0800 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: <1006495667.1617258.1487367509816@mail.yahoo.com> Message-ID: <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> On 02/18/2017 10:24 PM, Robert L Mathews wrote: > On 2/17/17 1:38 PM, chaouche yacine wrote: > >> Seems wrong to me too, Robert. If you put your private key inside >> your certificate, won't it be sent to the client along with it ? > > No; any SSL software that uses the file will extract the parts it needs > from it and convert them to its internal format for future use. It never > literally sends the file contents anywhere. > > It's common and often recommended for a PEM file to contain everything > needed; see, for example, the bottom section of: > > https://www.digicert.com/ssl-support/pem-ssl-creation.htm > > Doing this avoids the key and certificate files getting out of sync later. > I don't use Let's Encrypt but to avoid them getting out of sync, I simply put a time stamp in the filename, e.g. /etc/pki/tls/private/deviant.email-20160427.key /etc/pki/tls/certs/deviant.email-20160427.crt I never re-use a private key, when a cert expires I always generate a new private key with a new CSR. That's one of the reasons I don't like Let's Encrypt, with one year certs it is easier to look at the certs and see what is going to expire in the coming month needing a new private key. Let's Encrypt does 3 month certs and re-uses the private key when it generates a new cert. I'm sure it probably could be scripted to use a new private key every time but then I have to have to update the TLSA record frequently (and you have to have the new fingerprint TLSA record in DNS before you start using it) and that would be a hassle. I'm sure it probably could also be scripted to use a new private key every fourth time, too. But for me its just easier to have certs that last a year and I can easily visually see what is going to need my action. From kevin at my.walr.us Sun Feb 19 13:39:43 2017 From: kevin at my.walr.us (KT Walrus) Date: Sun, 19 Feb 2017 08:39:43 -0500 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> Message-ID: > That's one of the reasons I don't like Let's Encrypt, with one year certs it is easier to look at the certs and see what is going to expire in the coming month needing a new private key. I use dehydrated (with Cloudflare DNS challenges) and as far as I know, it seems to generate a new private key every time. All newly generated certs are generated with the timestamp in the filenames and the soft links updated to point to the latest timestamped files. I have 4 domains each with an average of 70 alt names, so Let?s Encrypt is saving me money. I simply run the dehydrated script every week in a cron job to regenerate the certs (if there is less than 30 days until the current cert is set to expire) and rotate in any new certs. Of course, I run my sites using Docker and it is very easy to automate renewing certs. Note that I had the dehydrated script fail occasionally (mostly with 500 Server Busy errors for the Let?s Encrypt ACME server that sometimes cause me to have to wait a week before the script will succeed). Automating cert renewal and cert rotation into production using Let?s Encrypt and Docker is a huge win for me, and has taken the pain out of manually doing this once a year for each domain (and paying high fees for the privilege). And using the DNS-01 challenge type means that I can easily generate certs for my mail domain (that doesn?t have a web server). In fact, using Cloudflare DNS is free so even DNS for my mail domain doesn?t cost anything. Kevin > On Feb 19, 2017, at 2:00 AM, Michael A. Peters wrote: > > On 02/18/2017 10:24 PM, Robert L Mathews wrote: >> On 2/17/17 1:38 PM, chaouche yacine wrote: >> >>> Seems wrong to me too, Robert. If you put your private key inside >>> your certificate, won't it be sent to the client along with it ? >> >> No; any SSL software that uses the file will extract the parts it needs >> from it and convert them to its internal format for future use. It never >> literally sends the file contents anywhere. >> >> It's common and often recommended for a PEM file to contain everything >> needed; see, for example, the bottom section of: >> >> https://www.digicert.com/ssl-support/pem-ssl-creation.htm >> >> Doing this avoids the key and certificate files getting out of sync later. >> > > I don't use Let's Encrypt but to avoid them getting out of sync, I simply put a time stamp in the filename, e.g. > > /etc/pki/tls/private/deviant.email-20160427.key > /etc/pki/tls/certs/deviant.email-20160427.crt > > I never re-use a private key, when a cert expires I always generate a new private key with a new CSR. > > That's one of the reasons I don't like Let's Encrypt, with one year certs it is easier to look at the certs and see what is going to expire in the coming month needing a new private key. > > Let's Encrypt does 3 month certs and re-uses the private key when it generates a new cert. > > I'm sure it probably could be scripted to use a new private key every time but then I have to have to update the TLSA record frequently (and you have to have the new fingerprint TLSA record in DNS before you start using it) and that would be a hassle. > > I'm sure it probably could also be scripted to use a new private key every fourth time, too. > > But for me its just easier to have certs that last a year and I can easily visually see what is going to need my action. From mpeters at domblogger.net Mon Feb 20 01:39:14 2017 From: mpeters at domblogger.net (Michael A. Peters) Date: Sun, 19 Feb 2017 17:39:14 -0800 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> Message-ID: <6caf1f01-d2a2-139b-59f9-083b89d7d87d@domblogger.net> On 02/19/2017 05:39 AM, KT Walrus wrote: >> That's one of the reasons I don't like Let's Encrypt, with one year certs it is easier to look at the certs and see what is going to expire in the coming month needing a new private key. > > I use dehydrated (with Cloudflare DNS challenges) and as far as I know, it seems to generate a new private key every time. Yeah that would be a problem for me because I implement DANE. Every time I change the private key - A) I have to make a TLSA record for the new key B) I have to let that key propagate in DNS while the old cert is active. I use 8 hour TTL for DNS records, so that takes 16 hours (twice the TTL) C) Then I can switch to the new key / cert in the server. I use TLSA records for everything TLS, even dovecot - despite the fact I am not aware of any IMAP clients that will validate via DANE - because it is the right thing to do and sooner or later IMAP clients will support DNSSEC and DANE. Having to do that every three months for every service I run, I really do not see what real world benefit I or my users would gain. From gedalya at gedalya.net Mon Feb 20 01:55:45 2017 From: gedalya at gedalya.net (Gedalya) Date: Sun, 19 Feb 2017 20:55:45 -0500 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <6caf1f01-d2a2-139b-59f9-083b89d7d87d@domblogger.net> References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> <6caf1f01-d2a2-139b-59f9-083b89d7d87d@domblogger.net> Message-ID: <8882a929-fc69-ea80-add6-de6fff713805@gedalya.net> On 02/19/2017 08:39 PM, Michael A. Peters wrote: > Every time I change the private key - > > A) I have to make a TLSA record for the new key You're actually expected to pin the CA in your TLSA record, not your own key. https://community.letsencrypt.org/t/please-avoid-3-0-1-and-3-0-2-dane-tlsa-records-with-le-certificates/7022 http://www.internetsociety.org/deploy360/blog/2016/01/lets-encrypt-certificates-for-mail-servers-and-dane-part-1-of-2/ https://www.internetsociety.org/deploy360/blog/2016/03/lets-encrypt-certificates-for-mail-servers-and-dane-part-2-of-2/ I had the privilege of being auto-yelled at by Viktor Dukhovni over forgetting to adjust my TLSA after changing certificates for SMTP. I would however prefer to automate the process of pushing new TLSA records, waiting out twice the TTL and then pushing the certificate. Going through this every time would ensure I have valid records every time, without having to worry about the CA key changing. This is on my to-do list, for SMTP, XMPP, IMAP etc. From aki.tuomi at dovecot.fi Mon Feb 20 06:57:07 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Mon, 20 Feb 2017 08:57:07 +0200 Subject: doveadm: Fatal: All your namespaces have a location setting In-Reply-To: <5997b5c9-ab94-3259-777f-17de4ee10418@list-subs.com> References: <5997b5c9-ab94-3259-777f-17de4ee10418@list-subs.com> Message-ID: <7dd759bc-444a-8a62-2238-221a3e210a48@dovecot.fi> On 18.02.2017 18:49, Ben wrote: > Hi, > > I am trying to migrate mail from an old server and am receiving the > following error : > > doveadm(user at example.com): Fatal: All your namespaces have a location > setting. Only namespaces with empty location settings are converted. > (One namespace should default to mail_location setting) > > I found an old thread > (http://www.dovecot.org/list/dovecot/2012-September/068269.html) that > referred to "location" being set. > > However this is not the case with me, my config reads: > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > > My dsync conf looks as follows : > imapc_host = old-server.example.com > imapc_user = %u > imapc_features = rfc822.size > imapc_features = $imapc_features fetch-headers > mail_prefetch_count = 20 > imapc_port = 993 > imapc_ssl = imaps > imapc_ssl_verify = yes > > The command I'm calling is: > > sudo -u my_dovecot_user doveadm -c /etc/dovecot/dsync_config.conf -o > mail_fsync=never backup -R -u user at example.com imapc: Hi! Can you post doveconf -n Aki From skdovecot at smail.inf.fh-brs.de Mon Feb 20 08:32:26 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 20 Feb 2017 09:32:26 +0100 (CET) Subject: Issue connecting to dovecot from remote machine In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sat, 18 Feb 2017, Antoine Sirven wrote: > I've set up a postfix +dovecot configuration on my debian jessie. > But I have a connection issue. When I try to connect from thunderbird it > doesn not work. When I check out my debug logs I get : > auth-worker(22252): Info: pam(myuser,hostIP): pam_authenticate() > failed: Authentication failure (password mismatch?) (given password: > correctPassword) > > Running doveadm auth test tells me I can authenticate with the same > password. > So I tried connecting via openssl. > When I connect from my local host, everything goes fine : > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. > a login myUser myPassword > a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS > THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN > NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH > ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY > MOVE] Logged in > > > But when I run the same thing from my remote laptop, I never get any > reply from the imap server. It's as if the request wasn't reaching the > server. Although I do get a first reply from the server, I just can't > log in. > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. > a login myUser myPassword > > (... and then nothing...) > I would: 1) sniff the connection on the mail server, if there is traffic after sending the login 2) test if you can log in locally while the remote connection hangs 3) does PAM performs some IP checks? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWKqpmnz1H7kL/d9rAQLMCAf8CWN6vjosNYc9YjF7gGtXTLTUjqLon0an fIO/ia+esglLuKLzu1VQ2cBSDcHgUhXscrMVwwf8SZxpK/xGL9FrwYEQf48dGym2 hp/csY5GjR2erDeKJYKQHkcgEpKhK1pveNIbVs5Z50mjHwbAuKIm92uNlxi8SJ08 DI5fNynUwisfcugAyaD3rcmylkDNSwRFaJMqyGGsIlSGBhy/e6rwxGZqpVsYoCk2 Tx6wHWRcg1p48EzqQbH+qpImRJiQhmsvrSDbqAcj0JDQr6ZvNw6ZyCmjniHZrgt3 GtSwupY8Wp3q0RXV+B2D6D12AjJC+vBfDJalS7Bu0cLDIVdSyv/RRQ== =Hw5M -----END PGP SIGNATURE----- From antoine.sirven at riseup.net Mon Feb 20 08:34:25 2017 From: antoine.sirven at riseup.net (Antoine Sirven) Date: Mon, 20 Feb 2017 09:34:25 +0100 Subject: Issue connecting to dovecot from remote machine In-Reply-To: References: Message-ID: <7e1a8d20-8ff3-6060-4e4b-aca2744b8705@riseup.net> Thanks for your reply Steffen. Well I basically uninstalled everything and started from scratch. it now works just fine. I guess I messed something up along the way the first time. Thanks Antoine Le 20/02/2017 ? 09:32, Steffen Kaiser a ?crit : > On Sat, 18 Feb 2017, Antoine Sirven wrote: > > > I've set up a postfix +dovecot configuration on my debian jessie. > > But I have a connection issue. When I try to connect from thunderbird it > > doesn not work. When I check out my debug logs I get : > > auth-worker(22252): Info: pam(myuser,hostIP): pam_authenticate() > > failed: Authentication failure (password mismatch?) (given password: > > correctPassword) > > > Running doveadm auth test tells me I can authenticate with the same > > password. > > So I tried connecting via openssl. > > When I connect from my local host, everything goes fine : > > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > > IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. > > a login myUser myPassword > > a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS > > THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN > > NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH > > ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY > > MOVE] Logged in > > > > But when I run the same thing from my remote laptop, I never get any > > reply from the imap server. It's as if the request wasn't reaching the > > server. Although I do get a first reply from the server, I just can't > > log in. > > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > > IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. > > a login myUser myPassword > > > (... and then nothing...) > > > I would: > > 1) sniff the connection on the mail server, if there is traffic after > sending the login > > 2) test if you can log in locally while the remote connection hangs > > 3) does PAM performs some IP checks? > > -- Steffen Kaiser From yacinechaouche at yahoo.com Mon Feb 20 09:32:32 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Mon, 20 Feb 2017 09:32:32 +0000 (UTC) Subject: Problem with Let's Encrypt Certificate In-Reply-To: <6caf1f01-d2a2-139b-59f9-083b89d7d87d@domblogger.net> References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> <6caf1f01-d2a2-139b-59f9-083b89d7d87d@domblogger.net> Message-ID: <1889272825.902200.1487583152217@mail.yahoo.com> What is the motivation behind using a new pair of keys and CSR ? From tss at iki.fi Mon Feb 20 10:09:48 2017 From: tss at iki.fi (Timo Sirainen) Date: Mon, 20 Feb 2017 12:09:48 +0200 Subject: Replication Troubles In-Reply-To: <571885AA-C2DA-4240-A0B2-F0681D347DE4@wogri.com> References: <571885AA-C2DA-4240-A0B2-F0681D347DE4@wogri.com> Message-ID: <95751283-49CE-4A3C-A1BF-0AC8AAEED08A@iki.fi> On 17 Feb 2017, at 21.37, Wolfgang Hennerbichler wrote: > > Hi Dovecot Users, > > I?ve configured dovecot dsync replication and I see troubles in the logs and get user complaints which I can?t explain. I found similar threads on this mailinglist, but I couldn?t find a solution anywhere. Does anybody have dsync running without problems on a high volume mailserver? > > I see the following logs, examples given: > > Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: /var/mail/zoechi/dovecot-uidlist: Duplicate file entry at line 10395: 1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377 (uid 41092 -> 41093) - retrying by re-reading from beginning > > with this one I?m not sure - it might be that this is completely OK because due to replication UIDs clash. Maybe that?s OK, but I couldn?t find a confirmation. > > Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: Maildir /var/mail/zoechi: Expunged message reappeared, giving a new UID (old uid=41092, file=1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377) There seems to be something weird with using Maildir and replication. Haven't had time to debug it and it's likely not an easy bug to fix, so for now the solution would be to use only sdbox/mdbox with replication. From tss at iki.fi Mon Feb 20 10:15:34 2017 From: tss at iki.fi (Timo Sirainen) Date: Mon, 20 Feb 2017 12:15:34 +0200 Subject: v2.2.28 release candidate released Message-ID: http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz.sig Pretty large release. Please test before the final v2.2.28, which should be out in a few days. BTW. Our plan is to start making new releases approximately every month from now on. * director: "doveadm director move" to same host now refreshes user's timeout. This allows keeping user constantly in the same backend by just periodically moving the user there. * When new mailbox is created, use initially INBOX's dovecot.index.cache caching decisions. * Expunging mails writes GUID to dovecot.index.log now only if the GUID is quickly available from index/cache. * imap: When BINARY FETCH sees invalid content, return NO [PARSE] reply instead of [UNKNOWNCTE] (which is now used only for actually unknown Content-Transfer-Encoding headers). * pop3c: Increase timeout for PASS command to 5 minutes. * Mail access errors are no longer ignored when searching or sorting. With IMAP the untagged SEARCH/SORT reply is still sent the same as before, but NO reply is returned instead of OK. + Make dovecot.list.index's filename configurable. This is needed when there are multiple namespaces pointing to the same mail root (e.g. lazy_expunge namespace for mdbox). + Add size.virtual to dovecot.index when folder vsizes are accessed (e.g. quota=count). This is mainly a workaround to avoid slow quota recalculation performance when message sizes get lost from dovecot.index.cache due to corruption or some other reason. + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them in lib-dsasl for client side. + auth: Support filtering by SASL mechanism: passdb { mechanisms } + Shrink the mail processes' memory usage by not storing settings duplicated unnecessarily many times. + imap: Add imap_fetch_failure setting to control what happens when FETCH fails for some mails (see example-config). + imap: Include info about last command in disconnection log line. + imap: Created new SEARCH=X-MIMEPART extension. It's currently not advertised by default, since it's not fully implemented. + fts-solr: Add support for basic authentication. + Cassandra: Support automatically retrying failed queries if execution_retry_interval and execution_retry_times are set. + doveadm: Added "mailbox path" command. + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the wanted fields only if it doesn't require opening the email. + mail_vsize_bg_after_count setting added (see example-config). + mail_sort_max_read_count setting added (see example-config). - Index files: day_first_uid wasn't updated correctly since v2.2.26. This caused dovecot.index.cache to be non-optimal. - imap: SEARCH/SORT may have assert-crashed in client_check_command_hangs - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. - imap: Running time in tagged command reply was often wrongly 0. - search: Using NOT n:* or NOT UID n:* wasn't handled correctly - director: doveadm director kick was broken - director: Fix crash when using director_flush_socket - director: Fix some bugs when moving users between backends - imapc: Various error handling fixes and improvements - master: doveadm process status output had a lot of duplicates. - autoexpunge: If mailbox's rename timestamp is newer than mail's save-timestamp, use it instead. This is useful when autoexpunging e.g. Trash/* and an entire mailbox is deleted by renaming it under Trash to prevent it from being autoexpunged too early. - autoexpunge: Multiple processes may have been trying to expunge the same mails simultaneously. This was problematic especially with lazy_expunge plugin. - auth: %{passdb:*} was empty in auth-worker processes - auth-policy: hashed_password was always sent empty. - dict-sql: Merge multiple UPDATEs to a single statement if possible. - fts-solr: Escape {} chars when sending queries - fts: fts_autoexpunge_exclude = \Special-use caused crashes - doveadm-server: Fix leaks and other problems when process is reused for multiple requests (service_count != 1) - sdbox: Fix assert-crash on mailbox create race - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve was used. especially %{storage_id} was broken. From stu at spacehopper.org Mon Feb 20 11:40:52 2017 From: stu at spacehopper.org (Stuart Henderson) Date: Mon, 20 Feb 2017 11:40:52 +0000 (UTC) Subject: Problem with Let's Encrypt Certificate References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> Message-ID: On 2017-02-19, KT Walrus wrote: >> That's one of the reasons I don't like Let's Encrypt, with one year >> certs it is easier to look at the certs and see what is going to expire >> in the coming month needing a new private key. > > I use dehydrated (with Cloudflare DNS challenges) and as far as I > know, it seems to generate a new private key every time. This is client-dependent, the CA doesn't care either way. From odhiambo at gmail.com Mon Feb 20 11:43:18 2017 From: odhiambo at gmail.com (Odhiambo Washington) Date: Mon, 20 Feb 2017 14:43:18 +0300 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: On 20 February 2017 at 13:15, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz.sig > > Pretty large release. Please test before the final v2.2.28, which should > be out in a few days. > > BTW. Our plan is to start making new releases approximately every month > from now on. > So by December we'll have had at least 10 new releases? Why do many though? Just curious. I am running this on two servers already - and monitoring... -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft." From stu at spacehopper.org Mon Feb 20 11:46:01 2017 From: stu at spacehopper.org (Stuart Henderson) Date: Mon, 20 Feb 2017 11:46:01 +0000 (UTC) Subject: Sieve not filtering References: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> <47B7C984-DB29-4371-899E-8D32BEF57990@mail.sermon-archive.info> Message-ID: On 2017-02-18, Ben wrote: > >> What I did when encountering a similar issue was to take one of the messages from INBOX that should have been moved elsewhere and use sieve-test on it: >> >> sieve-test -Tlevel=matching >> >> That generates a lot of output as it goes through every line of the sieve file and shows the actual values that are used for the tests. However, it pointed out my problem quite clearly. >> > > Thank you for this. > > Actually, after many hours of head-bashing, I discovered the problem. > > sieve doesn't work when you're just using telnet port 25 ! > > I was doing : > ehlo test > mail from:sender at example.com > rcpt to:recip at example.com > data > Subject: hello world > Hello World ! > . > > With the above, sieve was simply sending everything to INBOX > > When I changed my methodology : > ehlo test > mail from:sender at example.com > rcpt to:recip at example.com > data > From: > To: > Subject: hello world > Hello World ! > . > > It worked as expected. > The first one works as expected too; your rule used "address" so it is correct that it didn't look at the envelope address. You want e.g. envelope "to" "foo at example.org" From toni at solu.fi Mon Feb 20 11:47:06 2017 From: toni at solu.fi (Toni Mattila) Date: Mon, 20 Feb 2017 13:47:06 +0200 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: <7d1d4aba-3862-37e3-01ba-948207fd9d69@solu.fi> Hi, On 20-Feb-17 12:15, Timo Sirainen wrote: > imap: When BINARY FETCH sees invalid content, return NO [PARSE] reply > instead of [UNKNOWNCTE] (which is now used only for actually unknown > Content-Transfer-Encoding headers). Has this been tested with Roundcube webmail? I know Roundcube has some workarounds when dovecot now responds with that "[UNKNOWNCTE]". Best regards, Toni From tlx at leuxner.net Mon Feb 20 11:56:44 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 20 Feb 2017 12:56:44 +0100 Subject: Sieve removeflag Action In-Reply-To: <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> References: <20170113081514.GA60507@nihlus.leuxner.net> <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> Message-ID: <20170220115644.GA14379@nihlus.leuxner.net> * Stephan Bosch 2017.02.17 14:34: > Couldn't reproduce this with v2.3.devel yesterday (i.e. no flags set for the > Security mailbox and all flags set for the message in INBOX), but I will try > later with some older version. I did install a newer build this morning: Feb 20 07:00:23 nihlus dovecot: master: Dovecot v2.2.devel (8f42a89) starting up for imap, lmtp This one processed the dovecot-news mail for 2.2.28.rc1 fine which uses a similar sieve rule. I will monitor global rules with this build and report back. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From tss at iki.fi Mon Feb 20 12:01:19 2017 From: tss at iki.fi (Timo Sirainen) Date: Mon, 20 Feb 2017 14:01:19 +0200 Subject: v2.2.28 release candidate released In-Reply-To: <7d1d4aba-3862-37e3-01ba-948207fd9d69@solu.fi> References: <7d1d4aba-3862-37e3-01ba-948207fd9d69@solu.fi> Message-ID: <4346D70B-8C78-4183-8F76-CEBE90E211A4@iki.fi> On 20 Feb 2017, at 13.47, Toni Mattila wrote: > > Hi, > > On 20-Feb-17 12:15, Timo Sirainen wrote: >> imap: When BINARY FETCH sees invalid content, return NO [PARSE] reply >> instead of [UNKNOWNCTE] (which is now used only for actually unknown >> Content-Transfer-Encoding headers). > > Has this been tested with Roundcube webmail? I know Roundcube has some workarounds when dovecot now responds with that "[UNKNOWNCTE]". No. Hmm. This was one of the changes that I was wondering whether it should be in v2.2 or if it should be left to v2.3. Actually looks like the change was done already last May, so I must have decided earlier not to put it to v2.2, but now semi-accidentally added it. I'll remove it from the final v2.2.28, but I suppose should tell Roundcube to start preparing for this change in v2.3. From alec at alec.pl Mon Feb 20 12:06:15 2017 From: alec at alec.pl (A.L.E.C) Date: Mon, 20 Feb 2017 13:06:15 +0100 Subject: v2.2.28 release candidate released In-Reply-To: <7d1d4aba-3862-37e3-01ba-948207fd9d69@solu.fi> References: <7d1d4aba-3862-37e3-01ba-948207fd9d69@solu.fi> Message-ID: On 20.02.2017 12:47, Toni Mattila wrote: >> imap: When BINARY FETCH sees invalid content, return NO [PARSE] reply >> instead of [UNKNOWNCTE] (which is now used only for actually unknown >> Content-Transfer-Encoding headers). > > Has this been tested with Roundcube webmail? I know Roundcube has some > workarounds when dovecot now responds with that "[UNKNOWNCTE]". I suppose we'll have to fix the Roundcube code where we do: // handle UNKNOWN-CTE response - RFC 3516, try again with standard BODY request if ($binary && !$found && preg_match('/^' . $key . ' NO \[UNKNOWN-CTE\]/i', $line)) { Create a ticket for this, please. In RFC3501 [PARSE] is described as "The human-readable text represents an error in parsing the [RFC-2822] header or [MIME-IMB] headers of a message in the mailbox.". So, is this really an appropriate error code for this case? -- Aleksander 'A.L.E.C' Machniak Kolab Groupware Developer [http://kolab.org] Roundcube Webmail Developer [http://roundcube.net] ---------------------------------------------------- PGP: 19359DC1 # Blog: https://kolabian.wordpress.com From lista at xdrv.co.uk Mon Feb 20 12:06:37 2017 From: lista at xdrv.co.uk (James) Date: Mon, 20 Feb 2017 12:06:37 +0000 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: <7f741908-f666-59ed-a117-e2e1ec5d0b17@xdrv.co.uk> On 20/02/2017 10:15, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz I had a minor problem on build. I have configured without compression, [ZFS is providing it]: ./configure \ ... \ --with-bzlib=no \ --with-zlib=no \ --with-lzma=no \ --with-lz4=no \ src/lib-compression/istream-zlib.c uses "#ifdef HAVE_ZLIB" to prevent compiling the compression methods but test-compression.c tries to use these: Undefined first referenced symbol in file i_stream_create_gz test-compression.o ld: fatal: symbol referencing errors. No output written to test-compression I avoided the problem by hacking out compression-test from the makefile src/lib-compression/Makefile.in Otherwise rc1 is working with Solaris 10 / Sun Studio 12.5 - thank you. From tss at iki.fi Mon Feb 20 12:17:09 2017 From: tss at iki.fi (Timo Sirainen) Date: Mon, 20 Feb 2017 14:17:09 +0200 Subject: v2.2.28 release candidate released In-Reply-To: References: <7d1d4aba-3862-37e3-01ba-948207fd9d69@solu.fi> Message-ID: <07729891-1DD6-4CBD-8CEE-827CD6D65F7E@iki.fi> On 20 Feb 2017, at 14.06, A.L.E.C wrote: > > In RFC3501 [PARSE] is described as "The human-readable text represents > an error in parsing the [RFC-2822] header or [MIME-IMB] headers of a > message in the mailbox.". So, is this really an appropriate error code > for this case? I think it's the closest to being correct. It's saying that there was some kind of a parsing problem with the email. RFC 3501 only talks about headers, but that's probably because they hadn't really considered parsing errors in email bodies. The IMAP BINARY RFC only talks about [UNKNOWN-CTE], but it says it should be sent on unknown Content-Transfer-Encoding. So reusing it for other purposes seems more wrong than using [PARSE]. This was discussed some years ago in imap mailing list: http://mailman13.u.washington.edu/pipermail/imap-protocol/2012-December/001914.html http://mailman13.u.washington.edu/pipermail/imap-protocol/2012-December/001925.html From tss at iki.fi Mon Feb 20 12:23:23 2017 From: tss at iki.fi (Timo Sirainen) Date: Mon, 20 Feb 2017 14:23:23 +0200 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: On 20 Feb 2017, at 13.43, Odhiambo Washington wrote: > >> BTW. Our plan is to start making new releases approximately every month >> from now on. >> > > So by December we'll have had at least 10 new releases? Maybe. Lets see :) > Why do many though? Just curious. There's a lot of work to do and doesn't make sense to do as huge releases as they now seem to be. BTW. Related to this: We're now looking for someone to do even more coding work for us, ideally to be located in Finland: https://www.linkedin.com/jobs/view/275257998/ From stephan at rename-it.nl Mon Feb 20 12:38:49 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 20 Feb 2017 13:38:49 +0100 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: Pigeonhole will follow later today. Op 20-2-2017 om 11:15 schreef Timo Sirainen: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz.sig > > Pretty large release. Please test before the final v2.2.28, which should be out in a few days. > > BTW. Our plan is to start making new releases approximately every month from now on. > > * director: "doveadm director move" to same host now refreshes user's > timeout. This allows keeping user constantly in the same backend by > just periodically moving the user there. > * When new mailbox is created, use initially INBOX's > dovecot.index.cache caching decisions. > * Expunging mails writes GUID to dovecot.index.log now only if the > GUID is quickly available from index/cache. > * imap: When BINARY FETCH sees invalid content, return NO [PARSE] reply > instead of [UNKNOWNCTE] (which is now used only for actually unknown > Content-Transfer-Encoding headers). > * pop3c: Increase timeout for PASS command to 5 minutes. > * Mail access errors are no longer ignored when searching or sorting. > With IMAP the untagged SEARCH/SORT reply is still sent the same as > before, but NO reply is returned instead of OK. > > + Make dovecot.list.index's filename configurable. This is needed when > there are multiple namespaces pointing to the same mail root > (e.g. lazy_expunge namespace for mdbox). > + Add size.virtual to dovecot.index when folder vsizes are accessed > (e.g. quota=count). This is mainly a workaround to avoid slow quota > recalculation performance when message sizes get lost from > dovecot.index.cache due to corruption or some other reason. > + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them > in lib-dsasl for client side. > + auth: Support filtering by SASL mechanism: passdb { mechanisms } > + Shrink the mail processes' memory usage by not storing settings > duplicated unnecessarily many times. > + imap: Add imap_fetch_failure setting to control what happens when > FETCH fails for some mails (see example-config). > + imap: Include info about last command in disconnection log line. > + imap: Created new SEARCH=X-MIMEPART extension. It's currently not > advertised by default, since it's not fully implemented. > + fts-solr: Add support for basic authentication. > + Cassandra: Support automatically retrying failed queries if > execution_retry_interval and execution_retry_times are set. > + doveadm: Added "mailbox path" command. > + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the > wanted fields only if it doesn't require opening the email. > + mail_vsize_bg_after_count setting added (see example-config). > + mail_sort_max_read_count setting added (see example-config). > > - Index files: day_first_uid wasn't updated correctly since v2.2.26. > This caused dovecot.index.cache to be non-optimal. > - imap: SEARCH/SORT may have assert-crashed in > client_check_command_hangs > - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. > - imap: Running time in tagged command reply was often wrongly 0. > - search: Using NOT n:* or NOT UID n:* wasn't handled correctly > - director: doveadm director kick was broken > - director: Fix crash when using director_flush_socket > - director: Fix some bugs when moving users between backends > - imapc: Various error handling fixes and improvements > - master: doveadm process status output had a lot of duplicates. > - autoexpunge: If mailbox's rename timestamp is newer than mail's > save-timestamp, use it instead. This is useful when autoexpunging > e.g. Trash/* and an entire mailbox is deleted by renaming it under > Trash to prevent it from being autoexpunged too early. > - autoexpunge: Multiple processes may have been trying to expunge the > same mails simultaneously. This was problematic especially with > lazy_expunge plugin. > - auth: %{passdb:*} was empty in auth-worker processes > - auth-policy: hashed_password was always sent empty. > - dict-sql: Merge multiple UPDATEs to a single statement if possible. > - fts-solr: Escape {} chars when sending queries > - fts: fts_autoexpunge_exclude = \Special-use caused crashes > - doveadm-server: Fix leaks and other problems when process is reused > for multiple requests (service_count != 1) > - sdbox: Fix assert-crash on mailbox create race > - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve > was used. especially %{storage_id} was broken. From basti at unix-solution.de Fri Feb 17 17:39:42 2017 From: basti at unix-solution.de (basti) Date: Fri, 17 Feb 2017 18:39:42 +0100 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> Hello, I had the same problem. LE is not in the CA list. Best Regards, On 17.02.2017 17:58, Bastian Sebode wrote: > Hello Folks, > > my StartCom SSL-Certificate expires soon and so I wanted to switch to > Let's Encrypt Certificates instead. Unfortunatelly Thunderbird seems not > to like it, although all -tested- other Clients work without any problems. > > When I connect with Thunderbird it sends an "Encrypted Alert" directly > after the TLS handshake although Dovecot wants to continue the session. > > In the Dovecot Log it says: > Feb 17 17:27:17 imap-login: Debug: SSL: where=0x20, ret=1: SSL > negotiation finished successfully [82.100.242.26] > Feb 17 17:27:17 imap-login: Debug: SSL: where=0x2002, ret=1: SSL > negotiation finished successfully [82.100.242.26] > Feb 17 17:27:17 imap-login: Warning: SSL alert: where=0x4004, ret=554: > fatal bad certificate [82.100.242.26] > > But the certificate is okay, cause it works with other Mailclients and > openssl also says so. What certificate is Thunderbird complaining about? > > Thunderbird says something like "There's no supported authentication > method". I don't use any Certificates for Client Authentication, neither > in Dovecot nor in Thunderbird. When I do, it fails the same way. > > Weirdly my friend uses the same Dovecot Version with Let's Encrypt on > his Server and it works with Thunderbird without any flaws. Mine fails > the same way in his Thunderbird and also in a fresh installation. > > After two weeks of investigating I still have no clue why it behaves > like this. > > I uploaded two Wireshark tracefiles, further logs and dovecot -n, may be > someone sees any possible reasons for this weird behavior or has any > further tips on solving this issue. > https://sebode-online.de/dovecot-letsencrypt/ > > Every hint is highly appreciated! > > Best Regards > Bastian > From bp at list-subs.com Mon Feb 20 09:46:40 2017 From: bp at list-subs.com (Ben) Date: Mon, 20 Feb 2017 09:46:40 +0000 Subject: doveadm: Fatal: All your namespaces have a location setting In-Reply-To: <7dd759bc-444a-8a62-2238-221a3e210a48@dovecot.fi> References: <5997b5c9-ab94-3259-777f-17de4ee10418@list-subs.com> <7dd759bc-444a-8a62-2238-221a3e210a48@dovecot.fi> Message-ID: <189e1af0-8399-9078-c48b-9395fea136a3@list-subs.com> > Hi! > > Can you post doveconf -n > > Aki # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.6.1.el7.x86_64 x86_64 CentOS Linux release 7.3.1611 (Core) auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1 first_valid_uid = 1000 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body environment mailbox date ihave enotify mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = ~/.dovecot.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = its_virtmail mode = 0660 user = its_virtmail } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = its_virtmail } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieves { address = port = 5190 ssl = yes } } ssl = required ssl_cert = was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 20 } From bp at list-subs.com Mon Feb 20 12:02:26 2017 From: bp at list-subs.com (Ben) Date: Mon, 20 Feb 2017 12:02:26 +0000 Subject: Sieve not filtering In-Reply-To: References: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> <47B7C984-DB29-4371-899E-8D32BEF57990@mail.sermon-archive.info> Message-ID: <22f19afd-2481-f765-6f9d-2191adc87779@list-subs.com> >> Thank you for this. >> >> Actually, after many hours of head-bashing, I discovered the problem. >> >> sieve doesn't work when you're just using telnet port 25 ! >> >> I was doing : >> ehlo test >> mail from:sender at example.com >> rcpt to:recip at example.com >> data >> Subject: hello world >> Hello World ! >> . >> >> With the above, sieve was simply sending everything to INBOX >> >> When I changed my methodology : >> ehlo test >> mail from:sender at example.com >> rcpt to:recip at example.com >> data >> From: >> To: >> Subject: hello world >> Hello World ! >> . >> >> It worked as expected. >> > The first one works as expected too; your rule used "address" so it > is correct that it didn't look at the envelope address. You want e.g. > > envelope "to" "foo at example.org" Will take a look. Thanks ! From yacinechaouche at yahoo.com Mon Feb 20 14:43:31 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Mon, 20 Feb 2017 14:43:31 +0000 (UTC) Subject: Problem with Let's Encrypt Certificate In-Reply-To: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> References: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> Message-ID: <1806248583.1038101.1487601811691@mail.yahoo.com> Hello Basti. Maybe you tried LE too early when it was not universally accepted as a trusted CA ? On Monday, February 20, 2017 2:22 PM, basti wrote: Hello, I had the same problem. LE is not in the CA list. Best Regards, On 17.02.2017 17:58, Bastian Sebode wrote: > Hello Folks, > > my StartCom SSL-Certificate expires soon and so I wanted to switch to > Let's Encrypt Certificates instead. Unfortunatelly Thunderbird seems not > to like it, although all -tested- other Clients work without any problems. > > When I connect with Thunderbird it sends an "Encrypted Alert" directly > after the TLS handshake although Dovecot wants to continue the session. > > In the Dovecot Log it says: > Feb 17 17:27:17 imap-login: Debug: SSL: where=0x20, ret=1: SSL > negotiation finished successfully [82.100.242.26] > Feb 17 17:27:17 imap-login: Debug: SSL: where=0x2002, ret=1: SSL > negotiation finished successfully [82.100.242.26] > Feb 17 17:27:17 imap-login: Warning: SSL alert: where=0x4004, ret=554: > fatal bad certificate [82.100.242.26] > > But the certificate is okay, cause it works with other Mailclients and > openssl also says so. What certificate is Thunderbird complaining about? > > Thunderbird says something like "There's no supported authentication > method". I don't use any Certificates for Client Authentication, neither > in Dovecot nor in Thunderbird. When I do, it fails the same way. > > Weirdly my friend uses the same Dovecot Version with Let's Encrypt on > his Server and it works with Thunderbird without any flaws. Mine fails > the same way in his Thunderbird and also in a fresh installation. > > After two weeks of investigating I still have no clue why it behaves > like this. > > I uploaded two Wireshark tracefiles, further logs and dovecot -n, may be > someone sees any possible reasons for this weird behavior or has any > further tips on solving this issue. > https://sebode-online.de/dovecot-letsencrypt/ > > Every hint is highly appreciated! > > Best Regards > Bastian > From mpeters at domblogger.net Mon Feb 20 14:49:17 2017 From: mpeters at domblogger.net (Michael A. Peters) Date: Mon, 20 Feb 2017 06:49:17 -0800 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1889272825.902200.1487583152217@mail.yahoo.com> References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> <6caf1f01-d2a2-139b-59f9-083b89d7d87d@domblogger.net> <1889272825.902200.1487583152217@mail.yahoo.com> Message-ID: <0c80fd4f-94b0-61a3-1ea1-7800b5829649@domblogger.net> On 02/20/2017 01:32 AM, chaouche yacine wrote: > What is the motivation behind using a new pair of keys and CSR ? > Every now and then, a bug in the OpenSSL API is found that leaked the private key under certain conditions. By replacing the private key once a year with a new one, you are at lower risk of having a private key that is exposed by such a bug even if the bug isn't published and only a few know about it. heartbleed was one such bug, DROWN was another. Obviously when a bug of that type is found and reported and your server was potentially vulnerable you change right away - but when you use the same private key for a long time, you risk a scenario where the NSA knew about it, you stopped using the protocol or cipher before it became public, it becomes public several years later but you aren't worried because you haven't run that protocol or cipher suite in quite some time - yet the NSA already has your private key from years ago. That's why I always generate new private key once a year. It just reduces exploitable exposure in the unlikely but possible scenario that the private key was compromised and I did not know it. That's also why I only allow ciphers that use forward secrecy for connections from mail clients. From mailinglist at unix-solution.de Mon Feb 20 15:09:52 2017 From: mailinglist at unix-solution.de (basti) Date: Mon, 20 Feb 2017 16:09:52 +0100 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1806248583.1038101.1487601811691@mail.yahoo.com> References: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> <1806248583.1038101.1487601811691@mail.yahoo.com> Message-ID: <39db1cc2-c069-1ca1-9c6f-c1fc89acb5c0@unix-solution.de> I have try LE on October 2016 and use Icedove 45.6.0. I can't found any certificate of LE in certificate manager -> authorities On 20.02.2017 15:43, chaouche yacine wrote: > Hello Basti. Maybe you tried LE too early when it was not universally accepted as a trusted CA ? > > > On Monday, February 20, 2017 2:22 PM, basti wrote: > > > Hello, > I had the same problem. LE is not in the CA list. > > Best Regards, From yacinechaouche at yahoo.com Mon Feb 20 16:35:27 2017 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Mon, 20 Feb 2017 16:35:27 +0000 (UTC) Subject: Problem with Let's Encrypt Certificate In-Reply-To: <39db1cc2-c069-1ca1-9c6f-c1fc89acb5c0@unix-solution.de> References: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> <1806248583.1038101.1487601811691@mail.yahoo.com> <39db1cc2-c069-1ca1-9c6f-c1fc89acb5c0@unix-solution.de> Message-ID: <1997639749.1125700.1487608527460@mail.yahoo.com> Bast, the way I understand it is that Let's Encrypt is not a Root Certificate Authority, it's an intermediate. The root CA of Let's Encrypt is " DST_Root_CA_X3.crt", you should find it in /etc/ssl/certs/. I have sucessfully installed a Let's Encrypt certificate on a debian machine by Octobre 2016th too and it worked just fine. -- Yassine. From basti at unix-solution.de Mon Feb 20 20:10:59 2017 From: basti at unix-solution.de (basti) Date: Mon, 20 Feb 2017 21:10:59 +0100 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1997639749.1125700.1487608527460@mail.yahoo.com> References: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> <1806248583.1038101.1487601811691@mail.yahoo.com> <39db1cc2-c069-1ca1-9c6f-c1fc89acb5c0@unix-solution.de> <1997639749.1125700.1487608527460@mail.yahoo.com> Message-ID: <351d847d-f65f-b9e5-27e7-8bad1140bf33@unix-solution.de> Hello, I have fixed my problem. I had used the wrong cert-file. ssl_cert = Bast, the way I understand it is that Let's Encrypt is not a Root Certificate Authority, it's an intermediate. The root CA of Let's Encrypt is " > DST_Root_CA_X3.crt", you should find it in /etc/ssl/certs/. I have sucessfully installed a Let's Encrypt certificate on a debian machine by Octobre 2016th too and it worked just fine. > > > -- Yassine. From mailinglist at unix-solution.de Mon Feb 20 20:11:52 2017 From: mailinglist at unix-solution.de (basti) Date: Mon, 20 Feb 2017 21:11:52 +0100 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1997639749.1125700.1487608527460@mail.yahoo.com> References: <38532248-7cf3-11d9-337e-146d2cf1484d@unix-solution.de> <1806248583.1038101.1487601811691@mail.yahoo.com> <39db1cc2-c069-1ca1-9c6f-c1fc89acb5c0@unix-solution.de> <1997639749.1125700.1487608527460@mail.yahoo.com> Message-ID: <612f24de-5d79-1f3e-cbdf-a76bde26e13a@unix-solution.de> Hello, I have fixed my problem. I had used the wrong cert-file. ssl_cert = Bast, the way I understand it is that Let's Encrypt is not a Root Certificate Authority, it's an intermediate. The root CA of Let's Encrypt is " > DST_Root_CA_X3.crt", you should find it in /etc/ssl/certs/. I have sucessfully installed a Let's Encrypt certificate on a debian machine by Octobre 2016th too and it worked just fine. > > > -- Yassine. From ad+lists at uni-x.org Mon Feb 20 20:51:26 2017 From: ad+lists at uni-x.org (Alexander Dalloz) Date: Mon, 20 Feb 2017 21:51:26 +0100 Subject: Sieve not filtering In-Reply-To: References: <329c1c0d-7442-4512-8d51-3fec904df86e@list-subs.com> <47B7C984-DB29-4371-899E-8D32BEF57990@mail.sermon-archive.info> Message-ID: <0c67718b-05ea-f7ed-432b-3341ecca6c05@uni-x.org> Am 18.02.2017 um 17:44 schrieb Ben: > [ ... ] > Actually, after many hours of head-bashing, I discovered the problem. > > sieve doesn't work when you're just using telnet port 25 ! > > I was doing : > ehlo test > mail from:sender at example.com > rcpt to:recip at example.com > data > Subject: hello world > Hello World ! > . > > With the above, sieve was simply sending everything to INBOX > > When I changed my methodology : > ehlo test > mail from:sender at example.com > rcpt to:recip at example.com > data > From: > To: > Subject: hello world > Hello World ! > . > > It worked as expected. Just for the record: please don't use that invalid syntax. Alexander From jtam.home at gmail.com Mon Feb 20 21:01:37 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 20 Feb 2017 13:01:37 -0800 (PST) Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: yacinechaouche at yahoo.com writes: > Interesting. Is there any particular benefit in having only one file > for both certificate and private key ? I find that putting private key > in a separate file feels more secure. It's convenient to have key and cert in one place if you don't need the certificate to be publically readable. Keeping it in separate files would add slightly more security (defense in depth), that would protect from, for example, an admin fumble or bug in the SSL library. "Michael A. Peters" writes: >> I use dehydrated (with Cloudflare DNS challenges) and as far as I know, >> it seems to generate a new private key every time. > > Yeah that would be a problem for me because I implement DANE. It's on my to-do list, but I think you can use dehydrated in signing mode. --signcsr (-s) path/to/csr.pem Sign a given CSR, output CRT on stdout (advanced usage) In this way, you can reuse private key, as well as making it more secure by removing a privileged operations (private key acces) allowing dehydrated to be run as a non-privilged/separate user. Joseph Tam From tanstaafl at libertytrek.org Mon Feb 20 21:56:47 2017 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Mon, 20 Feb 2017 16:56:47 -0500 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: On Mon Feb 20 2017 05:15:34 GMT-0500 (Eastern Standard Time), Timo Sirainen wrote: > + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them > in lib-dsasl for client side. So... does this mean dovecot now has OAUTH2 support? If so... yay! and I'll go open a Thunderbird bug to add support for dovecot's OAUTH2... From reuben-dovecot at reub.net Mon Feb 20 22:08:48 2017 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Tue, 21 Feb 2017 09:08:48 +1100 Subject: Replication Troubles In-Reply-To: <95751283-49CE-4A3C-A1BF-0AC8AAEED08A@iki.fi> References: <571885AA-C2DA-4240-A0B2-F0681D347DE4@wogri.com> <95751283-49CE-4A3C-A1BF-0AC8AAEED08A@iki.fi> Message-ID: On 20/02/2017 9:09 PM, Timo Sirainen wrote: > On 17 Feb 2017, at 21.37, Wolfgang Hennerbichler > wrote: >> >> Hi Dovecot Users, >> >> I?ve configured dovecot dsync replication and I see troubles in the >> logs and get user complaints which I can?t explain. I found similar >> threads on this mailinglist, but I couldn?t find a solution >> anywhere. Does anybody have dsync running without problems on a >> high volume mailserver? >> >> I see the following logs, examples given: >> >> Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: >> /var/mail/zoechi/dovecot-uidlist: Duplicate file entry at line >> 10395: 1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377 >> (uid 41092 -> 41093) - retrying by re-reading from beginning >> >> with this one I?m not sure - it might be that this is completely OK >> because due to replication UIDs clash. Maybe that?s OK, but I >> couldn?t find a confirmation. >> >> Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: Maildir >> /var/mail/zoechi: Expunged message reappeared, giving a new UID >> (old uid=41092, >> file=1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377) > > There seems to be something weird with using Maildir and replication. > Haven't had time to debug it and it's likely not an easy bug to fix, > so for now the solution would be to use only sdbox/mdbox with > replication. I have experienced this quite a few times. It is tricky to reliably reproduce. I am also using replication with Maildir. This appears to be a timing/locking issue of some sort. One way to trigger this seems to be to have a new mail come in and be delivered, and then as soon as it appears in Thunderbird, delete it straight away. Thunderbird is set to move mails to Trash when they are deleted and expunge the Inbox on exit. My replication partner sees the new mail has come in, and then it pushes it back to the original. Meanwhile the one on the original has already been deleted by the client so the mail re-appears back on the original again. The latency to my replication partner is 150ms or so but this problem might become easier to reproduce if the delay is increased. I can't say that this is definitely how to reproduce it all of the time but when I have experience this, that's roughly the sequence of events. Reuben From stephan at rename-it.nl Mon Feb 20 22:18:00 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 20 Feb 2017 23:18:00 +0100 Subject: Released Pigeonhole v0.4.17.rc1 for Dovecot v2.2.28.rc1. Message-ID: <1d2b6d4a-6e9e-3f3d-fda3-acd43870a064@rename-it.nl> Hello Dovecot users, Several bugs were found in the course of the last few months, but no new features were created. So, this will be strictly a bug-fixing release. The previous Pigeonhole release should still work with the new Dovecot release, which means you are not required to upgrade. However, my advice would be to upgrade once this release candidate matures into a final release, since at least the first bug listed below is pretty severe. Changelog v0.4.17: - LDA Sieve plugin: Fixed handling of an early explicit keep during multiscript execution. Action side-effects and the message snapshot would be lost at the final stage where the implicit keep is evaluated. This could result in the IMAP flags assigned to the message to be forgotten or that headers modified by the "editheader" extension would revert to their original state. - file script storage: Amended the up-to-date time stamp comparison for on-disk binaries to include nanoseconds. This will fix problems occurring when both binary and script are saved within the same second. This fix is ineffective on older systems that have no support for nanoseconds in stat() timestamps, which should be pretty rare nowadays. - file script storage: Improve saving and listing permission error to include more details. - imapsieve plugin: Make sure "INBOX" is upper case in static mailbox rules. Otherwise, the mailbox name would never match, since matching is performed case-sensitively and Dovecot only returns the upper-cased "INBOX". - imapsieve plugin: Fixed assert failure occurring when used with virtual mailboxes. - doveadm sieve plugin: Fixed crash when setting Sieve script via attribute's string value. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.17.rc1.tar.gz http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.17.rc1.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From linux at c5ace.com Mon Feb 20 23:42:51 2017 From: linux at c5ace.com (C5ace) Date: Tue, 21 Feb 2017 09:42:51 +1000 Subject: IMAP directory structure. Message-ID: <58AB7EFB.3060405@c5ace.com> Hello, I use Dovecot 2.1.7. and like to know how to force the IMAP directory structure to be: The IMAP server's desired directory structure is: root at server-2:/var/vmail/c5ace.com/test/Maildir# /.INBOX /.INBOX.Archives /.INBOX.Drafts /.INBOX.Junk /.INBOX.Sent /.INBOX.Templates /.INBOX.Trash and to prevent the mail clients like Thunderbird, Claws Mail, etc. to add additional out of tree directories. Thanks in advance. Elmar From dclist at list.jmatt.net Tue Feb 21 02:45:11 2017 From: dclist at list.jmatt.net (Matt Simpson) Date: Mon, 20 Feb 2017 21:45:11 -0500 Subject: Released Pigeonhole v0.4.17.rc1 for Dovecot v2.2.28.rc1. In-Reply-To: <1d2b6d4a-6e9e-3f3d-fda3-acd43870a064@rename-it.nl> References: <1d2b6d4a-6e9e-3f3d-fda3-acd43870a064@rename-it.nl> Message-ID: > On Feb 20, 2017, at 5:18 PM, Stephan Bosch wrote: > > Changelog v0.4.17: > I didn?t see this reported bug listed in the changelog, did it get fixed? >> >> On January 12, 2017 at 9:55 PM Matt Simpson wrote: >> >> >> I?m running dovecot 2.2.27 and pigeonhole 0.4.16 on FreeBSD 11. >> >> I?m using the pigeonhole/sieve external pipe plugin to run a Perl program to send a Pushover notification when certain messages are received. >> >> The Perl script is executed, and the notification is sent. But then the script task seems to go zombie until it is killed after a timeout. >> > >> I?m not a Unix programming ace, but from what I?ve been able to find out, this seems to mean that the lda process is forking another process to run the pipe script, and not getting the proper notification when it finishes (not issuing a wait?). So after 10 seconds, it sends a TERM to the task which is no longer running, and when that doesn?t work, it sends a KILL. Anybody know what?s happening here? >> > > Seems that we are not doing waitpid() on your program when it's killed. Also, I guess we should wait longer than 0 msecs. I'll try and see if I can replicate this. > > Aki From aki.tuomi at dovecot.fi Tue Feb 21 07:27:33 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 21 Feb 2017 09:27:33 +0200 Subject: Released Pigeonhole v0.4.17.rc1 for Dovecot v2.2.28.rc1. In-Reply-To: References: <1d2b6d4a-6e9e-3f3d-fda3-acd43870a064@rename-it.nl> Message-ID: <1099dca7-053c-113b-57e3-9b520a7c2b45@dovecot.fi> On 21.02.2017 04:45, Matt Simpson wrote: >> On Feb 20, 2017, at 5:18 PM, Stephan Bosch wrote: >> >> Changelog v0.4.17: >> > I didn?t see this reported bug listed in the changelog, did it get fixed? >>> On January 12, 2017 at 9:55 PM Matt Simpson wrote: >>> >>> >>> I?m running dovecot 2.2.27 and pigeonhole 0.4.16 on FreeBSD 11. >>> >>> I?m using the pigeonhole/sieve external pipe plugin to run a Perl program to send a Pushover notification when certain messages are received. >>> >>> The Perl script is executed, and the notification is sent. But then the script task seems to go zombie until it is killed after a timeout. >>> >> >>> I?m not a Unix programming ace, but from what I?ve been able to find out, this seems to mean that the lda process is forking another process to run the pipe script, and not getting the proper notification when it finishes (not issuing a wait?). So after 10 seconds, it sends a TERM to the task which is no longer running, and when that doesn?t work, it sends a KILL. Anybody know what?s happening here? >>> >> Seems that we are not doing waitpid() on your program when it's killed. Also, I guess we should wait longer than 0 msecs. I'll try and see if I can replicate this. >> >> Aki Hi! The bug is probably in dovecot core, and not in pigeonhole. I wasn't able to yet try and replicate this issue. Aki From aki.tuomi at dovecot.fi Tue Feb 21 07:37:54 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 21 Feb 2017 09:37:54 +0200 Subject: v2.2.28 release candidate released In-Reply-To: References: Message-ID: On 20.02.2017 23:56, Tanstaafl wrote: > On Mon Feb 20 2017 05:15:34 GMT-0500 (Eastern Standard Time), Timo > Sirainen wrote: >> + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them >> in lib-dsasl for client side. > So... does this mean dovecot now has OAUTH2 support? If so... yay! and > I'll go open a Thunderbird bug to add support for dovecot's OAUTH2... Yes. It support OAUTHBEARER and XOAUTH2 bearer authentication. See https://wiki.dovecot.org/PasswordDatabase/oauth2 Aki From skdovecot at smail.inf.fh-brs.de Tue Feb 21 08:04:31 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 21 Feb 2017 09:04:31 +0100 (CET) Subject: IMAP directory structure. In-Reply-To: <58AB7EFB.3060405@c5ace.com> References: <58AB7EFB.3060405@c5ace.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 21 Feb 2017, C5ace wrote: > I use Dovecot 2.1.7. and like to know how to force the IMAP directory > structure to be: > > The IMAP server's desired directory structure is: > root at server-2:/var/vmail/c5ace.com/test/Maildir# > /.INBOX > /.INBOX.Archives > /.INBOX.Drafts > /.INBOX.Junk > /.INBOX.Sent > /.INBOX.Templates > /.INBOX.Trash > > and to prevent the mail clients like Thunderbird, Claws Mail, etc. to add > additional out of tree directories. Define the prefix of the default namespace as "INBOX." and deploy ACLs that deny to create new sub-mailboxes. (However, if I remember correctly the own has the permission to change the permissions implicitly.) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWKv0j3z1H7kL/d9rAQLq4Af/TIbcJ1BCkk+0l6RSEeHRscXhghKzNU0G pjOQyHx/DkUQEQMb+cWPR6smUEVzfNmL6zdzYx6FXyg5VpgfxOfUnxSreCc7OXIi olY8l1fB46Yi7qEfcW2uNp83OeX4W0wXgXJtEYGDatzL9C6wWbPEzPQFP4w4GYD2 DSZI7/haslTMZDeFiVQUlXtAdddFkl7vYA9e146QvNz0RiwIV4ITfO+caCsRFQPz a8hcoWCufTOjTXaEA91FiDBrPiRTgNeqJRfS8tca1Zyli9/rM6yzLHQLnCf6VQRf o7ZOLKVsIcYBE0ulBnVzCc47tv5B0HY/fe4J6LVMUDE8JbAl522dYQ== =6nsR -----END PGP SIGNATURE----- From dbetz at df.eu Tue Feb 21 10:49:17 2017 From: dbetz at df.eu (Daniel Betz) Date: Tue, 21 Feb 2017 10:49:17 +0000 Subject: How to dsync mdbox compressed to maildir uncompressed Message-ID: Hello, we are using doveadm sync to export mdbox to maildir format, so we can use an external tool to convert into an pst file. Since we have enabled zlib compression doveadm sync always exports the maildir gzip compressed. Are there any ways to prevent the doveadm sync to export the maildir compressed ? Have tried this: doveadm -o "maildir_copy_with_hardlinks=no" sync -u imap at test.de maildir:~/Maildir >From Wiki: If you want to use dsync to convert to a compressed Maildir you may need -o maildir_copy_with_hardlinks=no (this is set to yes by default and will prevent compression). Regards, Daniel # 2.2.27 (c0f36b0): /usr/local/dovecot2/etc/dovecot/dovecot.conf doveconf: Warning: service auth { client_limit=60000 } is lower than required under max. load (500500) # OS: Linux 3.10.0-327.36.3.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_cache_negative_ttl = 1 mins auth_cache_size = 64 M auth_cache_ttl = 2 hours auth_mechanisms = plain login auth_username_chars = base_dir = /var/run/dovecot/ debug_log_path = /dev/null default_login_user = dovecot default_vsz_limit = 750 M disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 first_valid_gid = 1001 first_valid_uid = 1001 info_log_path = /var/log/dovecot/messages lda_mailbox_autocreate = yes lda_original_recipient_header = X-Envelope-To log_path = /dev/stderr login_log_format_elements = user=[%u] method=%m rip=%r lip=%l %c mail_gid = 1001 mail_location = mdbox:~:INDEX=%h/INDEX mail_plugins = quota notify mail_log zlib mail_uid = 1001 mbox_write_locks = fcntl namespace { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox "Gesendete Elemente" { auto = no special_use = \Sent } mailbox "Infizierte Objekte" { auto = no special_use = \Junk } mailbox Sent { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = no special_use = \Trash } prefix = separator = . type = private } namespace inbox { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap2.conf driver = ldap } plugin { quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_rule1 = Trash:storage=+100M quota_rule2 = INBOX.Trash:storage=+100M quota_warning = storage=85%% quota-warning 85 %u quota_warning1 = storage=95%% quota-warning 95 %u quota_warning2 = storage=99%% quota-warning 99 %u zlib_save = gz zlib_save_level = 6 } replication_max_conns = 30 sendmail_path = /usr/local/exim/bin/exim service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = popuser } unix_listener replication-notify { mode = 0666 user = popuser } } service anvil { client_limit = 60000 } service auth { client_limit = 60000 unix_listener auth-userdb { mode = 0666 user = popuser } unix_listener auth { mode = 0666 user = popuser } } service config { unix_listener config { user = popuser } } service dict { unix_listener dict { mode = 0666 user = popuser } } service dns_client { process_limit = 6000 process_min_avail = 12 unix_listener dns-client { mode = 0666 user = popuser } } service doveadm { inet_listener { port = 12345 } user = popuser } service imap-login { chroot = login client_limit = 6000 process_limit = 100 process_min_avail = 16 service_count = 0 } service imap { executable = /usr/local/dovecot2/libexec/dovecot/imap process_limit = 250000 process_min_avail = 50 service_count = 250 } service ipc { client_limit = 60000 unix_listener ipc { mode = 0650 user = dovecot } unix_listener login/ipc-proxy { mode = 0650 user = dovecot } } service lmtp { unix_listener lmtp { mode = 0666 user = popuser } } service pop3-login { chroot = login client_limit = 6000 process_limit = 100 process_min_avail = 16 service_count = 0 } service pop3 { executable = /usr/local/dovecot2/libexec/dovecot/pop3 process_limit = 250000 process_min_avail = 50 service_count = 250 } service quota-warning { executable = script /usr/local/dovecot2/bin/quota-warning.sh unix_listener quota-warning { mode = 0600 user = popuser } user = popuser } service replicator { unix_listener replicator-doveadm { mode = 0600 user = popuser } } ssl_cert = References: Message-ID: On 21 Feb 2017, at 12.49, Daniel Betz wrote: > > Hello, > > we are using doveadm sync to export mdbox to maildir format, so we can use an external tool to convert into an pst file. > Since we have enabled zlib compression doveadm sync always exports the maildir gzip compressed. > > Are there any ways to prevent the doveadm sync to export the maildir compressed ? > > Have tried this: doveadm -o "maildir_copy_with_hardlinks=no" sync -u imap at test.de maildir:~/Maildir > From Wiki: If you want to use dsync to convert to a compressed Maildir you may need -o maildir_copy_with_hardlinks=no (this is set to yes by default and will prevent compression). Run it via two processes so you can give separate settings for them, something like: doveadm sync -u imap at test.d 'doveadm -o mail=~/Maildir -o mail_plugins=everything-but-zlib dsync-server' From tlx at leuxner.net Tue Feb 21 11:02:29 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 21 Feb 2017 12:02:29 +0100 Subject: How to dsync mdbox compressed to maildir uncompressed In-Reply-To: References: Message-ID: <20170221110229.GA40596@nihlus.leuxner.net> * Daniel Betz 2017.02.21 11:49: > Have tried this: doveadm -o "maildir_copy_with_hardlinks=no" sync -u imap at test.de maildir:~/Maildir > From Wiki: If you want to use dsync to convert to a compressed Maildir you may need -o maildir_copy_with_hardlinks=no (this is set to yes by default and will prevent compression). doveadm -o plugin/quota= -o plugin/zlib_save= backup -u imap at test.de maildir:~/Maildir Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From dbetz at df.eu Tue Feb 21 12:15:47 2017 From: dbetz at df.eu (Daniel Betz) Date: Tue, 21 Feb 2017 12:15:47 +0000 Subject: How to dsync mdbox compressed to maildir uncompressed Message-ID: Hi Timo, thank you for the hint, but it doesnt seems to work. doveadm sync -u imap at test.de 'doveadm -o mail="maildir:~/Maildir" -o "mail_plugins=quota" dsync-server -u imap at test.de' Also tried -o mail=~/Maildir .. -o maildir:~/Maildir .. The log throws an error: Feb 21 13:05:35 doveadm: Error: Panic: io_add(0x1) called twice fd=9, callback=0x7f49baa06840 -> 0x7f49ba991e30 Feb 21 13:05:35 doveadm: Error: Error: Raw backtrace: /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(+0x92d70) [0x7f49ba9efd70] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f49ba9efdda] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f49ba98b4e0] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(ioloop_iolist_add+0x83) [0x7f49baa03dc3] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f49baa046db] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(+0xa599f) [0x7f49baa0299f] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(io_add+0xd) [0x7f49baa02a4d] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(master_service_io_listeners_add+0x65) [0x7f49ba9916d5] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(master_service_init_finish+0xb7) [0x7f49ba9917a7] -> /usr/local/dovecot2/bin/doveadm(main+0x189) [0x4143a9] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f49ba5bcb15] -> /usr/local/dovecot2/bin/doveadm() [0x414785] Feb 21 13:05:35 dsync-local(imap at test.de): Error: read(remote) failed: EOF (version not received) Feb 21 13:05:35 dsync-local(imap at test.de): Error: Remote command died with signal 6: doveadm -o mail="maildir:~/Maildir" -o "mail_plugins=quota" dsync-server -u imap at test.de dsync-server Regards, Daniel Von: Timo Sirainen [mailto:tss at iki.fi] Gesendet: Dienstag, 21. Februar 2017 12:01 An: Daniel Betz Cc: dovecot at dovecot.org Betreff: Re: How to dsync mdbox compressed to maildir uncompressed On 21 Feb 2017, at 12.49, Daniel Betz wrote: Hello, we are using doveadm sync to export mdbox to maildir format, so we can use an external tool to convert into an pst file. Since we have enabled zlib compression doveadm sync always exports the maildir gzip compressed. Are there any ways to prevent the doveadm sync to export the maildir compressed ? Have tried this: doveadm -o "maildir_copy_with_hardlinks=no" sync -u imap at test.de maildir:~/Maildir >From Wiki: If you want to use dsync to convert to a compressed Maildir you may need -o maildir_copy_with_hardlinks=no (this is set to yes by default and will prevent compression). Run it via two processes so you can give separate settings for them, something like: doveadm sync -u imap at test.d?'doveadm -o mail=~/Maildir -o mail_plugins=everything-but-zlib dsync-server' From aki.tuomi at dovecot.fi Tue Feb 21 12:54:40 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 21 Feb 2017 14:54:40 +0200 Subject: doveadm: Fatal: All your namespaces have a location setting In-Reply-To: <189e1af0-8399-9078-c48b-9395fea136a3@list-subs.com> References: <5997b5c9-ab94-3259-777f-17de4ee10418@list-subs.com> <7dd759bc-444a-8a62-2238-221a3e210a48@dovecot.fi> <189e1af0-8399-9078-c48b-9395fea136a3@list-subs.com> Message-ID: On 20.02.2017 11:46, Ben wrote: > >> Hi! >> >> Can you post doveconf -n >> >> Aki > > # 2.2.10: /etc/dovecot/dovecot.conf > # OS: Linux 3.10.0-514.6.1.el7.x86_64 x86_64 CentOS Linux release > 7.3.1611 (Core) > auth_mechanisms = plain login > auth_verbose = yes > auth_verbose_passwords = sha1 > first_valid_uid = 1000 > mail_location = maildir:~/Maildir > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body environment > mailbox date ihave enotify > mbox_write_locks = fcntl > namespace inbox { > inbox = yes > location = Try removing this Aki From kevin at my.walr.us Tue Feb 21 14:49:39 2017 From: kevin at my.walr.us (KT Walrus) Date: Tue, 21 Feb 2017 09:49:39 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server Message-ID: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/ about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? I think the current implementation of having a separate process manage each active IMAP session (w/ the possibility of moving idling sessions to a single hibernate process) will never be able to deploy a single server managing 10 Million IMAP sessions. But, would it be possible to implement a new IMAP server plugin that uses a fixed configurable pool of ?worker? processes, much like NGINX or PHP-FPM does. These servers can probably scale to 10 Million TCP connections, if the server is carefully tuned and has enough cores/memory to support that many active sessions. I?m thinking that the new IMAP server could use some external database (e.g., Redis or Memcached) to save all the sessions state and have the ?worker? processes poll the TCP sockets for new IMAP commands to process (fetching the session state from the external database when it has a command that is waiting on a response). The Dovecot IMAP proxies could even queue incoming commands to proxy many incoming requests to a smaller number of backend connections (like ProxySQL does for MySQL requests). That might allow each Dovecot proxy to support 10 Million IMAP sessions and a single backend could support multiple front end Dovecot proxies (to scale to 100 Million concurrent IMAP connections using 10 proxies for 100 Million connections and 1 backend server for 10 Million connections). Of course, the backend server may need to be beefy and have very fast NVMe SSDs for local storage, but changing the IMAP server to manage a pool of workers instead of requiring a process per active session, would allow bigger scale up and could save large sites a lot of money. Is this a good idea? Or, am I missing something? Kevin From basdove at rediffmail.com Tue Feb 21 15:35:23 2017 From: basdove at rediffmail.com (Basdove) Date: 21 Feb 2017 15:35:23 -0000 Subject: =?utf-8?B?Q291bGQgbm90IGxvZ2luIGFzIHJvb3Qgb3Igb3RoZXIgTGludXggdXNlciBhY2NvdW50?= Message-ID: <20170221153523.17528.qmail@f5mail-224-159.rediffmail.com> Ubuntu server 16.04.2samba has upgraded from As per repository (latest version)I was configuring samba as per document from wiki "ActivedirectoryWINbindHowto"After editing the common-account and common-auth I rebooted the server.I could notlogin as root or any Linux user. Server says "Incorrect login" But I tried with all otherLinux user login which  are all logged well before inducing  root.How to login nowBelow is part of document :------------------------------------------------------------------------------------------------------------------------------- Note: You can use pam-auth-update to add the necessary entries for winbind authentication. If you installed libpam-winbind above, this step is all you need to do to configure pam. You may want to add the line to automatically create the home directory.sudo pam-auth-updateThis PAM configuration does not acquire a Kerberos TGT at login. To acquire a ticket, use kinit after logging in, and consider using kdestroy in a logout script.file: /etc/pam.d/common-accountaccount sufficient       pam_winbind.soaccount required         pam_unix.sofile: /etc/pam.d/common-authauth sufficient pam_winbind.soauth sufficient pam_unix.so nullok_secure use_first_passauth required   pam_deny.so--------------------------------------------------------------------------------------------------------------------  From aki.tuomi at dovecot.fi Tue Feb 21 15:45:00 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 21 Feb 2017 17:45:00 +0200 Subject: Could not login as root or other Linux user account In-Reply-To: <20170221153523.17528.qmail@f5mail-224-159.rediffmail.com> References: <20170221153523.17528.qmail@f5mail-224-159.rediffmail.com> Message-ID: On 2017-02-21 17:35, Basdove wrote: > Ubuntu server 16.04.2samba has upgraded from As per repository (latest version)I was configuring samba as per document from wiki "ActivedirectoryWINbindHowto"After editing the common-account and common-auth I rebooted the server.I could notlogin as root or any Linux user. Server says "Incorrect login" But I tried with all otherLinux user login which  are all logged well before inducing  root.How to login nowBelow is part of document :------------------------------------------------------------------------------------------------------------------------------- Note: You can use pam-auth-update to add the necessary entries for winbind authentication. If you installed libpam-winbind above, this step is all you need to do to configure pam. You may want to add the line to automatically create the home directory.sudo pam-auth-updateThis PAM configuration does not acquire a Kerberos TGT at login. To acquire a ticket, use kinit after logging in, and > consider using kdestroy in a logout script.file: /etc/pam.d/common-accountaccount sufficient       pam_winbind.soaccount required         pam_unix.sofile: /etc/pam.d/common-authauth sufficient pam_winbind.soauth sufficient pam_unix.so nullok_secure use_first_passauth required   pam_deny.so--------------------------------------------------------------------------------------------------------------------  Could you send your email in some readable format? Aki From listserv at xtlv.cn Tue Feb 21 16:04:13 2017 From: listserv at xtlv.cn (Mario Arnold) Date: Tue, 21 Feb 2017 17:04:13 +0100 Subject: segfault in lib20_expire_plugin Message-ID: <58AC64FD.7070607@xtlv.cn> Hello, after upgrade from [2.2.devel (34f7cc3)] to [2.2.devel (b3443fc)] dovecot stops with a segfault: Fatal: master: service(imap): child 21179 killed with signal 11 (core dumped) imap[21179]: segfault at 0 ip 00000000f726eef1 sp 00000000ffa3b050 error 4 in lib20_expire_plugin.so[f726d000+3000] gdb /usr/lib/dovecot/imap /var/_core/core_imap-11-5000-5000-21179 GNU gdb (Debian 7.12-6) 7.12.0.20161007-git Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug/.build-id/99/6f1cf1a262cf5738f075ec046d9a7d344d9693.debug...done. done. [New LWP 21179] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1". Core was generated by `dovecot/imap imap-postlogin'. Program terminated with signal SIGSEGV, Segmentation fault. #0 expire_mail_namespaces_created (ns=0xf814db90) at expire-plugin.c:428 428 expire-plugin.c: Datei oder Verzeichnis nicht gefunden. (gdb) bt full #0 expire_mail_namespaces_created (ns=0xf814db90) at expire-plugin.c:428 user = 0xf814d028 v = 0x0 db = 0xf815b960 error = 0xf81522c0 "Trash" #1 0xf75f9b21 in hook_mail_namespaces_created (namespaces=0xf8151008) at mail-storage-hooks.c:304 _data_stack_cur_id = 4 hooks__foreach_end = 0xf814dae8 hooks = 0xf814dad4 #2 0xf75ebf9f in mail_namespaces_init_finish (namespaces=0xf8151008, error_r=0xffa3b23c) at mail-namespace.c:383 _data_stack_cur_id = 3 ns = 0x0 prefixless_found = false __FUNCTION__ = "mail_namespaces_init_finish" #3 0xf75ec1bb in mail_namespaces_init (user=0xf814d028, error_r=0xffa3b23c) at mail-namespace.c:438 mail_set = 0xf814d118 ns_set = unexpanded_ns_set = namespaces = 0xf8151008 ns_p = i = count = count2 = __FUNCTION__ = "mail_namespaces_init" #4 0xf75fcd30 in mail_storage_service_init_post (ctx=, error_r=, mail_user_r=, priv=, user=) at mail-storage-service.c:728 mail_set = 0xf814d118 mail_user = 0xf814d028 #5 mail_storage_service_next_real (mail_user_r=, user=, ctx=) at mail-storage-service.c:1426 len = 4162116496 priv = {uid = 5000, gid = 5000, uid_source = 0xf76afeb4 "userdb lookup", gid_source = 0xf76afeb4 "userdb lookup", home = 0xf813ea71 "/srv/vmail/xtlv.de/1000", chroot = 0xf8130a20 ""} error = 0xf75b9934 "4\210\024" #6 mail_storage_service_next (ctx=0xf814d118, user=0xf813da90, mail_user_r=0xffa3b304) at mail-storage-service.c:1444 No locals. #7 0xf75fd0ff in mail_storage_service_lookup_next (ctx=0xf81399b0, input=0xffa3b368, user_r=0xffa3b300, mail_user_r=0xffa3b304, error_r=0xffa3b360) at mail-storage-service.c:1477 user = 0xf813da90 ret = #8 0xf77832c9 in client_create_from_input (input=0xffa3b368, fd_in=15, fd_out=15, client_r=0xffa3b35c, error_r=0xffa3b360) at main.c:228 user = 0x81a4 mail_user = 0xc34a5 ns = 0xf814d000 client = 0xffa3b304 imap_set = 0xffa3b360 lda_set = 0xffa3b304 errstr = 0xf814db90 "" mail_error = 49663 #9 0xf77834ea in login_client_connected (login_client=0xf813b450, username=0xf81300c8 "1000 at xtlv.de", extra_fields=0xf81300ac) at main.c:316 input = {module = 0xf778b616 "imap", service = 0xf778b616 "imap", username = 0xf81300c8 "1000 at xtlv.de", session_id = 0xf813b4c0 "doleKgxJ3s8l6zee", session_id_prefix = 0x0, session_create_time = 0, local_ip = { family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "T&K\217", '\000' , __u6_addr16 = {9812, 36683, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {2404066900, 0, 0, 0}}}, ip4 = {s_addr = 2404066900}}}, remote_ip = { family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "%\353\067\236", '\000' , __u6_addr16 = {60197, 40503, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {2654464805, 0, 0, 0}}}, ip4 = {s_addr = 2654464805}}}, local_port = 0, remote_port = 0, userdb_fields = 0xf81300ac, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0, debug = 0} client = 0xf750331c flags = error = 0xf8130130 "auth_token=3763fd48bfdfeea2a3617cbda148915c19e125fa" __FUNCTION__ = "login_client_connected" #10 0xf749b0d1 in master_login_auth_finish (client=0xf813b450, auth_args=0xf814d000, auth_args at entry=0xf81300a8) at master-login.c:210 login = 0xf813ab88 service = 0xf81383e8 __FUNCTION__ = "master_login_auth_finish" #11 0xf749b6a9 in master_login_postlogin_input (pl=0xf813d720) at master-login.c:284 login = 0xf813ab88 buf = "1000 at xtlv.de\tquota_rule=*:storage=5M\tuid=5000\tgid=5000\thome=/srv/vmail/xtlv.de/1000\tauth_token=3763fd48bfdfeea2a3617cbda148915c19e125fa\nQ\345td", '\000' , "\006\000\000\000\020\000\000\000R\345td\260.\000\000\260>\000\000\260>\000\000P\001\000\000P\001\000\000\004\000\000\000\001\000\000\000"... auth_args = 0xf81300a8 p = 0xf81300c0 ret = -1 fd = -1 ---Type to continue, or q to quit--- #12 0xf751b73e in io_loop_call_io (io=0xf813b358) at ioloop.c:599 ioloop = 0xf81384b8 t_id = 2 __FUNCTION__ = "io_loop_call_io" #13 0xf751cece in io_loop_handler_run_internal (ioloop=0xf81384b8) at ioloop-epoll.c:223 ctx = 0xf813a210 list = 0xf813b390 io = tv = {tv_sec = 59, tv_usec = 999219} events_count = msecs = ret = 1 i = j = 0 call = __FUNCTION__ = "io_loop_handler_run_internal" #14 0xf751b7ef in io_loop_handler_run (ioloop=0xf81384b8) at ioloop.c:648 No locals. #15 0xf751b999 in io_loop_run (ioloop=0xf81384b8) at ioloop.c:623 __FUNCTION__ = "io_loop_run" #16 0xf749d985 in master_service_run (service=0xf81383e8, callback=0xf7782b60 ) at master-service.c:641 No locals. #17 0xf7766ece in main (argc=2, argv=0xffa3bab4) at main.c:460 set_roots = {0xf779d2a0 , 0xf7732ac0 , 0x0} login_set = {auth_socket_path = 0xf8130030 "ta_rule=*:storage=5M", postlogin_socket_path = 0xf8130060 "/vmail/xtlv.de/1000", postlogin_timeout_secs = 60, callback = 0xf7783430 , failure_callback = 0xf7782cc0 , request_auth_token = 1} service_flags = storage_service_flags = username = auth_socket_path = c = doveconf -n # 2.2.devel (b3443fc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (0d78a30) # OS: Linux 3.2.64.stk32 x86_64 Debian 9.0 auth_debug = yes auth_debug_passwords = yes auth_failure_delay = 5 secs auth_master_user_separator = * auth_mechanisms = digest-md5 cram-md5 auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot/dov_debug.log deliver_log_format = msgid=%m: %$: DeliveryTime=%{delivery_time}: LMTPSessionTime=%{session_time} dict { expire = db:/var/lib/dovecot/expire.db } doveadm_password = # hidden, use -P to show it doveadm_port = 1090 imap_logout_format = in=%i out=%o MailHeaderR=%{fetch_hdr_count} MailBodyR=%{fetch_body_count} MailDel=%{deleted} MailExpung=%{expunged} MailTrash=%{trashed} lmtp_rcpt_check_quota = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_debug = yes mail_gid = vmail mail_location = sdbox:%h/sdbox mail_plugins = " quota mail_log notify expire zlib acl notify replication" mail_privileged_group = vmail mail_uid = vmail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { ignore_on_failure = no inbox = no list = children location = sdbox:%%h/sdbox prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = scheme=PLAIN username_format=%Lu /etc/dovecot/user_pw/passwd driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/srv/vmail/shared-db/shared-mailboxes expire = Trash 2h expire_cache = yes expire_dict = proxy::expire mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename save mailbox_create mail_log_fields = uid box msgid size subject mail_replica = tcps:mx00.vtlx.cn quota = dict:User quota::file:%h/dovecot-quota quota_exceeded_message = Die Mailbox des Empfaengers ist voll -- Quota exceeded -- Please contact quota_grace = 12%% quota_rule = *:storage=350M quota_rule2 = Trash:storage=+15%% quota_status_nouser = DUNNO quota_status_overquota = "552 5.5.2 Die Mailbox des Empfaengers ist voll ## Mailbox is full ## Please contact " quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=80%% quota-warning 80 %u quota_warning4 = storage=70%% quota-warning 70 %u sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz zlib_save_level = 6 } pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m/%{deleted_bytes}, size=%s postmaster_address = postmaster at xtlv.cn protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { mode = 0660 user = vmail } unix_listener replication-notify { mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service config { unix_listener config { mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service doveadm { inet_listener { address = 84.38.75.143, 2a00:5080:1:16::8 port = 1090 ssl = yes } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = 84.38.75.143 port = 993 ssl = yes } } service imap-postlogin { executable = script-login -d /usr/local/sbin/dov-last-login-imap.sh user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { address = 84.38.75.143 port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = 84.38.75.143 port = 995 ssl = yes } } service pop3-postlogin { executable = script-login -d /usr/local/sbin/dov-last-login-pop3.sh user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_ca = Hello Community, I am currently facing the following: - dovecot+postfix+sieve are running smoothly using passwd-file authentication - if a add a second authentication scheme (let's say mysql), I face a problem with sieve: -- receiving thru postfix is ok on both passwd-file and mysql entries and correctly stored -- I am able to send from the server as before ** BUT sieve == does not authenticate anymore from a client (using the same configuration as before ie using imap credentials) == does not process the messages anymore Digging in the sieve logs, it reports not finding the scripts anymore for existing accounts found in the passwd-file Any idea? Thank you! - - - - + - - - - # Here is the 90-sieve.conf: plugin { sieve = file:/sd/MAIL_IMAP_POP/%d/%n/_dovecot-sieve;active=/sd/MAIL_IMAP_POP/%d/%n/_dovecot-sieve-active sieve_default = /sd/myhost/var/lib/dovecot/sieve/default.sieve sieve = /sd/MAIL_IMAP_POP/%d/%n/__Sieve sieve_global_dir = /sd/myhost/var/lib/dovecot/sieve/global/ sieve_before = /sd/MAIL_IMAP_POP/SieveBefore sieve_after = /sd/MAIL_IMAP_POP/%d/SieveAfter/ sieve_after2 = /sd/MAIL_IMAP_POP/SieveAfter/ sieve_plugins = sieve_extprograms sieve_extensions = +vnd.dovecot.filter sieve_filter_bin_dir = /etc/dovecot/sieve-filters } - - - - + - - - - # Authentication for SQL users. Included from 10-auth.conf. passdb sql { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext # Associated query: # password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; } userdb sql { driver = static args = uid=vmail gid=vmail home=/sd/MAIL_IMAP_POP/%d/%n:LAYOUT=fs } From aki.tuomi at dovecot.fi Tue Feb 21 16:19:29 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 21 Feb 2017 18:19:29 +0200 (EET) Subject: segfault in lib20_expire_plugin In-Reply-To: <58AC64FD.7070607@xtlv.cn> References: <58AC64FD.7070607@xtlv.cn> Message-ID: <272366175.514.1487693970401@appsuite-dev.open-xchange.com> > On February 21, 2017 at 6:04 PM Mario Arnold wrote: > > > Hello, > > after upgrade from [2.2.devel (34f7cc3)] to [2.2.devel (b3443fc)] dovecot > stops with a segfault: > > Fatal: master: service(imap): child 21179 killed with signal 11 (core dumped) > imap[21179]: segfault at 0 ip 00000000f726eef1 sp 00000000ffa3b050 error 4 in > lib20_expire_plugin.so[f726d000+3000] > Hi! Thank you for your report, we'll look into it. Aki From stephan at rename-it.nl Tue Feb 21 17:09:17 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 21 Feb 2017 18:09:17 +0100 Subject: Sieve and multi-auth databases In-Reply-To: <1acb087d-4e00-cb61-b91c-1622104a7af8@avv.solutions> References: <1acb087d-4e00-cb61-b91c-1622104a7af8@avv.solutions> Message-ID: Op 2/21/2017 om 5:19 PM schreef dovecot at avv.solutions: > Hello Community, > > I am currently facing the following: > > - dovecot+postfix+sieve are running smoothly using passwd-file > authentication > - if a add a second authentication scheme (let's say mysql), I face a > problem with sieve: > -- receiving thru postfix is ok on both passwd-file and mysql > entries and correctly stored > -- I am able to send from the server as before > ** BUT sieve > == does not authenticate anymore from a client (using the same > configuration as before ie using imap credentials) > == does not process the messages anymore > > Digging in the sieve logs, it reports not finding the scripts anymore > for existing accounts found in the passwd-file > > Any idea? > You should enable mail_debug. That will provide details on what Sieve is doing regarding file system storage paths. Also a full `dovecot -n` output is helpful. Regards, Stephan. > Thank you! > > > - - - - + - - - - > # Here is the 90-sieve.conf: > > plugin { > sieve = > file:/sd/MAIL_IMAP_POP/%d/%n/_dovecot-sieve;active=/sd/MAIL_IMAP_POP/%d/%n/_dovecot-sieve-active > sieve_default = /sd/myhost/var/lib/dovecot/sieve/default.sieve > > sieve = /sd/MAIL_IMAP_POP/%d/%n/__Sieve > > sieve_global_dir = /sd/myhost/var/lib/dovecot/sieve/global/ > > sieve_before = /sd/MAIL_IMAP_POP/SieveBefore > sieve_after = /sd/MAIL_IMAP_POP/%d/SieveAfter/ > sieve_after2 = /sd/MAIL_IMAP_POP/SieveAfter/ > > sieve_plugins = sieve_extprograms > sieve_extensions = +vnd.dovecot.filter > sieve_filter_bin_dir = /etc/dovecot/sieve-filters > > } > > - - - - + - - - - > # Authentication for SQL users. Included from 10-auth.conf. > passdb sql { > driver = sql > args = /etc/dovecot/dovecot-sql.conf.ext > # Associated query: > # password_query = SELECT email as user, password FROM virtual_users > WHERE email='%u'; > } > > userdb sql { > driver = static > args = uid=vmail gid=vmail home=/sd/MAIL_IMAP_POP/%d/%n:LAYOUT=fs > } From stephan at rename-it.nl Tue Feb 21 18:55:45 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 21 Feb 2017 19:55:45 +0100 Subject: Sieve and multi-auth databases In-Reply-To: References: <1acb087d-4e00-cb61-b91c-1622104a7af8@avv.solutions> Message-ID: <9ecc793a-18d0-11b2-1761-0d3eaa23caa2@rename-it.nl> Op 2/21/2017 om 6:09 PM schreef Stephan Bosch: > Op 2/21/2017 om 5:19 PM schreef dovecot at avv.solutions: > >> - - - - + - - - - >> # Authentication for SQL users. Included from 10-auth.conf. >> passdb sql { >> driver = sql >> args = /etc/dovecot/dovecot-sql.conf.ext >> # Associated query: >> # password_query = SELECT email as user, password FROM virtual_users >> WHERE email='%u'; >> } >> >> userdb sql { >> driver = static >> args = uid=vmail gid=vmail home=/sd/MAIL_IMAP_POP/%d/%n:LAYOUT=fs >> } Based on the log file you sent me, the above sql userdb is the problem. The configured home field makes no sense. A home directory is strictly a filesystem path and does not accept options such as LAYOUT. That only applies to a mail storage location; i.e., the "mail" field. What I find puzzling though is that that userdb is not in the configuration you sent me. Regards, Stephan. From chibi at gol.com Wed Feb 22 04:12:49 2017 From: chibi at gol.com (Christian Balzer) Date: Wed, 22 Feb 2017 13:12:49 +0900 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> Message-ID: <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> On Tue, 21 Feb 2017 09:49:39 -0500 KT Walrus wrote: > I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/ about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. > While that's a nice article, nothing in it was news to me or particular complex when one does large scale stuff, like Ceph for example. > Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? > I'm sure Timo's answer will (or would, if he could be bothered) be along the lines of: "Sure, if you give me all your gold and then some for a complete rewrite of, well, everything". What you're missing and what the bad idea here is that as mentioned before scale-up only goes so far. I was feeling that my goal of 500k users/sessions in 2-node active/active cluster was quite ambitious and currently I'm looking at 200k sessions as something achievable with the current Dovecot and other limitations. But even if you were to implement something that can handle 1 million or more sessions per server, would you want to? As in, if that server goes down, the resulting packet, authentication storm will be huge and most like result in a proverbial shit storm later. Having more than 10% or so of your customers on one machine and thus involved in an outage that you KNOW will hit you eventually strikes me as a bad idea. I'm not sure how the design below meshes with Timo's lofty goals and standards when it comes to security as well. And a push with the right people (clients) to support IMAP NOTIFY would of course reduce the number of sessions significantly. Finally, Dovecot in proxy mode already scales quite well. Christian > I think the current implementation of having a separate process manage each active IMAP session (w/ the possibility of moving idling sessions to a single hibernate process) will never be able to deploy a single server managing 10 Million IMAP sessions. > > But, would it be possible to implement a new IMAP server plugin that uses a fixed configurable pool of ?worker? processes, much like NGINX or PHP-FPM does. These servers can probably scale to 10 Million TCP connections, if the server is carefully tuned and has enough cores/memory to support that many active sessions. > > I?m thinking that the new IMAP server could use some external database (e.g., Redis or Memcached) to save all the sessions state and have the ?worker? processes poll the TCP sockets for new IMAP commands to process (fetching the session state from the external database when it has a command that is waiting on a response). The Dovecot IMAP proxies could even queue incoming commands to proxy many incoming requests to a smaller number of backend connections (like ProxySQL does for MySQL requests). That might allow each Dovecot proxy to support 10 Million IMAP sessions and a single backend could support multiple front end Dovecot proxies (to scale to 100 Million concurrent IMAP connections using 10 proxies for 100 Million connections and 1 backend server for 10 Million connections). > > Of course, the backend server may need to be beefy and have very fast NVMe SSDs for local storage, but changing the IMAP server to manage a pool of workers instead of requiring a process per active session, would allow bigger scale up and could save large sites a lot of money. > > Is this a good idea? Or, am I missing something? > > Kevin -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From ruga at protonmail.com Wed Feb 22 07:42:03 2017 From: ruga at protonmail.com (Ruga) Date: Wed, 22 Feb 2017 02:42:03 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> Message-ID: A more efficient algorithm would reduce computational complexity, and the need for expensive power-hungry CPUs. Sent from ProtonMail Mobile On Wed, Feb 22, 2017 at 5:12 AM, Christian Balzer <'chibi at gol.com'> wrote: On Tue, 21 Feb 2017 09:49:39 -0500 KT Walrus wrote: > I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/ about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. > While that's a nice article, nothing in it was news to me or particular complex when one does large scale stuff, like Ceph for example. > Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? > I'm sure Timo's answer will (or would, if he could be bothered) be along the lines of: "Sure, if you give me all your gold and then some for a complete rewrite of, well, everything". What you're missing and what the bad idea here is that as mentioned before scale-up only goes so far. I was feeling that my goal of 500k users/sessions in 2-node active/active cluster was quite ambitious and currently I'm looking at 200k sessions as something achievable with the current Dovecot and other limitations. But even if you were to implement something that can handle 1 million or more sessions per server, would you want to? As in, if that server goes down, the resulting packet, authentication storm will be huge and most like result in a proverbial shit storm later. Having more than 10% or so of your customers on one machine and thus involved in an outage that you KNOW will hit you eventually strikes me as a bad idea. I'm not sure how the design below meshes with Timo's lofty goals and standards when it comes to security as well. And a push with the right people (clients) to support IMAP NOTIFY would of course reduce the number of sessions significantly. Finally, Dovecot in proxy mode already scales quite well. Christian > I think the current implementation of having a separate process manage each active IMAP session (w/ the possibility of moving idling sessions to a single hibernate process) will never be able to deploy a single server managing 10 Million IMAP sessions. > > But, would it be possible to implement a new IMAP server plugin that uses a fixed configurable pool of "worker" processes, much like NGINX or PHP-FPM does. These servers can probably scale to 10 Million TCP connections, if the server is carefully tuned and has enough cores/memory to support that many active sessions. > > I?m thinking that the new IMAP server could use some external database (e.g., Redis or Memcached) to save all the sessions state and have the "worker" processes poll the TCP sockets for new IMAP commands to process (fetching the session state from the external database when it has a command that is waiting on a response). The Dovecot IMAP proxies could even queue incoming commands to proxy many incoming requests to a smaller number of backend connections (like ProxySQL does for MySQL requests). That might allow each Dovecot proxy to support 10 Million IMAP sessions and a single backend could support multiple front end Dovecot proxies (to scale to 100 Million concurrent IMAP connections using 10 proxies for 100 Million connections and 1 backend server for 10 Million connections). > > Of course, the backend server may need to be beefy and have very fast NVMe SSDs for local storage, but changing the IMAP server to manage a pool of workers instead of requiring a process per active session, would allow bigger scale up and could save large sites a lot of money. > > Is this a good idea? Or, am I missing something? > > Kevin -- Christian Balzer Network/Systems Engineer chibi at gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ From dbetz at df.eu Wed Feb 22 08:37:44 2017 From: dbetz at df.eu (Daniel Betz) Date: Wed, 22 Feb 2017 08:37:44 +0000 Subject: How to dsync mdbox compressed to maildir uncompressed Message-ID: <6bfc43d4c0ba48d59ebc11cc37f842d9@EXDAG08-1.EXCHANGE.INT> Hi Thomas, this doesnt work: doveadm -o plugin/quota= -o plugin/zlib_save= backup -u imap at test.de maildir:~/Maildir file Maildir/cur/1487752497.M128284P11684.mail,S=7691,W=7887:2,S Maildir/cur/1487752497.M128284P11684.mail,S=7691,W=7887:2,S: gzip compressed data, from Unix Regards, Daniel From tlx at leuxner.net Wed Feb 22 09:22:16 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 22 Feb 2017 10:22:16 +0100 Subject: How to dsync mdbox compressed to maildir uncompressed In-Reply-To: <6bfc43d4c0ba48d59ebc11cc37f842d9@EXDAG08-1.EXCHANGE.INT> References: <6bfc43d4c0ba48d59ebc11cc37f842d9@EXDAG08-1.EXCHANGE.INT> Message-ID: <20170222092216.GA64181@nihlus.leuxner.net> * Daniel Betz 2017.02.22 09:37: > doveadm -o plugin/quota= -o plugin/zlib_save= backup -u imap at test.de maildir:~/Maildir > > file Maildir/cur/1487752497.M128284P11684.mail,S=7691,W=7887:2,S > Maildir/cur/1487752497.M128284P11684.mail,S=7691,W=7887:2,S: gzip compressed data, from Unix Hi Daniel, I _did_ test this with a mdbox to maildir conversion and could read the maildir files as clear text. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From sami.ketola at dovecot.fi Wed Feb 22 09:39:53 2017 From: sami.ketola at dovecot.fi (Sami Ketola) Date: Wed, 22 Feb 2017 11:39:53 +0200 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> Message-ID: <476B4F1A-8882-4D42-BE4B-5A6228D6ACCB@dovecot.fi> > On 22 Feb 2017, at 6.12, Christian Balzer wrote: > > On Tue, 21 Feb 2017 09:49:39 -0500 KT Walrus wrote: > >> I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/ about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. >> > > While that's a nice article, nothing in it was news to me or particular > complex when one does large scale stuff, like Ceph for example. > >> Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? >> > I'm sure Timo's answer will (or would, if he could be bothered) be along > the lines of: > "Sure, if you give me all your gold and then some for a complete rewrite > of, well, everything?. Well. The current bottleneck in achieving that would probably be the memory amount required. With 12M active sessions (non-hibernated) the memory requirement for that 12M active user single instance server would be huge. Approximately 10TB. If 12M active sessions is the target then the architecture of one user per imap process needs to be abandoned. Sami From dbetz at df.eu Wed Feb 22 10:07:46 2017 From: dbetz at df.eu (Daniel Betz) Date: Wed, 22 Feb 2017 10:07:46 +0000 Subject: How to dsync mdbox compressed to maildir uncompressed Message-ID: <0259694797754683afac573bf538e418@EXDAG08-1.EXCHANGE.INT> Hi Thomas, thank you for your help. This is very strange that it wont work here. can you doveconf -n the relevant parts especially mail_plugins= and plugin { } i think i have misconfigured something :( Regards, Daniel From tlx at leuxner.net Wed Feb 22 10:17:14 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 22 Feb 2017 11:17:14 +0100 Subject: How to dsync mdbox compressed to maildir uncompressed In-Reply-To: <0259694797754683afac573bf538e418@EXDAG08-1.EXCHANGE.INT> References: <0259694797754683afac573bf538e418@EXDAG08-1.EXCHANGE.INT> Message-ID: <20170222101713.GA4798@nihlus.leuxner.net> * Daniel Betz 2017.02.22 11:07: Here you go: # 2.2.devel (d89a40c): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (0d78a30) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.7 auth_cache_size = 16 k auth_verbose = yes deliver_log_format = msgid=%m, time=%{delivery_time}ms, status=%$ hostname = host.domain.tld imap_hibernate_timeout = 1 mins imap_id_log = * imap_logout_format = in=%i out=%o hdr=%{fetch_hdr_count} body=%{fetch_body_count} del=%{deleted} exp=%{expunged} trash=%{trashed} login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %k session=<%{session}> mail_location = mdbox:~/mdbox mail_plugins = acl quota stats zlib virtual mailbox_list_index = yes mdbox_rotate_size = 10 M namespace { list = yes location = mdbox:/var/vmail/public:INDEXPVT=~/mdbox/public prefix = :public/ separator = / subscriptions = no type = public } namespace { list = children location = mdbox:/var/vmail/domains/%%d/%%n/mdbox:INDEXPVT=~/mdbox/shared/%%u prefix = :shared/%%u/ separator = / subscriptions = no type = shared } namespace { location = virtual:~/mdbox/virtual prefix = :virtual/ separator = / } namespace inbox { hidden = no inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = username_format=%u /var/vmail/auth.d/%d/passwd driver = passwd-file } plugin { acl = vfile:/var/vmail/conf.d/%d/global-acl:cache_secs=300 acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes imapsieve_mailbox1_before = file:/var/vmail/conf.d/domain.tld/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/vmail/conf.d/domain.tld/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * mail_log_events = expunge mailbox_delete quota = count:User quota quota_grace = 10%% quota_rule = *:storage=1GB quota_rule2 = Trash:storage=+10%% quota_status_nouser = DUNNO quota_status_success = DUNNO quota_vsizes = yes sieve = file:~/sieve;active=~/.dovecot.sieve sieve_global_dir = /var/vmail/conf.d/%d/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /var/vmail/conf.d/domain.tld/sieve sieve_plugins = sieve_imapsieve sieve_extprograms stats_refresh = 30s stats_track_cmds = yes zlib_save = gz zlib_save_level = 6 } protocols = " imap lmtp" quota_full_tempfail = yes service auth-worker { unix_listener auth-worker { user = doveauth } user = doveauth } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = doveauth } service imap-hibernate { unix_listener imap-hibernate { user = vmail } } service imap-login { inet_listener imap { address = 188.138.4.217 [2001:470:1f0b:bd0::3] port = 143 reuse_port = yes } inet_listener imaps { port = 0 } process_min_avail = 8 } service imap-postlogin { executable = script-login /var/vmail/conf.d/scripts/postlogin.sh user = vmail } service imap { executable = imap unix_listener imap-master { user = dovecot } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_ca = From wogri at wogri.com Wed Feb 22 14:51:54 2017 From: wogri at wogri.com (Wolfgang Hennerbichler) Date: Wed, 22 Feb 2017 15:51:54 +0100 Subject: Replication Troubles In-Reply-To: <95751283-49CE-4A3C-A1BF-0AC8AAEED08A@iki.fi> References: <571885AA-C2DA-4240-A0B2-F0681D347DE4@wogri.com> <95751283-49CE-4A3C-A1BF-0AC8AAEED08A@iki.fi> Message-ID: <8247506D-9EB9-4B1A-8B11-B05C5B57B74A@wogri.com> On 20 Feb 2017, at 11:09, Timo Sirainen wrote: > > On 17 Feb 2017, at 21.37, Wolfgang Hennerbichler wrote: >> >> Hi Dovecot Users, >> >> I?ve configured dovecot dsync replication and I see troubles in the logs and get user complaints which I can?t explain. I found similar threads on this mailinglist, but I couldn?t find a solution anywhere. Does anybody have dsync running without problems on a high volume mailserver? >> >> I see the following logs, examples given: >> >> Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: /var/mail/zoechi/dovecot-uidlist: Duplicate file entry at line 10395: 1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377 (uid 41092 -> 41093) - retrying by re-reading from beginning >> >> with this one I?m not sure - it might be that this is completely OK because due to replication UIDs clash. Maybe that?s OK, but I couldn?t find a confirmation. >> >> Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: Maildir /var/mail/zoechi: Expunged message reappeared, giving a new UID (old uid=41092, file=1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377) > > There seems to be something weird with using Maildir and replication. Haven't had time to debug it and it's likely not an easy bug to fix, so for now the solution would be to use only sdbox/mdbox with replication. Thank you, I can confirm that after the migration to sdbox I don?t see those errors anymore. I could not find good documentation for converting maildir to sdbox on the fly, so I wrote a script. If there?s interest I can share this. wogri From kevin at my.walr.us Wed Feb 22 15:07:45 2017 From: kevin at my.walr.us (KT Walrus) Date: Wed, 22 Feb 2017 10:07:45 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> Message-ID: <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> > On Feb 21, 2017, at 11:12 PM, Christian Balzer wrote: > > On Tue, 21 Feb 2017 09:49:39 -0500 KT Walrus wrote: > >> I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/ > about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. >> > > While that's a nice article, nothing in it was news to me or particular > complex when one does large scale stuff, like Ceph for example. > >> Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? >> > I'm sure Timo's answer will (or would, if he could be bothered) be along > the lines of: > "Sure, if you give me all your gold and then some for a complete rewrite > of, well, everything?. It will be a long time before I would need to scale to 10 Million users and I will be happy to pay for the rewrite of the IMAP plugin when the time comes, if not done before then by someone else. I have seen proposals for a new client protocol called JMAP that seem to be all about running a mail server at scale like an NGINX https web server can scale. That got me thinking about wether there is anything fundamental about IMAP that causes it to be difficult to scale. After looking into Dovecot?s current IMAP implementation, I think the approach was taken that fundamentally would have scaling issues (as in, one backend process per IMAP session). I see a couple years ago, work was done to ?migrate? idling IMAP sessions to a single process that ?remembers? the state of the IMAP session and can restore it back to a backend process when the idling is done. But, the only estimate that I have read about the ?migrate idling? is that you are likely to see only a 20% reduction of the number of concurrent processes you need if you are running at 50,000 IMAP sessions per mail server. 20% reduction is not nearly enough of a benefit for scale. I would need to see at least an order of magnitude improvement to scale (and hopefully, several orders of magnitude). So, in my mind, since these IMAP sessions are long lived with infrequent bursts of activity, a better approach would be to manage the session data in memory or in an external datastore and only process using the session data when there is activity. Much like Web Sockets and even HTTPS requests are handled today for installations that need to scale to support millions of active users. As for Dovecot, I would think the work done to ?migrate? idling IMAP sessions would be a good start to implementing managing a large number of sessions with a fixed pool of worker processes like other web servers do. So, my question really is: Is there anything about the IMAP protocol that would prevent an implementation from scaling to 10 Million users per server? Or, do we need to push for a new protocol like JMAP that has been designed to scale better (by being stateless with the server requests)? Kevin From kevin at my.walr.us Wed Feb 22 15:24:30 2017 From: kevin at my.walr.us (KT Walrus) Date: Wed, 22 Feb 2017 10:24:30 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> Message-ID: <2B812188-36DF-4919-94DA-67F5A316751E@my.walr.us> > On Feb 21, 2017, at 11:12 PM, Christian Balzer wrote: > But even if you were to implement something that can handle 1 million or > more sessions per server, would you want to? > As in, if that server goes down, the resulting packet, authentication > storm will be huge and most like result in a proverbial shit storm later. > Having more than 10% or so of your customers on one machine and thus > involved in an outage that you KNOW will hit you eventually strikes me as > a bad idea. The idea would be to store session state in an external database like Redis. I use Redis for PHP session data on the web servers and Redis is implemented as a high-availability cluster (using Redis Sentinels). If the IMAP session state is maintained externally in a high-availability datastore, then rebooting a mail server or having it go down unexpectedly should not mean that all existing sessions are ?kicked? and the clients would need to log in again. Rather, a backup mail server or servers could take the load and just use the high-availability datastore to manage the sessions that were on the old server. One potential problem, if not using shared storage for the mailboxes, is that dovecot replication is asynchronous so a small number of IMAP sessions might be out of date with the data on the replacement server, so some of the data in Redis might need to be re-cached to reflect the state of the backup mailstore. Other than that, I don?t think there would be much of a "proverbial shit storm? caused by the failure of one mail server, even if that server were to handle 1 million or more sessions per server. The remaining mail servers in the cluster would need to be able to absorb the load (maybe cluster in 3 server clusters would be the norm so each remaining server would only have to be able to take 50% of the sessions from the failed server while it is unavailable). Kevin From basdove at rediffmail.com Wed Feb 22 15:32:12 2017 From: basdove at rediffmail.com (Basdove) Date: Wed, 22 Feb 2017 21:02:12 +0530 Subject: How to add Ubuntu 16.04.2 server to join and authenticate with domain controller windows2012r2 Message-ID: Hi, 1. I have Ubuntu 16.0.4.2 server edition 2. I have windows server 2012R2 domain controller (Active directory installed) 3. I want to join and Authenticate Ubuntu with windows server 2012 domain controller 4. Please provide the clean document Thank you, Baskaranand From tss at iki.fi Wed Feb 22 19:18:34 2017 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2017 21:18:34 +0200 Subject: v2.2.28 release candidate 2 released Message-ID: <4F0D45B1-965A-4110-881C-0AAD11604BF6@iki.fi> http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc2.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc2.tar.gz.sig I'm assuming that most of the bugs are now found and fixed, so the final 2.2.28 should be out in a day or two. Changes since rc1: * Reverted [UNKNOWN-CTE] -> [PARSE] change + pop3c: Added pop3c_features=no-pipelining setting to prevent using PIPELINING extension even though it's advertised. - lmtp_user_concurrency_limit didn't work if userdb changed username - Fixed various bugs that were introduced in rc1 Here's the full list of changes again: * director: "doveadm director move" to same host now refreshes user's timeout. This allows keeping user constantly in the same backend by just periodically moving the user there. * When new mailbox is created, use initially INBOX's dovecot.index.cache caching decisions. * Expunging mails writes GUID to dovecot.index.log now only if the GUID is quickly available from index/cache. * pop3c: Increase timeout for PASS command to 5 minutes. * Mail access errors are no longer ignored when searching or sorting. With IMAP the untagged SEARCH/SORT reply is still sent the same as before, but NO reply is returned instead of OK. + Make dovecot.list.index's filename configurable. This is needed when there are multiple namespaces pointing to the same mail root (e.g. lazy_expunge namespace for mdbox). + Add size.virtual to dovecot.index when folder vsizes are accessed (e.g. quota=count). This is mainly a workaround to avoid slow quota recalculation performance when message sizes get lost from dovecot.index.cache due to corruption or some other reason. + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them in lib-dsasl for client side. + auth: Support filtering by SASL mechanism: passdb { mechanisms } + Shrink the mail processes' memory usage by not storing settings duplicated unnecessarily many times. + imap: Add imap_fetch_failure setting to control what happens when FETCH fails for some mails (see example-config). + imap: Include info about last command in disconnection log line. + imap: Created new SEARCH=X-MIMEPART extension. It's currently not advertised by default, since it's not fully implemented. + fts-solr: Add support for basic authentication. + Cassandra: Support automatically retrying failed queries if execution_retry_interval and execution_retry_times are set. + doveadm: Added "mailbox path" command. + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the wanted fields only if it doesn't require opening the email. + mail_vsize_bg_after_count setting added (see example-config). + mail_sort_max_read_count setting added (see example-config). + pop3c: Added pop3c_features=no-pipelining setting to prevent using PIPELINING extension even though it's advertised. - Index files: day_first_uid wasn't updated correctly since v2.2.26. This caused dovecot.index.cache to be non-optimal. - imap: SEARCH/SORT may have assert-crashed in client_check_command_hangs - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. - imap: Running time in tagged command reply was often wrongly 0. - search: Using NOT n:* or NOT UID n:* wasn't handled correctly - director: doveadm director kick was broken - director: Fix crash when using director_flush_socket - director: Fix some bugs when moving users between backends - imapc: Various error handling fixes and improvements - master: doveadm process status output had a lot of duplicates. - autoexpunge: If mailbox's rename timestamp is newer than mail's save-timestamp, use it instead. This is useful when autoexpunging e.g. Trash/* and an entire mailbox is deleted by renaming it under Trash to prevent it from being autoexpunged too early. - autoexpunge: Multiple processes may have been trying to expunge the same mails simultaneously. This was problematic especially with lazy_expunge plugin. - auth: %{passdb:*} was empty in auth-worker processes - auth-policy: hashed_password was always sent empty. - dict-sql: Merge multiple UPDATEs to a single statement if possible. - fts-solr: Escape {} chars when sending queries - fts: fts_autoindex_exclude = \Special-use caused crashes - doveadm-server: Fix leaks and other problems when process is reused for multiple requests (service_count != 1) - sdbox: Fix assert-crash on mailbox create race - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve was used. especially %{storage_id} was broken. - lmtp_user_concurrency_limit didn't work if userdb changed username From tlx at leuxner.net Wed Feb 22 19:40:42 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 22 Feb 2017 20:40:42 +0100 Subject: Sieve removeflag Action In-Reply-To: <20170220115644.GA14379@nihlus.leuxner.net> References: <20170113081514.GA60507@nihlus.leuxner.net> <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> <20170220115644.GA14379@nihlus.leuxner.net> Message-ID: <20170222194041.GA3860@nihlus.leuxner.net> * Thomas Leuxner 2017.02.20 12:56: > Feb 20 07:00:23 nihlus dovecot: master: Dovecot v2.2.devel (8f42a89) starting up for imap, lmtp > > This one processed the dovecot-news mail for 2.2.28.rc1 fine which uses a similar sieve rule. I will monitor global rules with this build and report back. The results seem to be arbitrary. Personal rules _always_ work for flag actions, global rules included do _sometimes_. This means one day the same rule may trigger, the next day it won't. I found no way to reproduce it for global rules in order to narrow down the issue. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From tss at iki.fi Wed Feb 22 19:44:44 2017 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2017 21:44:44 +0200 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> Message-ID: <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> On 22 Feb 2017, at 17.07, KT Walrus wrote: > > I have seen proposals for a new client protocol called JMAP that seem to be all about running a mail server at scale like an NGINX https web server can scale. That got me thinking about wether there is anything fundamental about IMAP that causes it to be difficult to scale. After looking into Dovecot?s current IMAP implementation, I think the approach was taken that fundamentally would have scaling issues (as in, one backend process per IMAP session). I see a couple years ago, work was done to ?migrate? idling IMAP sessions to a single process that ?remembers? the state of the IMAP session and can restore it back to a backend process when the idling is done. > > But, the only estimate that I have read about the ?migrate idling? is that you are likely to see only a 20% reduction of the number of concurrent processes you need if you are running at 50,000 IMAP sessions per mail server. 20% reduction is not nearly enough of a benefit for scale. I would need to see at least an order of magnitude improvement to scale (and hopefully, several orders of magnitude). My long-term plans are something like this: * imap-hibernate process can be used more aggressively. Not necessarily even for just IDLEing sessions, but for any session that isn't actively being used. And actually if the server is too busy, even active sessions could be hibernated. That would be somewhat similar to cooperative multitasking. When this is done, you can think of the current imap processes as the worker processes. * More state will be transferred to imap-hibernate process, so it can perform simpler commands without recreating the IMAP process. For example STATUS replies can be returned from cached state as long as it hasn't actually changed. * imap-hibernate is currently tracking changed state via inotify (etc.) This mostly work, but it's also unnecessarily sometimes waking up. For example just because one IMAP session performed a FETCH that added something to dovecot.index.cache, it doesn't mean that there are any real changes. We'll need some mail plugin that notifies imap-hibernate process when some real change has happened. * Hibernated sessions can even be moved away entirely from backends into IMAP proxies. The IMAP proxy can then reconnect to backend to re-establish the session. This allows even switching backends entirely, as long as the storage is shared. This requires that backends notify the proxy whenever something changes to the user, which is mostly a continuation of the previous item (just TCP notification instead of UNIX socket notification). * IMAP proxies can also perform similar limited functionality as imap-hibernate processes. Possibly running the same imap-hibernate processes. * And kind of a reverse of hibernation: imap processes can also preserve the user's imap session and opened folder indexes in memory even after the IMAP client has disconnected. If the same user connects back, the imap process can quickly be re-used with all the state already open. This is especially useful for client that create many short-lived connections, such as webmails. So after all these changes there would practically be something like 1000 imap processes constantly open and either doing work or waiting for a recently disconnected IMAP client to come back. As Christian already mentioned, the Dovecot proxies are supposed to be able to handle quite a lot of connections. I wouldn't be surprised if you can already do millions of connections with them. Most of our customers haven't tried scaling them very hard because they don't really want to create multiple IP addresses for servers, which is required to avoid running out of TCP ports (or I guess there could be multiple destination ports, but that also complicates things and Dovecot doesn't currently support that in an easy way either). > Is there anything about the IMAP protocol that would prevent an implementation from scaling to 10 Million users per server? Or, do we need to push for a new protocol like JMAP that has been designed to scale better (by being stateless with the server requests)? I guess mainly the message sequence numbers in IMAP protocol makes this more difficult, but it's not an impossible problem to solve. From kevin at my.walr.us Wed Feb 22 20:46:08 2017 From: kevin at my.walr.us (KT Walrus) Date: Wed, 22 Feb 2017 15:46:08 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> Message-ID: <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> > On Feb 22, 2017, at 2:44 PM, Timo Sirainen wrote: > > I guess mainly the message sequence numbers in IMAP protocol makes this more difficult, but it's not an impossible problem to solve. Any thoughts on the wisdom of supporting an external database for session state or even mailbox state (like using Redis or even MySQL)? Also, would it help reliability or scalability to store a copy of the index data in an external database? I want to use mdbox format but I have heard that these index files do get corrupted occasionally and have to be rebuilt (possibly using an older version of the index file to construct a new one). I worry that using mdbox might cause my users to see the IMAP flags suddenly reset back to a previous state (like seeing previously read messages becoming unread in their mail clients). If a copy of the index data were stored in an external database, such problems of duplicate messages occurring in a dovecot cluster could be handled by having the cluster ?lookup? the index data using the external database instead of the local copy stored on the server. An external database could easily implement unique serial numbers cluster-wide. In the site I?m working on building, I even use Redis to implement ?message queues? between Postfix and Dovecot (via redis push/pop feature). Currently, I am only delivering new messages via IMAP instead of LMTP (no LMTP will be available to my backend mail servers, only IMAP). If you stored the MD5 checksum of the index files (and even the message files) in the external database, you could also run a background process that would periodically check for corruption of the local index files using the checksums from the database, making mdbox format even more bulletproof. And, the best thing about using an external database is that making the external database highly available is not a problem (as most sites already do that). The index data stored in the database would become the ?source of truth? with the local index files/session data being an efficient cache for the mailstore. And, re-caching could occur as needed to make the whole cluster more reliable. Kevin From kremels at kreme.com Thu Feb 23 06:15:23 2017 From: kremels at kreme.com (@lbutlr) Date: Wed, 22 Feb 2017 23:15:23 -0700 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> References: <1006495667.1617258.1487367509816@mail.yahoo.com> <1b4b5a90-df4c-cbd4-5c51-c47ecf57fa7a@domblogger.net> Message-ID: <9EEBFBE0-95D1-49D3-8AE3-9589FEEC1012@kreme.com> On 19 Feb 2017, at 00:00, Michael A. Peters wrote: > That's one of the reasons I don't like Let's Encrypt, with one year certs it is easier to look at the certs and see what is going to expire in the coming month needing a new private key. Since renewal is entirely automatic, when certs expire is meaningless. There is no need to check unless a cert has NOT renewed, which you are notified of so you have time to figure out what you broke. -- Apple broke AppleScripting signatures in Mail.app, so no random signatures. From ruga at protonmail.com Thu Feb 23 07:33:21 2017 From: ruga at protonmail.com (Ruga) Date: Thu, 23 Feb 2017 02:33:21 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> Message-ID: Comparison of Dovecot, Uwash, Courier, Cyrus and M-Box: http://www.isode.com/whitepapers/mbox-benchmark.html From skdovecot at smail.inf.fh-brs.de Thu Feb 23 07:48:20 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 23 Feb 2017 08:48:20 +0100 (CET) Subject: How to add Ubuntu 16.04.2 server to join and authenticate with domain controller windows2012r2 In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 22 Feb 2017, Basdove wrote: > 1. I have Ubuntu 16.0.4.2 server edition > > 2. I have windows server 2012R2 domain controller (Active directory > installed) > > 3. I want to join and Authenticate Ubuntu with windows server 2012 domain > controller > > 4. Please provide the clean document http://lmgtfy.com/?q=ubuntu+join+windows+ad How is this question related to Dovecot? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWK6TxHz1H7kL/d9rAQKdVAf/cEtO1icmE/fdB05SPloYKOy9T5otMJxN nn1dCmJoop4U9OhM9YOMZJYePnwIgCGqF7cRTih8u5pPmQIpFeNAKmIfrLHmk2Jl GuGdvbq3GHyH3kpfUuLsjhJ3p+x4hI9/xDSVmEc16C9606LLzrf+/JK89JHlHxh9 4tzvDUrYU8CYQGebi4RVXIRpnb128xfubFNY6+Tbumg+GoeeubUfOrSgTbfj5IG+ CZ9LHzKqXQcyE5AkOFzv5p2NQCViNskDKtzhEqmKVNVvLAzkR/fC/d2qpLPvAVp4 dkfNNn6PWa5rbpvYViTyH720Q0OsIawmznZf3IIcn9fF7R8J7Zex2w== =v/cc -----END PGP SIGNATURE----- From dovecot at r.paypc.com Thu Feb 23 07:52:45 2017 From: dovecot at r.paypc.com (M. Balridge) Date: Wed, 22 Feb 2017 23:52:45 -0800 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> Message-ID: <1487836365.58ae94cd89d773.47871896@www.paypc.com> Quoting Ruga : > Comparison of Dovecot, Uwash, Courier, Cyrus and M-Box: > http://www.isode.com/whitepapers/mbox-benchmark.html Wow. That comparison is only 11.5 years old. The "default" file system of reiserfs and gcc-3.3 were dead giveaways. I suspect Dovecot's changed a tad since that test. =R= From mail at ojkastl.de Thu Feb 23 08:26:06 2017 From: mail at ojkastl.de (Johannes Kastl) Date: Thu, 23 Feb 2017 09:26:06 +0100 Subject: Problem with Let's Encrypt Certificate In-Reply-To: <64ead80c-795c-5674-5576-cb15de9f8ac8@linet-services.de> References: <67651323.902.1487365112484@appsuite-dev.open-xchange.com> <64ead80c-795c-5674-5576-cb15de9f8ac8@linet-services.de> Message-ID: <620f4839-ecc4-fcab-2636-fe6c287f5b55@ojkastl.de> On 17.02.17 22:57 Bastian Sebode wrote: > Finally I found the issue! :-) But I still have no idea why the > problem happens with Thunderbird. > > I used dehydrated to fetch the certificates from Let's Encrypt and > as I said, it works for most clients pretty well. (Tried: Mulberry, > Claws Mail, Outlook 2010, Android (HTC), iPhone, ...) Also it works > perfectly with all my HTTPS-Services > > Whatever, Thunderbird didn't like that cert saying "bad > certificate" (SSL Alert 42). > > Now I fetched the cert with Certbot and it works. Really strange > though! Have you contacted the author of dehydrated? Either this is a bug in the program, that should be fixed. Or it is an error in the programs configuration (yours or the default), and that should be fixed, too. I am just setting up a dovecot installation and was planning on using letsencrypt with dehydrated, too. ;-) Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 244 bytes Desc: OpenPGP digital signature URL: From edgaras.lukosevicius at gmail.com Thu Feb 23 08:30:50 2017 From: edgaras.lukosevicius at gmail.com (=?UTF-8?Q?Edgaras_Luko=c5=a1evi=c4=8dius?=) Date: Thu, 23 Feb 2017 10:30:50 +0200 Subject: imap/pop3-login assertion failed in dovecot 2.2.27 (c0f36b0) Message-ID: Hello, I have noticed few errors like this. I don't recall seeing that with version 2.2.10 Feb 23 05:20:03 mail21 dovecot[6569]: imap-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 05:50:03 mail21 dovecot[15044]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 05:50:03 mail21 dovecot[15044]: imap-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 06:30:04 mail21 dovecot[13266]: imap-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 06:30:04 mail21 dovecot[13266]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 07:10:04 mail21 dovecot[14298]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 07:10:04 mail21 dovecot[14298]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 07:10:04 mail21 dovecot[14298]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) From kremels at kreme.com Thu Feb 23 09:32:35 2017 From: kremels at kreme.com (@lbutlr) Date: Thu, 23 Feb 2017 02:32:35 -0700 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> Message-ID: On 23 Feb 2017, at 00:33, Ruga wrote: > Comparison of Dovecot, Uwash, Courier, Cyrus and M-Box: > http://www.isode.com/whitepapers/mbox-benchmark.html Uwash? as in UW IMAP that I used briefly in 1999? That hasn't seen an update in a decade? -- Apple broke AppleScripting signatures in Mail.app, so no random signatures. From ruga at protonmail.com Thu Feb 23 13:24:20 2017 From: ruga at protonmail.com (Ruga) Date: Thu, 23 Feb 2017 08:24:20 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <1487836365.58ae94cd89d773.47871896@www.paypc.com> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> <1487836365.58ae94cd89d773.47871896@www.paypc.com> Message-ID: Yes, and they (isode) still use it as marketing evidence. The benchmarking tool project also seems out of maintenance. Sent from ProtonMail Mobile On Thu, Feb 23, 2017 at 8:52 AM, M. Balridge <'dovecot at r.paypc.com'> wrote: Quoting Ruga : > Comparison of Dovecot, Uwash, Courier, Cyrus and M-Box: > http://www.isode.com/whitepapers/mbox-benchmark.html Wow. That comparison is only 11.5 years old. The "default" file system of reiserfs and gcc-3.3 were dead giveaways. I suspect Dovecot's changed a tad since that test. =R= From bytesplit at gmail.com Thu Feb 23 14:20:29 2017 From: bytesplit at gmail.com (Philon) Date: Thu, 23 Feb 2017 15:20:29 +0100 Subject: How to add Ubuntu 16.04.2 server to join and authenticate with domain controller windows2012r2 In-Reply-To: References: Message-ID: Let me make a guess and argument he wanted to authenticate Dovecot against AD... Then there is this article in the wiki: http://wiki2.dovecot.org/HowTo/ActiveDirectoryNtlm Should do as requested clean document...!? Philon Am 23.02.2017 08:48, schrieb Steffen Kaiser: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 22 Feb 2017, Basdove wrote: > >> 1. I have Ubuntu 16.0.4.2 server edition >> >> 2. I have windows server 2012R2 domain controller (Active directory >> installed) >> >> 3. I want to join and Authenticate Ubuntu with windows server 2012 >> domain controller >> >> 4. Please provide the clean document > > http://lmgtfy.com/?q=ubuntu+join+windows+ad > > How is this question related to Dovecot? > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBWK6TxHz1H7kL/d9rAQKdVAf/cEtO1icmE/fdB05SPloYKOy9T5otMJxN > nn1dCmJoop4U9OhM9YOMZJYePnwIgCGqF7cRTih8u5pPmQIpFeNAKmIfrLHmk2Jl > GuGdvbq3GHyH3kpfUuLsjhJ3p+x4hI9/xDSVmEc16C9606LLzrf+/JK89JHlHxh9 > 4tzvDUrYU8CYQGebi4RVXIRpnb128xfubFNY6+Tbumg+GoeeubUfOrSgTbfj5IG+ > CZ9LHzKqXQcyE5AkOFzv5p2NQCViNskDKtzhEqmKVNVvLAzkR/fC/d2qpLPvAVp4 > dkfNNn6PWa5rbpvYViTyH720Q0OsIawmznZf3IIcn9fF7R8J7Zex2w== > =v/cc > -----END PGP SIGNATURE----- From aki.tuomi at dovecot.fi Thu Feb 23 17:50:04 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Thu, 23 Feb 2017 19:50:04 +0200 Subject: Pigeonhole External pipe script going zombie? In-Reply-To: References: Message-ID: <00576f9c-ce60-1de7-6ed3-86c8dcf13034@dovecot.fi> On 2017-01-12 21:55, Matt Simpson wrote: > I?m running dovecot 2.2.27 and pigeonhole 0.4.16 on FreeBSD 11. > > I?m using the pigeonhole/sieve external pipe plugin to run a Perl program to send a Pushover notification when certain messages are received. > > The Perl script is executed, and the notification is sent. But then the script task seems to go zombie until it is killed after a timeout. > > In the user?s sieve log, I get a message like > > error: msgid=<20170112191921.66140.qmail at v1.redhorse.me>: pipe action: failed to pipe message to program `sievepush.pl': refer to server log for more information. [2017-01-12 14:19:36]. > > (even though the message really was piped to the program successfully) > > In the dovecot server log, I see > > Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: sieve: Executing script from `/usr/home/matt/maildoms/.dovecot.svbin' > Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: sieve: action pipe: running program: sievepush.pl > Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: Mailbox stdin: Opened mail UID=1 because: mail stream > Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: waiting for program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl' to finish after 0 msecs > Jan 12 14:19:31 v1 dovecot: lda(matt): Debug: program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl'(66145) execution timed out after 10000 milliseconds: sending TERM signal > Jan 12 14:19:36 v1 dovecot: lda(matt): Debug: program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl' (66145) did not die after 5000 milliseconds: sending KILL signal > > In the process list during that 10 second interval, I see > > matt 66142 29972 801 801 0 S - 0:00.00 bin/qmail-local -- matt /home/matt/maildoms jmn-matt - jmn-m > matt 66143 66142 801 801 0 S - 0:00.00 /var/qmail/bin/preline -f /usr/local/libexec/dovecot/dovecot > matt 66144 66143 801 801 0 S - 0:00.01 /usr/local/libexec/dovecot/dovecot-lda > matt 66145 66144 801 801 0 Z - 0:00.65 > > I?m not a Unix programming ace, but from what I?ve been able to find out, this seems to mean that the lda process is forking another process to run the pipe script, and not getting the proper notification when it finishes (not issuing a wait?). So after 10 seconds, it sends a TERM to the task which is no longer running, and when that doesn?t work, it sends a KILL. Anybody know what?s happening here? > > doveconf -n > # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.16 (fed8554) > # OS: FreeBSD 11.0-RELEASE-p2 amd64 > auth_verbose = yes > default_vsz_limit = 128 M > lock_method = flock > mail_debug = yes > mail_location = maildir:~/Maildir > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe vnd.dovecot.execute > namespace inbox { > inbox = yes > location = > prefix = > } > passdb { > args = imap > driver = pam > } > plugin { > recipient_delimiter = - > sieve = file:~/sieve;active=~/.dovecot.sieve > sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve-pipe > sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute > sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe > sieve_pipe_exec_timeout = 10s > sieve_plugins = sieve_extprograms > } > protocols = imap > service auth { > unix_listener auth-master { > group = qnofiles > mode = 0660 > user = alias > } > user = root > } > service imap-login { > process_min_avail = 3 > vsz_limit = 94 M > } > ssl_cert = ssl_key = # hidden, use -P to show it > syslog_facility = local0 > userdb { > driver = passwd > } > verbose_proctitle = yes > protocol lda { > mail_plugins = " sieve" > } Hi! I tried reproducing your issue, but unfortunately using the exact version you are using yielded nothing useful. Feb 23 19:47:45 lda(testuser1): Debug: program `/home/cmouse/dovecot22/sieve-pipe/test-program.pl'(8691) execution timed out after 10000 milliseconds: sending TERM signal Feb 23 19:47:45 lda(testuser1): Error: program `/home/cmouse/dovecot22/sieve-pipe/test-program.pl' was forcibly terminated with signal 15 ... Feb 23 19:47:45 lda(testuser1): Info: sieve: Execution of script /home/cmouse/dovecot22/home/vmail/testuser1/.dovecot.sieve failed, but implicit keep was successful (user logfile /home/cmouse/dovecot22/home/vmail/testuser1/.dovecot.sieve.log may reveal additional details) Do you think you could try if v2.2.28rc2 fixes the problem? Aki From dclist at list.jmatt.net Thu Feb 23 18:43:10 2017 From: dclist at list.jmatt.net (Matt Simpson) Date: Thu, 23 Feb 2017 13:43:10 -0500 Subject: Pigeonhole External pipe script going zombie? In-Reply-To: <00576f9c-ce60-1de7-6ed3-86c8dcf13034@dovecot.fi> References: <00576f9c-ce60-1de7-6ed3-86c8dcf13034@dovecot.fi> Message-ID: <414937BC-A64B-44BE-ABC1-E67D3C7A8B8E@list.jmatt.net> > On Feb 23, 2017, at 12:50 PM, Aki Tuomi wrote: > > I tried reproducing your issue, but unfortunately using the exact version you are using yielded nothing useful. > > Do you think you could try if v2.2.28rc2 fixes the problem? That wouldn?t be easy. Dovecot is installed on my system via FreeBSD package, not compiled from source. Installing a test version that isn?t released as a package would be a little inconvenient. From kevin at my.walr.us Thu Feb 23 20:43:55 2017 From: kevin at my.walr.us (KT Walrus) Date: Thu, 23 Feb 2017 15:43:55 -0500 Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: > On Feb 20, 2017, at 4:01 PM, Joseph Tam wrote: > > yacinechaouche at yahoo.com writes: > >> Interesting. Is there any particular benefit in having only one file >> for both certificate and private key ? I find that putting private key >> in a separate file feels more secure. > > It's convenient to have key and cert in one place if you don't need > the certificate to be publically readable. Keeping it in separate > files would add slightly more security (defense in depth), that would > protect from, for example, an admin fumble or bug in the SSL library. > > "Michael A. Peters" writes: > >>> I use dehydrated (with Cloudflare DNS challenges) and as far as I know, >>> it seems to generate a new private key every time. >> >> Yeah that would be a problem for me because I implement DANE. > > It's on my to-do list, but I think you can use dehydrated in signing > mode. > > --signcsr (-s) path/to/csr.pem Sign a given CSR, output CRT on stdout (advanced usage) > > In this way, you can reuse private key, as well as making it more > secure by removing a privileged operations (private key acces) allowing > dehydrated to be run as a non-privilged/separate user. You might want to check out this blog: http://www.internetsociety.org/deploy360/blog/2016/03/lets-encrypt-certificates-for-mail-servers-and-dane-part-2-of-2/ The author outlines a procedure for using DANE and Let?s Encrypt automatically generated certs in production. I don?t really know much about DANE, but those wanting to implement it with free certs might want to check out this blog. Kevin From tss at iki.fi Thu Feb 23 21:00:51 2017 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2017 23:00:51 +0200 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> Message-ID: <8A7DE969-EAAD-4B27-81F9-B6F8772A0388@iki.fi> On 22 Feb 2017, at 22.46, KT Walrus wrote: > >> On Feb 22, 2017, at 2:44 PM, Timo Sirainen wrote: >> >> I guess mainly the message sequence numbers in IMAP protocol makes this more difficult, but it's not an impossible problem to solve. > > Any thoughts on the wisdom of supporting an external database for session state or even mailbox state (like using Redis or even MySQL)? > > Also, would it help reliability or scalability to store a copy of the index data in an external database? I mainly see such external databases as additional reasons for things to break. And even if not, additional extra layers of latency. The thoughts I've had about storing such internal state in the Dovecot Proxy layer make sense because the IMAP sessions have to have active TCP connections. All the state can be stored by the process that is responsible for the TCP connection itself. There's not much point storing such state outside the process: If the process or the TCP connection dies, the state needs to be forgotten about in any case since there's no "state resume" command in IMAP (and even if there were, the state probably should then be stored in that command itself rather than on the server side). > I want to use mdbox format but I have heard that these index files do get corrupted occasionally and have to be rebuilt (possibly using an older version of the index file to construct a new one). I worry that using mdbox might cause my users to see the IMAP flags suddenly reset back to a previous state (like seeing previously read messages becoming unread in their mail clients). Both sdbox and mdbox formats have this problem in theory. Practically, there are many huge mdbox/sdbox installations and I don't think they see such problems much, if ever. Dovecot attempts pretty hard already not to lose flags with sdbox/mdbox. There are also separate dovecot.index.backup files that are kept just for this purpose. > If a copy of the index data were stored in an external database, such problems of duplicate messages occurring in a dovecot cluster could be handled by having the cluster ?lookup? the index data using the external database instead of the local copy stored on the server. This sounds a bit similar to the "obox" format that we use for storing emails and indexes to object storage in Dovecot Pro. That isn't open source though.. > If you stored the MD5 checksum of the index files (and even the message files) in the external database, you could also run a background process that would periodically check for corruption of the local index files using the checksums from the database, making mdbox format even more bulletproof. I don't see why this would need an external database. I've long had in my TODO to add hashes/checksums to all of the Dovecot index files so it could properly detect corruption and ignore that. Hopefully that's not too far into the future anymore. > And, the best thing about using an external database is that making the external database highly available is not a problem (as most sites already do that). The index data stored in the database would become the ?source of truth? with the local index files/session data being an efficient cache for the mailstore. And, re-caching could occur as needed to make the whole cluster more reliable. In my opinion external database is just shifting the problem from one place to another. Yes, sometimes it's still useful. Dovecot supports all kinds of databases for all kinds of purposes, like with dict API you can access LDAP, SQL or Cassanda. I mostly like Cassandra nowadays, but it has its problems as well (tombstones). I'm not aware of any highly available database that actually scales and really just works without problems. (I'm talking about clusters with more than just 2 servers. Ideally more than just 2 datacenters.) From tss at iki.fi Thu Feb 23 21:21:01 2017 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2017 23:21:01 +0200 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <8A7DE969-EAAD-4B27-81F9-B6F8772A0388@iki.fi> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> <8A7DE969-EAAD-4B27-81F9-B6F8772A0388@iki.fi> Message-ID: <60F4C810-01FC-4B34-8074-E44AB12E335A@iki.fi> On 23 Feb 2017, at 23.00, Timo Sirainen wrote: > > I mainly see such external databases as additional reasons for things to break. And even if not, additional extra layers of latency. Oh, just thought that I should clarify this and I guess other things I said. I think there are two separate things we're possibly talking about in here: 1) Temporary state: This is what I was mainly talking about. State related to a specific IMAP session. This doesn't take much space and can be stored in the proxy's memory since it's specific to the TCP session anyway. 2) Permanent state: This is mainly about the storage. A lot of people use Dovecot with NFS. So one possibility for storing the permanent state is NFS. Another possibility with Dovecot Pro is to store it to object storage as blobs and keep a local cache of the state. A 3rd possibility might be to use some kind of a database for storing the permanent state. I'm fine with the first two, but with 3rd I see a lot of problems and not a whole lot of benefit. But if you think of the databases (or even NFS) as blob storage, you can think of them the same as any object storage and use the same obox format with them. What I'm mainly against is attempting to create some kind of a database that has structured format like (imap_uid, flags, ...) - I'm sure that can be useful for various purposes but performance or scalability isn't one of them. From jtam.home at gmail.com Thu Feb 23 21:55:54 2017 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 23 Feb 2017 13:55:54 -0800 (PST) Subject: Problem with Let's Encrypt Certificate In-Reply-To: References: Message-ID: On Thu, 23 Feb 2017, KT Walrus wrote: >> It's on my to-do list, but I think you can use dehydrated in signing >> mode. >> >> --signcsr (-s) path/to/csr.pem Sign a given CSR, output CRT on stdout (advanced usage) >> >> In this way, you can reuse private key, as well as making it more >> secure by removing a privileged operations (private key acces) allowing >> dehydrated to be run as a non-privilged/separate user. > > You might want to check out this blog: > > http://www.internetsociety.org/deploy360/blog/2016/03/lets-encrypt-certificates-for-mail-servers-and-dane-part-2-of-2/ This was exactly the type of procedure I wanted: persistent key that can be protected. > The author outlines a procedure for using DANE and Let?s Encrypt > automatically generated certs in production. I don?t really know much > about DANE, but those wanting to implement it with free certs might > want to check out this blog. I don't use DANE either, but it looks fraught with stale-cache peril. If DANE with rotating keys is your thing, I would lower the DANE record TTL to something small like 60s one TTL period before cert renewal, then set it back after cert renewal. Some DNS software will auto-decrement TTL to expire at a certin time, then transition to the new definition. Joseph Tam From moseleymark at gmail.com Thu Feb 23 22:08:55 2017 From: moseleymark at gmail.com (Mark Moseley) Date: Thu, 23 Feb 2017 14:08:55 -0800 Subject: Director+NFS Experiences Message-ID: As someone who is about to begin the process of moving from maildir to mdbox on NFS (and therefore just about to start the 'director-ization' of everything) for ~6.5m mailboxes, I'm curious if anyone can share any experiences with it. The list is surprisingly quiet about this subject, and articles on google are mainly just about setting director up. I've yet to stumble across an article about someone's experiences with it. * How big of a director cluster do you use? I'm going to have millions of mailboxes behind 10 directors. I'm guessing that's plenty. It's actually split over two datacenters. In the larger, we've got about 200k connections currently, so in a perfectly-balanced world, each director would have 20k connections on it. I'm guessing that's child's play. Any good rule of thumb for ratio of 'backend servers::director servers'? In my larger DC, it's about 5::1. * Do you use the perl poolmon script or something else? The perl script was being weird for me, so I rewrote it in python but it basically does the exact same things. * Seen any issues with director? In testing, I managed to wedge things by having my poolmon script running on all the cluster boxes (I think). I've since rewritten it to run *only* on the lowest-numbered director. When it wedged, I had piles (read: hundreds per second) of log entries that said: Feb 12 06:25:03 director: Warning: director(10.1.20.5:9090/right): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.5:9090/right): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Because it was in testing, I didn't notice it and it was like this for several days till dovecot was restarted on all the director nodes. I'm not 100% on what happened, but my *guess* is that two boxes tried to update the status of the same backend server in rapid succession. * Assuming you're using NFS, do you still see non-trivial amounts of indexes getting corrupted? * Again, assuming NFS and assuming at least some corrupted indexes, what's your guess for success rate % for dovecot recovering them automatically? And how about success rate % for ones that dovecot wasn't able to do automatically but you had to use doveadm to repair it? Really what I'm trying to figure out is 1) how often sysops will need to manually recover indexes; and 2) how often admins *can't* manually recover indexes? * if you have unrecoverable indexes (and assuming you have snapshots on your NFS server), does grabbing the most recent indexes from the snapshots always work for recovery (obviously, up till the point that the snapshot was taken)? * Any gotchas you've seen anywhere in a director-fied stack? I realize that's a broad question :) * Does one of your director nodes going down cause any issues? E.g. issues with the left and right nodes syncing with each other? Or when the director node comes back up? * Does a backend node going down cause a storm of reconnects? In the time between deploying director and getting mailboxes converted to mdbox, reconnects for us will mean cold local-disk dovecot caches. But hopefully consistent hashing helps with that? * Do you have consistent hashing turned on? I can't think of any reason not to have it turned on, but who knows * Any other configuration knobs (including sysctl) that you needed to futz with, vs the default? I appreciate any feedback! From kevin at my.walr.us Thu Feb 23 22:28:42 2017 From: kevin at my.walr.us (KT Walrus) Date: Thu, 23 Feb 2017 17:28:42 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <60F4C810-01FC-4B34-8074-E44AB12E335A@iki.fi> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> <8A7DE969-EAAD-4B27-81F9-B6F8772A0388@iki.fi> <60F4C810-01FC-4B34-8074-E44AB12E335A@iki.fi> Message-ID: <405474FD-FD44-4364-9D12-5811C28EFAB4@my.walr.us> > On Feb 23, 2017, at 4:21 PM, Timo Sirainen wrote: > > On 23 Feb 2017, at 23.00, Timo Sirainen wrote: >> >> I mainly see such external databases as additional reasons for things to break. And even if not, additional extra layers of latency. > > Oh, just thought that I should clarify this and I guess other things I said. I think there are two separate things we're possibly talking about in here: > > 1) Temporary state: This is what I was mainly talking about. State related to a specific IMAP session. This doesn't take much space and can be stored in the proxy's memory since it's specific to the TCP session anyway. Moving the IMAP session state to the proxy so the backend can just have a fixed pool of worker processes is really what I think is necessary for scaling to millions of IMAP sessions. I still think it would be best to store this state in a way that you could at least ?remember? the backend server that is implementing the IMAP session and the auth data. To me, that would be to use Redis for session state. Redis is a very efficient in-memory database where the data is persistent and replicated. And, it is popular enough to be well tested and easy to use (the API is very simple). I use HAProxy for my web servers and HAProxy supports ?stick? tables to map a client IP to the same backend server that was selected when the session was first established. HAProxy then supports proxy ?peers? where the ?stick? tables are shared between multiple proxies. That way, if a proxy fails, I can move the VIP over (or let DNS round-robin) to another proxy and still get the same backend (which has session state) without having the proxy pick some other backend (losing the backend session state). It might be fairly complex for HAProxy to share these ?stick? tables across a cluster of proxies, but I would think it would be easy to use Redis to cache this data so all proxies could access this shared data. I?m not sure if Dovecot proxies would benefit from ?sticks and peers? for IMAP protocol, but it would be nice if Dovecot proxies could maintain the IMAP session if the connections needed to be moved to another proxy (for failover). Maybe it isn?t so bad if a dovecot proxy all of a sudden ?kicked? 10 Million IMAP sessions, but this might lead to a ?login? flood for the remaining proxies. So, at least the authorization data (the passdb queries) should be shared between proxies using Redis. > > 2) Permanent state: This is mainly about the storage. A lot of people use Dovecot with NFS. So one possibility for storing the permanent state is NFS. Another possibility with Dovecot Pro is to store it to object storage as blobs and keep a local cache of the state. A 3rd possibility might be to use some kind of a database for storing the permanent state. I'm fine with the first two, but with 3rd I see a lot of problems and not a whole lot of benefit. But if you think of the databases (or even NFS) as blob storage, you can think of them the same as any object storage and use the same obox format with them. What I'm mainly against is attempting to create some kind of a database that has structured format like (imap_uid, flags, ...) - I'm sure that can be useful for various purposes but performance or scalability isn't one of them. I would separate the permanent state into two: the indexes and the message data. As I understand it, the indexes are the meta data about the message data. I believe, that to scale, the indexes need fast read access so this means storing on local NVMe SSD storage. But, I want the indexes to be reliably shared between all backend servers in a dovecot cluster. Again, this means to me that you need some fast in-memory database like Redis to be the ?source of truth? for the indexes. I think doing read requests to Redis is very fast so you might not have to store a cache of the index on local NVMe SSD storage, but maybe I?m wrong. As for the message data, I would really like the option of storing this data in some external database like MongoDB. MongoDB stores documents as JSON (actually BSON) data which seems perfect for email storage since emails are all text files. This would allow me to manage storage using the tools/techniques that an external database uses. MongoDB is designed to be hugely scalable and supports High Availability. I would rather manage a cluster of MongoDB instances containing a petabyte of data than trying to distribute the data among many Dovecot IMAP servers. The IMAP servers would then only be responsible for implementing IMAP and not be loaded down with all sorts of I/O so might be able to scale to 10 Million IMAP sessions per server. If a MongoDB option wasn?t available, using cloud object storage would be a reasonable second choice. Unfortunately, the ?obox? support you mentioned doesn?t seem to be open source. So, I am stuck using local disks (hopefully SSDs, but this is pricey) on multiple backend servers. I had reliability problems using NFS for a previous project and I am hesitant to try this solution for scaling Dovecot. Fortunately, my mailboxes are all very small (maybe 2MBs per user) since I delete messages older than 30 days and I store attachments (photos and videos) in cloud object storage served with local web server caching. So, scaling message data shouldn't be an issue for me for a long time. Kevin From tss at iki.fi Thu Feb 23 23:15:01 2017 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2017 01:15:01 +0200 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: On 24 Feb 2017, at 0.08, Mark Moseley wrote: > > As someone who is about to begin the process of moving from maildir to > mdbox on NFS (and therefore just about to start the 'director-ization' of > everything) for ~6.5m mailboxes, I'm curious if anyone can share any > experiences with it. The list is surprisingly quiet about this subject, and > articles on google are mainly just about setting director up. I've yet to > stumble across an article about someone's experiences with it. > > * How big of a director cluster do you use? I'm going to have millions of > mailboxes behind 10 directors. I wouldn't use more than 10. > I'm guessing that's plenty. It's actually split over two datacenters. Two datacenters in the same director ring? This is dangerous. if there's a network connectivity problem between them, they split into two separate rings and start redirecting users to different backends. > * Do you have consistent hashing turned on? I can't think of any reason not > to have it turned on, but who knows Definitely turn it on. The setting only exists because of backwards compatibility and will be removed at some point. From moseleymark at gmail.com Thu Feb 23 23:32:54 2017 From: moseleymark at gmail.com (Mark Moseley) Date: Thu, 23 Feb 2017 15:32:54 -0800 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: > On 24 Feb 2017, at 0.08, Mark Moseley wrote: > > > > As someone who is about to begin the process of moving from maildir to > > mdbox on NFS (and therefore just about to start the 'director-ization' of > > everything) for ~6.5m mailboxes, I'm curious if anyone can share any > > experiences with it. The list is surprisingly quiet about this subject, > and > > articles on google are mainly just about setting director up. I've yet to > > stumble across an article about someone's experiences with it. > > > > * How big of a director cluster do you use? I'm going to have millions of > > mailboxes behind 10 directors. > > I wouldn't use more than 10. > > Cool > > I'm guessing that's plenty. It's actually split over two datacenters. > > Two datacenters in the same director ring? This is dangerous. if there's a > network connectivity problem between them, they split into two separate > rings and start redirecting users to different backends. > I was unclear. The two director rings are unrelated and won't ever need to talk to each other. I only mentioned the two rings to point out that all 6.5m mailboxes weren't behind one ring, but rather split between two > > > * Do you have consistent hashing turned on? I can't think of any reason > not > > to have it turned on, but who knows > > Definitely turn it on. The setting only exists because of backwards > compatibility and will be removed at some point. > > Out of curiosity (and possibly extremely naive), unless you've moved a mailbox via 'doveadm director', if someone is pointed to a box via consistent hashing, why would the directors need to share that mailbox mapping? Again, assuming they're not moved (I'm also assuming that the mailbox would always, by default, hash to the same value in the consistent hash), isn't their hashing all that's needed to get to the right backend? I.e. "I know what the mailbox hashes to, and I know what backend that hash points at, so I'm done", in which case, no need to communicate to the other directors. I could see that if you moved someone, it *would* need to communicate that mapping. Then the only maps traded by directors would be the consistent hash boundaries *plus* any "moved" mailboxes. Again, just curious. From zhb at iredmail.org Thu Feb 23 23:45:40 2017 From: zhb at iredmail.org (Zhang Huangbin) Date: Fri, 24 Feb 2017 07:45:40 +0800 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: > On Feb 24, 2017, at 6:08 AM, Mark Moseley wrote: > > * Do you use the perl poolmon script or something else? The perl script was > being weird for me, so I rewrote it in python but it basically does the > exact same things. Would you mind sharing it? :) ---- Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ Time zone: GMT+8 (China/Beijing). Available on Telegram: https://t.me/iredmail From stephan at rename-it.nl Fri Feb 24 09:20:17 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 24 Feb 2017 10:20:17 +0100 Subject: Sieve removeflag Action In-Reply-To: <20170222194041.GA3860@nihlus.leuxner.net> References: <20170113081514.GA60507@nihlus.leuxner.net> <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> <20170220115644.GA14379@nihlus.leuxner.net> <20170222194041.GA3860@nihlus.leuxner.net> Message-ID: <865f781e-1e1e-dc69-4f6b-2326a01df195@rename-it.nl> Op 2/22/2017 om 8:40 PM schreef Thomas Leuxner: > * Thomas Leuxner 2017.02.20 12:56: > >> Feb 20 07:00:23 nihlus dovecot: master: Dovecot v2.2.devel (8f42a89) starting up for imap, lmtp >> >> This one processed the dovecot-news mail for 2.2.28.rc1 fine which uses a similar sieve rule. I will monitor global rules with this build and report back. > The results seem to be arbitrary. Personal rules _always_ work for flag actions, global rules included do _sometimes_. This means one day the same rule may trigger, the next day it won't. I found no way to reproduce it for global rules in order to narrow down the issue. Could you show me your full configuration (`dovecot -n`)? Regards, Stephan From tlx at leuxner.net Fri Feb 24 09:30:57 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Fri, 24 Feb 2017 10:30:57 +0100 Subject: Sieve removeflag Action In-Reply-To: <865f781e-1e1e-dc69-4f6b-2326a01df195@rename-it.nl> References: <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> <20170220115644.GA14379@nihlus.leuxner.net> <20170222194041.GA3860@nihlus.leuxner.net> <865f781e-1e1e-dc69-4f6b-2326a01df195@rename-it.nl> Message-ID: <20170224093057.GA57205@nihlus.leuxner.net> * Stephan Bosch 2017.02.24 10:20: > Could you show me your full configuration (`dovecot -n`)? > > Regards, > > Stephan Live configuration and scripts sent off-list. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From trever at middleearth.sapphiresunday.org Fri Feb 24 10:39:18 2017 From: trever at middleearth.sapphiresunday.org (Trever L. Adams) Date: Fri, 24 Feb 2017 03:39:18 -0700 Subject: Replacement for antispam plugin In-Reply-To: <677646b9-26ad-bb18-3c34-f44374491635@rename-it.nl> References: <125A971B80F78A34.02b39013-cf0d-4965-8749-8bd8a1f6e1c0@mail.outlook.com> <677646b9-26ad-bb18-3c34-f44374491635@rename-it.nl> Message-ID: <40fbc423-23f7-9ad9-0cc9-438e8298534b@middleearth.sapphiresunday.org> On 02/12/2017 05:28 PM, Stephan Bosch wrote: > > Actually, Pigeonhole should be able to do that too: > > https://github.com/dovecot/pigeonhole/blob/master/doc/plugins/sieve_extprograms.txt#L112 > > Yes, I need to update the wiki. > > > Regards, > > Stephan. > For DSPAM, with --client, one also needs a --user set. http://hg.dovecot.org/dovecot-antispam-plugin/file/5ebc6aae4d7c/src/dspam.c did this. Is there a way to feed this into the scripts mentioned? I imagine this is imap.user or imap.email, but how would one pass it to the script? Thank you. Trever -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 872 bytes Desc: OpenPGP digital signature URL: From ben+dovecot at list-subs.com Fri Feb 24 11:49:19 2017 From: ben+dovecot at list-subs.com (Ben) Date: Fri, 24 Feb 2017 11:49:19 +0000 Subject: doveadm "-v" optoin doesn't do anything ? Message-ID: <8ee94035-4d76-6761-c787-11235649d79f@list-subs.com> Hi, On 2.2.10, running the following doveadm -v -o mail_fsync=never backup -R -u my at example.com imapc: There is zero output ? Running -D instead of -v it spews out debug messages. Any ideas ? Ben From tss at iki.fi Fri Feb 24 13:34:37 2017 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2017 15:34:37 +0200 Subject: v2.2.28 released Message-ID: http://dovecot.org/releases/2.2/dovecot-2.2.28.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.28.tar.gz.sig * director: "doveadm director move" to same host now refreshes user's timeout. This allows keeping user constantly in the same backend by just periodically moving the user there. * When new mailbox is created, use initially INBOX's dovecot.index.cache caching decisions. * Expunging mails writes GUID to dovecot.index.log now only if the GUID is quickly available from index/cache. * pop3c: Increase timeout for PASS command to 5 minutes. * Mail access errors are no longer ignored when searching or sorting. With IMAP the untagged SEARCH/SORT reply is still sent the same as before, but NO reply is returned instead of OK. + Make dovecot.list.index's filename configurable. This is needed when there are multiple namespaces pointing to the same mail root (e.g. lazy_expunge namespace for mdbox). + Add size.virtual to dovecot.index when folder vsizes are accessed (e.g. quota=count). This is mainly a workaround to avoid slow quota recalculation performance when message sizes get lost from dovecot.index.cache due to corruption or some other reason. + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them in lib-dsasl for client side. + auth: Support filtering by SASL mechanism: passdb { mechanisms } + Shrink the mail processes' memory usage by not storing settings duplicated unnecessarily many times. + imap: Add imap_fetch_failure setting to control what happens when FETCH fails for some mails (see example-config). + imap: Include info about last command in disconnection log line. + imap: Created new SEARCH=X-MIMEPART extension. It's currently not advertised by default, since it's not fully implemented. + fts-solr: Add support for basic authentication. + Cassandra: Support automatically retrying failed queries if execution_retry_interval and execution_retry_times are set. + doveadm: Added "mailbox path" command. + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the wanted fields only if it doesn't require opening the email. + mail_vsize_bg_after_count setting added (see example-config). + mail_sort_max_read_count setting added (see example-config). + pop3c: Added pop3c_features=no-pipelining setting to prevent using PIPELINING extension even though it's advertised. - Index files: day_first_uid wasn't updated correctly since v2.2.26. This caused dovecot.index.cache to be non-optimal. - imap: SEARCH/SORT may have assert-crashed in client_check_command_hangs - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. - imap: Running time in tagged command reply was often wrongly 0. - search: Using NOT n:* or NOT UID n:* wasn't handled correctly - director: doveadm director kick was broken - director: Fix crash when using director_flush_socket - director: Fix some bugs when moving users between backends - imapc: Various error handling fixes and improvements - master: doveadm process status output had a lot of duplicates. - autoexpunge: If mailbox's rename timestamp is newer than mail's save-timestamp, use it instead. This is useful when autoexpunging e.g. Trash/* and an entire mailbox is deleted by renaming it under Trash to prevent it from being autoexpunged too early. - autoexpunge: Multiple processes may have been trying to expunge the same mails simultaneously. This was problematic especially with lazy_expunge plugin. - auth: %{passdb:*} was empty in auth-worker processes - auth-policy: hashed_password was always sent empty. - dict-sql: Merge multiple UPDATEs to a single statement if possible. - fts-solr: Escape {} chars when sending queries - fts: fts_autoindex_exclude = \Special-use caused crashes - doveadm-server: Fix leaks and other problems when process is reused for multiple requests (service_count != 1) - sdbox: Fix assert-crash on mailbox create race - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve was used. especially %{storage_id} was broken. - lmtp_user_concurrency_limit didn't work if userdb changed username From rye at trojka.no Fri Feb 24 13:45:45 2017 From: rye at trojka.no (Eirik Rye) Date: Fri, 24 Feb 2017 14:45:45 +0100 Subject: Users with multiple password Message-ID: <33a85273-2385-52a5-5fa0-f4e0d5f4d392@trojka.no> Hi! ~ dovecot --version 2.2.22 (fe789d2) I am wondering if there is a way to set up virtual users with multiple valid passwords. We want to be able to provide users with device/app-specific passwords for their email accounts, as well as being able to create temporary "access tokens" for technical support when required. I quickly found out that passdb using passwd-file or an sql-backend does not support returning multiple entries ("Error: passwd-file /etc/dovecot/virtual.passwd: User rye exists more than once"). The documentation mentions that you can pass the plain-text password on to the MySQL-server for verification, and I suppose multiple passwords could could work, given a query like this (pseudo-SQL): `SELECT password FROM account WHERE user = '%u' AND domain = '%d' AND password = TO_BASE64((SHA2('%w', 512));` However, having Dovecot pass the plain-text password and letting the database deal with the hashing and encoding doesn't seem like a very "clean" solution. Preferably, dovecot should be the only piece of software touching the plain-text. Ideally, I would like the following behavior: 1. passdb results multiple possible hashed passwords for the user 2. dovecot attempts the passwords in order 3. login fails normally if none of the passdb results match Does anyone have any experience, or tips for setting up this type of behavior? Other ideas we have touched upon are: 1. Different usernames (eg. 'user_device' or 'user_application') 2. Multiple passdbs(?) Best regards, Eirik Rye From umutkacar at gmail.com Fri Feb 24 14:43:22 2017 From: umutkacar at gmail.com (=?UTF-8?Q?Umut_Erol_Ka=C3=A7ar?=) Date: Fri, 24 Feb 2017 14:43:22 +0000 Subject: Quota usage value shows 140% of actual disk usage Message-ID: Hello everyone, Our server has these installed: dovecot-2.0.21-2.el6.x86_64 dovecot-pigeonhole-2.0.21-2.el6.x86_64 dovecot-mysql-2.0.21-2.el6.x86_64 ...and has been running for quite a long time, with several hundred domains and thousands of accounts on it. My colleagues reported that it's been showing quota usage values that are more than actual disk usage, so I investigated and found that Dovecot shows a quota usage value roughly around 140% of actual disk usage. It's also valid on newly created accounts. My test account for example: doveadm quota get -u test at example.local Quota name Type Value Limit % User quota STORAGE 4359 512000 0 User quota MESSAGE 7 - 0 du -sc /home/vmail/example.local/test/Maildir/{*,.[!.]*} 1044 /home/vmail/example.local/test/Maildir/cur 28 /home/vmail/example.local/test/Maildir/dovecot.index.cache 8 /home/vmail/example.local/test/Maildir/dovecot.index.log 4 /home/vmail/example.local/test/Maildir/dovecot.mailbox.log 4 /home/vmail/example.local/test/Maildir/dovecot-uidlist 4 /home/vmail/example.local/test/Maildir/dovecot-uidvalidity 0 /home/vmail/example.local/test/Maildir/dovecot-uidvalidity.56a4dc8e 4 /home/vmail/example.local/test/Maildir/new 4 /home/vmail/example.local/test/Maildir/subscriptions 4 /home/vmail/example.local/test/Maildir/tmp 24 /home/vmail/example.local/test/Maildir/.Junk 1932 /home/vmail/example.local/test/Maildir/.Sent 44 /home/vmail/example.local/test/Maildir/.Trash 3104 total 4359/3104=1,40431701 So it shows roughly around 1,4*actualDiskUsage. The ratio is mostly the same for almost all the other accounts. It can vary between like 1,3-1,6. So, the gap gets insane when more disk space is used, say like with 2GB disk usage, Dovecot thinks 3,5GB quota is used... dovecot quota recalc does not fix the issue, it only sets the same value again (I've checked with tcpdump and saw the query with the same quota usage value). The method is Dictionary quota with SQL. I'm attaching the dovecot -n output with some other config files. I've tried setting messages and bytes value to -1 on the MariaDB database to force recalculation. But as soon as I run doveadm quota recalc, it gets the same wrong value again. What can I do to fix this? Thanks in advance. -------------- next part -------------- ## ## Quota configuration. ## # Note that you also have to enable quota plugin in mail_plugins setting. # ## ## Quota limits ## # Quota limits are set using "quota_rule" parameters. To get per-user quota # limits, you can set/override them by returning "quota_rule" extra field # from userdb. It's also possible to give mailbox-specific limits, for example # to give additional 100 MB when saving to Trash: plugin { quota_rule = *:storage=500M quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% } ## ## Quota warnings ## # You can execute a given command when user exceeds a specified quota limit. # Each quota root has separate limits. Only the command for the first # exceeded limit is excecuted, so put the highest limit first. # The commands are executed via script service by connecting to the named # UNIX socket (quota-warning below). # Note that % needs to be escaped as %%, otherwise "% " expands to empty. plugin { quota_warning = storage=99%% quota-warning 99 %u quota_warning2 = storage=80%% quota-warning 80 %u } # Quota Warning service service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail unix_listener quota-warning { user = vmail group = vmail mode = 0660 } } ## ## Quota backends ## # Multiple backends are supported: # dirsize: Find and sum all the files found from mail directory. # Extremely SLOW with Maildir. It'll eat your CPU and disk I/O. # dict: Keep quota stored in dictionary (eg. SQL) # maildir: Maildir++ quota # fs: Read-only support for filesystem quota plugin { #quota = dirsize:User quota #quota = maildir:User quota quota = dict:User quota::proxy::sqlquota #quota = dict:User quota::proxy::quota #quota = fs:User quota } # Multiple quota roots are also possible, for example this gives each user # their own 100MB quota and one shared 1GB quota within the domain: plugin { #quota = dict:user::proxy::quota #quota2 = dict:domain:%d:proxy::quota_domain #quota_rule = *:storage=102400 #quota2_rule = *:storage=1048576 quota_status_success = DUNNO quota_status_nouser = DUNNO quota_status_overquota = "552 5.2.2 Mailbox is full" } -------------- next part -------------- connect = host=192.168.95.8 dbname=postfix_masterdb user=postfix_user password=someStrongPassword map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages } -------------- next part -------------- # 2.0.21: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-642.6.2.el6.centos.plus.x86_64 x86_64 CentOS release 6.8 (Final) ext4 auth_master_user_separator = * auth_mechanisms = plain login debug_log_path = /var/log/dovecot-debug.log dict { sqlquota = mysql:/etc/dovecot/dovecot-quota-sql.conf.ext } disable_plaintext_auth = no hostname = mailbox.radore.net mail_location = maildir:/home/vmail/%Ld/%Ln/Maildir mail_plugins = " notify mail_log quota autocreate zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { autocreate = Trash autocreate2 = Junk autocreate3 = Sent autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::proxy::sqlquota quota_rule = *:storage=500M quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is fullradore quota_status_success = DUNNO quota_warning = storage=99%% quota-warning 99 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve-scripts/before/ sieve_dir = ~/sieve sieve_max_actions = 10 sieve_max_redirects = 2 sieve_max_script_size = 100K sieve_quota_max_scripts = 10 sieve_quota_max_storage = 1M zlib_save = bz2 zlib_save_level = 9 } postmaster_address = postmaster at mx01.some.where protocols = imap pop3 lmtp sieve service auth { client_limit = 15000 inet_listener smtp-auth { address = 192.168.95.9 port = 40025 } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 24 service_count = 0 vsz_limit = 1 G } service imap { process_limit = 8192 vsz_limit = 1 G } service lmtp { inet_listener lmtp { address = 192.168.95.9 port = 10025 } } service managesieve-login {kk inet_listener sieve { port = 4190 } process_min_avail = 8 service_count = 0 vsz_limit = 128 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_min_avail = 24 service_count = 0 vsz_limit = 1 G } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } user = vmail } ssl_ca = References: <33a85273-2385-52a5-5fa0-f4e0d5f4d392@trojka.no> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 24 Feb 2017, Eirik Rye wrote: > 2. Multiple passdbs(?) Check out http://wiki2.dovecot.org/PasswordDatabase result_failure = continue result_internalfail = continue result_success = return-ok - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWLBJDHz1H7kL/d9rAQLffwf/X7qZCiQ/a/dVYPVLC+Ie+RoFqwl/W96m syyTSsrwPa9GnkAygSpuBUByBDNwsNCTaheao3kwkhD51hBtHxkXXZlozyaJy9q7 ZA7UAQAwWGZlZpnUNzM4nRyFRyBLsZAWpMWQAZLy868kjXR75M4fxX4YdsHCp0Jf Ajp88Khcx04e11tmEpTRoDbcsWyoap8YKCblbgS6euKXYu4oQT2gV+iLQAkTBAPM Yh8Od3M7i9xf/6iP3lfj3HJtLb7KhtsgcmLQbGd+PPdWIOc9geeF9222ssP5QyYj OL5PlL3Mm7c/BrHbqKnFNILKcf31CHdahigDYNheGKeS43Zx89uRwA== =t9Fy -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Fri Feb 24 15:00:14 2017 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 24 Feb 2017 16:00:14 +0100 (CET) Subject: Quota usage value shows 140% of actual disk usage In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 24 Feb 2017, Umut Erol Ka?ar wrote: > investigated and found that Dovecot shows a quota usage value roughly > around 140% of actual disk usage. It's also valid on newly created > accounts. My test account for example: Quota does not count physical useage, but the amount of bytes allocated by the messages. Maildir may hardlink messages, hence, they count multiple times for the quota, but once for du. > The ratio is mostly the same for almost all the other accounts. It can vary > between like 1,3-1,6. So, the gap gets insane when more disk space is used, > say like with 2GB disk usage, Dovecot thinks 3,5GB quota is used... Hmm, are you sure? > dovecot quota recalc does not fix the issue, it only sets the same value > again (I've checked with tcpdump and saw the query with the same quota if quota recalc re-creates the same value, please check the hard link stuff. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBWLBKfnz1H7kL/d9rAQK0ugf9EmPnt5zWLiWeCxsRi9iFHAcEFW6qE4be 98G2N5gzkyyQZZsPkaSQ05fVj81zijeArFn0DwtnXJzhDHkCzI4C2yd1LgwbgnFg /wakFkjW8jd7m58Xv45UEmKlZC83o7mOQPNYqp23v2M7k06y4XDnGpNf8V5Ipu/F IaoK7ZS9DkaIrF6Rv6+WGtlLjjUj4By0ZJk6mGD3eNsTnQmM6j+tEcv75WXJMhNP M7QkqwqPaj/oPVUfNut+e5VzB5Y/w1UAJpHMAnsfXK5MC9DnzU4OQtoxhWtcLzm/ HbTs7rfGbGtcnLh9tgMtMy43awqSHe9V4v0/DASL/0oCQbFFo0AEmg== =71mb -----END PGP SIGNATURE----- From rye at trojka.no Fri Feb 24 15:16:06 2017 From: rye at trojka.no (Eirik Rye) Date: Fri, 24 Feb 2017 16:16:06 +0100 Subject: Users with multiple password In-Reply-To: References: <33a85273-2385-52a5-5fa0-f4e0d5f4d392@trojka.no> Message-ID: On 24/02/2017 15:54, Steffen Kaiser wrote: > Check out http://wiki2.dovecot.org/PasswordDatabase > > result_failure = continue > result_internalfail = continue > result_success = return-ok > > - -- Steffen Kaiser Thanks. I have looked at this, however it would still require the secondary passdb to be passing the plain-text password on to the backend in order to constrain the passdb-query to a single result, right? - Eirik Rye From moseleymark at gmail.com Fri Feb 24 19:28:12 2017 From: moseleymark at gmail.com (Mark Moseley) Date: Fri, 24 Feb 2017 11:28:12 -0800 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: On Thu, Feb 23, 2017 at 3:45 PM, Zhang Huangbin wrote: > > > On Feb 24, 2017, at 6:08 AM, Mark Moseley wrote: > > > > * Do you use the perl poolmon script or something else? The perl script > was > > being weird for me, so I rewrote it in python but it basically does the > > exact same things. > > Would you mind sharing it? :) > > ---- > Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ > Time zone: GMT+8 (China/Beijing). > Available on Telegram: https://t.me/iredmail > > Attached. No claims are made on the quality of my code :) -------------- next part -------------- A non-text attachment was scrubbed... Name: poolmon Type: application/octet-stream Size: 8595 bytes Desc: not available URL: From moseleymark at gmail.com Fri Feb 24 19:29:51 2017 From: moseleymark at gmail.com (Mark Moseley) Date: Fri, 24 Feb 2017 11:29:51 -0800 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: > > On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: > >> On 24 Feb 2017, at 0.08, Mark Moseley wrote: >> > >> > As someone who is about to begin the process of moving from maildir to >> > mdbox on NFS (and therefore just about to start the 'director-ization' >> of >> > everything) for ~6.5m mailboxes, I'm curious if anyone can share any >> > experiences with it. The list is surprisingly quiet about this subject, >> and >> > articles on google are mainly just about setting director up. I've yet >> to >> > stumble across an article about someone's experiences with it. >> > >> > * How big of a director cluster do you use? I'm going to have millions >> of >> > mailboxes behind 10 directors. >> >> I wouldn't use more than 10. >> >> > Cool > > > >> > I'm guessing that's plenty. It's actually split over two datacenters. >> >> Two datacenters in the same director ring? This is dangerous. if there's >> a network connectivity problem between them, they split into two separate >> rings and start redirecting users to different backends. >> > > I was unclear. The two director rings are unrelated and won't ever need to > talk to each other. I only mentioned the two rings to point out that all > 6.5m mailboxes weren't behind one ring, but rather split between two > > > >> >> > * Do you have consistent hashing turned on? I can't think of any reason >> not >> > to have it turned on, but who knows >> >> Definitely turn it on. The setting only exists because of backwards >> compatibility and will be removed at some point. >> >> > Out of curiosity (and possibly extremely naive), unless you've moved a > mailbox via 'doveadm director', if someone is pointed to a box via > consistent hashing, why would the directors need to share that mailbox > mapping? Again, assuming they're not moved (I'm also assuming that the > mailbox would always, by default, hash to the same value in the consistent > hash), isn't their hashing all that's needed to get to the right backend? > I.e. "I know what the mailbox hashes to, and I know what backend that hash > points at, so I'm done", in which case, no need to communicate to the other > directors. I could see that if you moved someone, it *would* need to > communicate that mapping. Then the only maps traded by directors would be > the consistent hash boundaries *plus* any "moved" mailboxes. Again, just > curious. > > Timo, Incidentally, on that error I posted: Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 any idea what would cause that? Is my guess that multiple directors tried to update the status simultaneously correct? From wgrcunha at gmail.com Fri Feb 24 19:41:17 2017 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Fri, 24 Feb 2017 16:41:17 -0300 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: In our experience. A ring with more of 4 servers is bad, we have sync problems everyone. Using 4 or less works perfect. Em 24 de fev de 2017 4:30 PM, "Mark Moseley" escreveu: > > > > On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: > > > >> On 24 Feb 2017, at 0.08, Mark Moseley wrote: > >> > > >> > As someone who is about to begin the process of moving from maildir to > >> > mdbox on NFS (and therefore just about to start the 'director-ization' > >> of > >> > everything) for ~6.5m mailboxes, I'm curious if anyone can share any > >> > experiences with it. The list is surprisingly quiet about this > subject, > >> and > >> > articles on google are mainly just about setting director up. I've yet > >> to > >> > stumble across an article about someone's experiences with it. > >> > > >> > * How big of a director cluster do you use? I'm going to have millions > >> of > >> > mailboxes behind 10 directors. > >> > >> I wouldn't use more than 10. > >> > >> > > Cool > > > > > > > >> > I'm guessing that's plenty. It's actually split over two datacenters. > >> > >> Two datacenters in the same director ring? This is dangerous. if there's > >> a network connectivity problem between them, they split into two > separate > >> rings and start redirecting users to different backends. > >> > > > > I was unclear. The two director rings are unrelated and won't ever need > to > > talk to each other. I only mentioned the two rings to point out that all > > 6.5m mailboxes weren't behind one ring, but rather split between two > > > > > > > >> > >> > * Do you have consistent hashing turned on? I can't think of any > reason > >> not > >> > to have it turned on, but who knows > >> > >> Definitely turn it on. The setting only exists because of backwards > >> compatibility and will be removed at some point. > >> > >> > > Out of curiosity (and possibly extremely naive), unless you've moved a > > mailbox via 'doveadm director', if someone is pointed to a box via > > consistent hashing, why would the directors need to share that mailbox > > mapping? Again, assuming they're not moved (I'm also assuming that the > > mailbox would always, by default, hash to the same value in the > consistent > > hash), isn't their hashing all that's needed to get to the right backend? > > I.e. "I know what the mailbox hashes to, and I know what backend that > hash > > points at, so I'm done", in which case, no need to communicate to the > other > > directors. I could see that if you moved someone, it *would* need to > > communicate that mapping. Then the only maps traded by directors would be > > the consistent hash boundaries *plus* any "moved" mailboxes. Again, just > > curious. > > > > > Timo, > Incidentally, on that error I posted: > > Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host > 10.1.17.3 is being updated before previous update had finished (up -> down) > - setting to state=down vhosts=0 > Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host > 10.1.17.3 is being updated before previous update had finished (down -> up) > - setting to state=up vhosts=0 > > any idea what would cause that? Is my guess that multiple directors tried > to update the status simultaneously correct? > From tss at iki.fi Fri Feb 24 19:53:26 2017 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2017 21:53:26 +0200 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: On 24 Feb 2017, at 21.29, Mark Moseley wrote: > > Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host > 10.1.17.3 is being updated before previous update had finished (up -> down) > - setting to state=down vhosts=0 > Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host > 10.1.17.3 is being updated before previous update had finished (down -> up) > - setting to state=up vhosts=0 > > any idea what would cause that? Is my guess that multiple directors tried > to update the status simultaneously correct? Most likely, yes. I'm not sure if it might happen also if the same server issues conflicting commands rapidly. From moseleymark at gmail.com Fri Feb 24 19:58:50 2017 From: moseleymark at gmail.com (Mark Moseley) Date: Fri, 24 Feb 2017 11:58:50 -0800 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: On Fri, Feb 24, 2017 at 11:41 AM, Francisco Wagner C. Freire < wgrcunha at gmail.com> wrote: > In our experience. A ring with more of 4 servers is bad, we have sync > problems everyone. Using 4 or less works perfect. > > Em 24 de fev de 2017 4:30 PM, "Mark Moseley" > escreveu: > >> > >> > On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen wrote: >> > >> >> On 24 Feb 2017, at 0.08, Mark Moseley wrote: >> >> > >> >> > As someone who is about to begin the process of moving from maildir >> to >> >> > mdbox on NFS (and therefore just about to start the >> 'director-ization' >> >> of >> >> > everything) for ~6.5m mailboxes, I'm curious if anyone can share any >> >> > experiences with it. The list is surprisingly quiet about this >> subject, >> >> and >> >> > articles on google are mainly just about setting director up. I've >> yet >> >> to >> >> > stumble across an article about someone's experiences with it. >> >> > >> >> > * How big of a director cluster do you use? I'm going to have >> millions >> >> of >> >> > mailboxes behind 10 directors. >> >> >> >> I wouldn't use more than 10. >> >> >> >> >> > Cool >> > Interesting. That's good feedback. One of the things I wondered about is whether it'd be better to deploy a 10-node ring or split it into 2x 5-node rings. Sounds like splitting it up might not be a bad idea. How often would you see those sync problems (and were they the same errors as I posted or something else)? And were you running poolmon from every node when you were seeing sync errors? From heiken at luis.uni-hannover.de Fri Feb 24 21:06:58 2017 From: heiken at luis.uni-hannover.de (Karsten Heiken) Date: Fri, 24 Feb 2017 22:06:58 +0100 Subject: Quota usage value shows 140% of actual disk usage In-Reply-To: References: Message-ID: <1d27e33b-b014-aff4-ef82-d668a213c0b0@luis.uni-hannover.de> Am 24.02.2017 um 16:00 schrieb Steffen Kaiser: > > Quota does not count physical useage, but the amount of bytes allocated by the messages. Maildir may hardlink messages, hence, they count multiple times for the quota, but once for du. And in your case dovecot even compressed the mails: According to your doveconf, you are using mail_plugins = [...] zlib. Dovecot's quota is calculated using the uncompressed size, whereas du shows you the space actually allocated. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From matthew.broadhead at nbmlaw.co.uk Fri Feb 24 22:16:13 2017 From: matthew.broadhead at nbmlaw.co.uk (Matthew Broadhead) Date: Fri, 24 Feb 2017 23:16:13 +0100 Subject: sieve_imapsieve centos 7 Message-ID: <5d5d80a9-9ff5-d369-dab5-84e187af56ce@nbmlaw.co.uk> i am using CentOS 7 centos-release-7-3.1611.el7.centos.x86_64 with dovecot dovecot-2.2.10-7.el7.x86_64. i am trying to set up AntiSpam with IMAPSieve but the package seems to be lacking sieve_imapsieve. is there anything i can do? i am not really interested in compiling from source because i like to receive security updates automatically. 2017-02-24 21:57:00auth: Error: net_connect_unix(anvil-auth-penalty) failed: Permission denied 2017-02-24 21:57:00master: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) 2017-02-24 21:57:00managesieve: Fatal: Plugin 'sieve_imapsieve' not found from directory /usr/lib64/dovecot/sieve 2017-02-24 21:57:00config: Error: managesieve-login: dump-capability process returned 89 2017-02-24 21:57:05imap: Fatal: Plugin 'imap_sieve' not found from directory /usr/lib64/dovecot 2017-02-24 21:57:05imap: Fatal: Plugin 'imap_sieve' not found from directory /usr/lib64/dovecot 2017-02-24 21:57:08imap: Fatal: Plugin 'imap_sieve' not found from directory /usr/lib64/dovecot 2017-02-24 21:57:08imap: Fatal: Plugin 'imap_sieve' not found from directory /usr/lib64/dovecot 2017-02-24 21:57:10imap: Fatal: Plugin 'imap_sieve' not found from directory /usr/lib64/dovecot 2017-02-24 21:57:10imap: Fatal: Plugin 'imap_sieve' not found from directory /usr/lib64/dovecot From ml+dovecot at valo.at Fri Feb 24 22:38:59 2017 From: ml+dovecot at valo.at (Christian Kivalo) Date: Fri, 24 Feb 2017 23:38:59 +0100 Subject: sieve_imapsieve centos 7 In-Reply-To: <5d5d80a9-9ff5-d369-dab5-84e187af56ce@nbmlaw.co.uk> References: <5d5d80a9-9ff5-d369-dab5-84e187af56ce@nbmlaw.co.uk> Message-ID: <47deb9a05a53068e82ee26516658ff65@valo.at> On 2017-02-24 23:16, Matthew Broadhead wrote: > i am using CentOS 7 centos-release-7-3.1611.el7.centos.x86_64 with > dovecot dovecot-2.2.10-7.el7.x86_64. i am trying to set up AntiSpam > with IMAPSieve but the package seems to be lacking sieve_imapsieve. is > there anything i can do? i am not really interested in compiling from > source because i like to receive security updates automatically. The imapsieve plugin for pigeonhole was introduced in version 0.4.14 for dovecot 2.2.24, with your current packages i'd say there is nothing you can do except to find some sort of extra packages (i'm not familiar with centos...) http://dovecot.markmail.org/message/mggbfw6vxhs2upa7?q=imapsieve&page=2 > 2017-02-24 21:57:00auth: Error: net_connect_unix(anvil-auth-penalty) > failed: Permission denied > 2017-02-24 21:57:00master: Warning: Killed with signal 15 (by pid=1 > uid=0 code=kill) > 2017-02-24 21:57:00managesieve: Fatal: Plugin 'sieve_imapsieve' not > found from directory /usr/lib64/dovecot/sieve > 2017-02-24 21:57:00config: Error: managesieve-login: dump-capability > process returned 89 > 2017-02-24 21:57:05imap: Fatal: Plugin 'imap_sieve' not found from > directory /usr/lib64/dovecot > 2017-02-24 21:57:05imap: Fatal: Plugin 'imap_sieve' not found from > directory /usr/lib64/dovecot > 2017-02-24 21:57:08imap: Fatal: Plugin 'imap_sieve' not found from > directory /usr/lib64/dovecot > 2017-02-24 21:57:08imap: Fatal: Plugin 'imap_sieve' not found from > directory /usr/lib64/dovecot > 2017-02-24 21:57:10imap: Fatal: Plugin 'imap_sieve' not found from > directory /usr/lib64/dovecot > 2017-02-24 21:57:10imap: Fatal: Plugin 'imap_sieve' not found from > directory /usr/lib64/dovecot -- Christian Kivalo From zhb at iredmail.org Sat Feb 25 04:24:03 2017 From: zhb at iredmail.org (Zhang Huangbin) Date: Sat, 25 Feb 2017 12:24:03 +0800 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: <4CF42B5B-7BC3-4AA7-8BAA-64823C2DF456@iredmail.org> > On Feb 25, 2017, at 3:28 AM, Mark Moseley wrote: > > Attached. No claims are made on the quality of my code :) Thank you for sharing. :) Some suggestions: - should replace log() by the standard logging module like "logging.debug(xx)? - add managesieve support - add lmtp support - how about store command line options in a config file? remove the ?optparse? module. - email notification support when server is up/down - lots of PEP8 style issue :) Would you like to publish this code in github/bitbucket/?? ---- Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ Time zone: GMT+8 (China/Beijing). Available on Telegram: https://t.me/iredmail From ben+dovecot at list-subs.com Sat Feb 25 10:33:17 2017 From: ben+dovecot at list-subs.com (Ben) Date: Sat, 25 Feb 2017 10:33:17 +0000 Subject: sieve_imapsieve centos 7 In-Reply-To: <47deb9a05a53068e82ee26516658ff65@valo.at> References: <5d5d80a9-9ff5-d369-dab5-84e187af56ce@nbmlaw.co.uk> <47deb9a05a53068e82ee26516658ff65@valo.at> Message-ID: <13febc7a-d749-e1c9-9b18-1d0367b7eb69@list-subs.com> >> i am using CentOS 7 centos-release-7-3.1611.el7.centos.x86_64 with >> dovecot dovecot-2.2.10-7.el7.x86_64. i am trying to set up AntiSpam >> with IMAPSieve but the package seems to be lacking sieve_imapsieve. is >> there anything i can do? i am not really interested in compiling from >> source because i like to receive security updates automatically. > The imapsieve plugin for pigeonhole was introduced in version 0.4.14 > for dovecot 2.2.24, with your current packages i'd say there is > nothing you can do except to find some sort of extra packages (i'm not > familiar with centos...) I have dovecot and sieve working on CentOS 7 Only using two packages dovecot.x86_64 1:2.2.10-7.el7 @base dovecot-pigeonhole.x86_64 1:2.2.10-7.el7 @base From aki.tuomi at dovecot.fi Sat Feb 25 11:46:08 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sat, 25 Feb 2017 13:46:08 +0200 Subject: sieve_imapsieve centos 7 Message-ID: imapsieve needs much newer dovecot and sieve. ---Aki TuomiDovecot oy -------- Original message --------From: Ben Date: 25/02/2017 12:33 (GMT+02:00) To: dovecot at dovecot.org Subject: Re: sieve_imapsieve centos 7 >> i am using CentOS 7 centos-release-7-3.1611.el7.centos.x86_64 with >> dovecot dovecot-2.2.10-7.el7.x86_64.? i am trying to set up AntiSpam >> with IMAPSieve but the package seems to be lacking sieve_imapsieve. is >> there anything i can do?? i am not really interested in compiling from >> source because i like to receive security updates automatically. > The imapsieve plugin for pigeonhole was introduced in version 0.4.14 > for dovecot 2.2.24, with your current packages i'd say there is > nothing you can do except to find some sort of extra packages (i'm not > familiar with centos...) I have dovecot and sieve working on CentOS 7 Only using two packages dovecot.x86_64 1:2.2.10-7.el7?????????????????? @base dovecot-pigeonhole.x86_64 1:2.2.10-7.el7?????????????????? @base From stephan at rename-it.nl Sat Feb 25 14:04:55 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 25 Feb 2017 15:04:55 +0100 Subject: sieve file backend: invalid option `active=~/.dovecot.sieve' In-Reply-To: References: <0ed0da6f-2a6a-d2ce-6458-3e164d608031@list-subs.com> <16ddcba3-d7f9-361a-b1b0-7effc67ad984@rename-it.nl> Message-ID: <66af1aa9-0d33-e6eb-4d73-d17a1383dfb8@rename-it.nl> Op 2/14/2017 om 8:45 PM schreef Ben: > > > On 14/02/2017 14:44, Stephan Bosch wrote: >> >> >> Op 13-2-2017 om 18:15 schreef Ben: >>> Hi, >>> >>> I am seeing the followin error in my logs (doveconf -n at the bottom >>> of this mail): >>> >>> Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: >>> cs3NOQ7moVj4sQXXXXX: sieve: sieve file backend: invalid option >>> `active=~/.dovecot.sieve' >>> Feb 13 16:59:59 mxf dovecot: lmtp(45560, bp at example.com): Error: >>> cs3NOQ7moVj4sQXXXXX: sieve: failed to access user's Sieve script >>> file:~/sieve;active=~/.dovecot.sieve (temporary failure) >>> >>> >>> Looking at http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration, >>> the syntax "sieve = file:~/sieve;active=~/.dovecot.sieve" in my config >>> is correct ? >>> >>> Is this a false error that is only appearing because this is a newly >>> created user with no sieve file ? If this is the case, how do I tell >>> dovecot not to error out and allow the mail ? >> >> Don't specify the "sieve_dir" setting when you're using the new location >> syntax for the "sieve" setting. That setting is deprecated and causes >> the "sieve" setting to be interpreted differently for backwards >> compatibility. >> >> > > I've now commented out sieve_dir and its no longer appearing in > doveconf -n, however the problem originally described still exists. Hmm, could be some bug in that old version you're using. What happens if you remove the sieve= setting as well? It is the default. Regards, Stephan. From tanstaafl at libertytrek.org Sat Feb 25 15:52:27 2017 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Sat, 25 Feb 2017 10:52:27 -0500 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: <819bb35a-0952-6efa-8fea-dce666576850@libertytrek.org> On Fri Feb 24 2017 14:41:17 GMT-0500 (Eastern Standard Time), Francisco Wagner C. Freire wrote: > In our experience. A ring with more of 4 servers is bad, we have sync > problems everyone. Using 4 or less works perfect. Since this contradicts Timo's recommendation not to use more than 10, it sounds to me like you either encountered a bug, or possibly it was not optimally deployed. Did you ever come here and ask for help? From ben+dovecot at list-subs.com Sat Feb 25 16:03:05 2017 From: ben+dovecot at list-subs.com (Ben) Date: Sat, 25 Feb 2017 16:03:05 +0000 Subject: sieve_imapsieve centos 7 In-Reply-To: References: Message-ID: <4b4529b1-9a80-7dda-6349-32ab4f4ab10c@list-subs.com> On 25/02/2017 11:46, Aki Tuomi wrote: > imapsieve needs much newer dovecot and sieve. In which case the OP would probably need to install from source, because that's the version in the CentOS 7 repo. From tss at iki.fi Sat Feb 25 17:25:15 2017 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2017 19:25:15 +0200 Subject: Director+NFS Experiences In-Reply-To: <819bb35a-0952-6efa-8fea-dce666576850@libertytrek.org> References: <819bb35a-0952-6efa-8fea-dce666576850@libertytrek.org> Message-ID: <17E95D96-CE93-4D99-B1E0-2D3D80E26EDC@iki.fi> On 25 Feb 2017, at 17.52, Tanstaafl wrote: > > On Fri Feb 24 2017 14:41:17 GMT-0500 (Eastern Standard Time), Francisco > Wagner C. Freire wrote: >> In our experience. A ring with more of 4 servers is bad, we have sync >> problems everyone. Using 4 or less works perfect. > > Since this contradicts Timo's recommendation not to use more than 10, it > sounds to me like you either encountered a bug, or possibly it was not > optimally deployed. > > Did you ever come here and ask for help? There have of course been various bugs in director once in a while. Also depends a bit on how director was being used. Currently I'm not aware of any bugs related to it. From matthew.broadhead at nbmlaw.co.uk Sat Feb 25 18:04:58 2017 From: matthew.broadhead at nbmlaw.co.uk (Matthew Broadhead) Date: Sat, 25 Feb 2017 19:04:58 +0100 Subject: sieve_imapsieve centos 7 In-Reply-To: <47deb9a05a53068e82ee26516658ff65@valo.at> References: <5d5d80a9-9ff5-d369-dab5-84e187af56ce@nbmlaw.co.uk> <47deb9a05a53068e82ee26516658ff65@valo.at> Message-ID: <49dab2c3-0a34-5fd9-93dc-4c9aa13f3579@nbmlaw.co.uk> thanks for the help everyone. i reverted to the old antispam plugin as a package from https://copr.fedorainfracloud.org/coprs/cottsay/dovecot-antispam/ plus this to help configure http://www.iredmail.org/forum/topic8169-iredmail-support-antispam-via-dovecot-and-spamassassin.html On 24/02/2017 23:38, Christian Kivalo wrote: > On 2017-02-24 23:16, Matthew Broadhead wrote: >> i am using CentOS 7 centos-release-7-3.1611.el7.centos.x86_64 with >> dovecot dovecot-2.2.10-7.el7.x86_64. i am trying to set up AntiSpam >> with IMAPSieve but the package seems to be lacking sieve_imapsieve. is >> there anything i can do? i am not really interested in compiling from >> source because i like to receive security updates automatically. > The imapsieve plugin for pigeonhole was introduced in version 0.4.14 > for dovecot 2.2.24, with your current packages i'd say there is > nothing you can do except to find some sort of extra packages (i'm not > familiar with centos...) > > http://dovecot.markmail.org/message/mggbfw6vxhs2upa7?q=imapsieve&page=2 > >> 2017-02-24 21:57:00auth: Error: net_connect_unix(anvil-auth-penalty) >> failed: Permission denied >> 2017-02-24 21:57:00master: Warning: Killed with signal 15 (by pid=1 >> uid=0 code=kill) >> 2017-02-24 21:57:00managesieve: Fatal: Plugin 'sieve_imapsieve' not >> found from directory /usr/lib64/dovecot/sieve >> 2017-02-24 21:57:00config: Error: managesieve-login: dump-capability >> process returned 89 >> 2017-02-24 21:57:05imap: Fatal: Plugin 'imap_sieve' not found from >> directory /usr/lib64/dovecot >> 2017-02-24 21:57:05imap: Fatal: Plugin 'imap_sieve' not found from >> directory /usr/lib64/dovecot >> 2017-02-24 21:57:08imap: Fatal: Plugin 'imap_sieve' not found from >> directory /usr/lib64/dovecot >> 2017-02-24 21:57:08imap: Fatal: Plugin 'imap_sieve' not found from >> directory /usr/lib64/dovecot >> 2017-02-24 21:57:10imap: Fatal: Plugin 'imap_sieve' not found from >> directory /usr/lib64/dovecot >> 2017-02-24 21:57:10imap: Fatal: Plugin 'imap_sieve' not found from >> directory /usr/lib64/dovecot > -- Matthew Broadhead NBM Solicitors See the latest jobs available at NBM @www.nbmlaw.co.uk/recruitment.htm 32 Rainsford Road Chelmsford Essex CM1 2QG Tel: 01245 269909 Fax: 01245 261932 www.nbmlaw.co.uk Partners: WJ Broadhead NP Eason SJ Lacey CR Broadhead D Seepaul T Carley NBM Solicitors are authorised and regulated by the Solicitors Regulation Authority. We are also bound by their code of conduct. Registered no. 00061052 NBM also provide a will writing service, see http://www.nbmlaw.co.uk/wills.htm for more information Confidentiality Information in this message is confidential and may be legally privileged. It is intended solely for the recipient to whom it is addressed. If you receive the message in error, please notify the sender and immediately destroy all copies. Security warning Please note that this e-mail has been created in the knowledge that e-mail is not a 100% secure communications medium. We advise you that you understand and observe this lack of security when e-mailing us. This e-mail does not constitute a legally binding document. No contracts may be concluded on behalf of Nigel Broadhead Mynard Solicitors by e-mail communications. If you have any queries, please contact administrator at nbmlaw.co.uk From ruga at protonmail.com Sat Feb 25 19:11:42 2017 From: ruga at protonmail.com (Ruga) Date: Sat, 25 Feb 2017 14:11:42 -0500 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) Message-ID: t_strftime and variants now .......................................... : ok test-time-util.c:123: Assert failed: strcmp(t_strftime(RFC2822_FMT, gmtime(&ts)), exp) == 0 test-time-util.c:124: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) == 0 t_strftime and variants fixed timestamp .............................. : FAILED timings 0 ............................................................ : ok -------------- next part -------------- A non-text attachment was scrubbed... Name: configure.ac.patch Type: application/octet-stream Size: 842 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dcrypt-openssl.c.patch Type: application/octet-stream Size: 2777 bytes Desc: not available URL: From tss at iki.fi Sat Feb 25 19:54:11 2017 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2017 21:54:11 +0200 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: References: Message-ID: On 25 Feb 2017, at 21.11, Ruga wrote: > > t_strftime and variants now .......................................... : ok > > > test-time-util.c:123: Assert failed: strcmp(t_strftime(RFC2822_FMT, gmtime(&ts)), exp) == 0 > > > test-time-util.c:124: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) == 0 > > > t_strftime and variants fixed timestamp .............................. : FAILED What OS is this? Could you try with the attached patch what it logs? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 554 bytes Desc: not available URL: -------------- next part -------------- > Why do you want to disable /dev/urandom? > Oh, I forgot to remove the #if OPENSSL_VERSION_NUMBER checks from lib-dcrypt. Will be removed in v2.2.29. Attached the planned patch that should do it. -------------- next part -------------- A non-text attachment was scrubbed... Name: openssl.diff Type: application/octet-stream Size: 6263 bytes Desc: not available URL: From tss at iki.fi Sat Feb 25 20:08:40 2017 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2017 22:08:40 +0200 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: References: Message-ID: <5FACDAE6-952E-42EF-8A7F-20C89B2B22BC@iki.fi> On 25 Feb 2017, at 21.54, Timo Sirainen wrote: > > Oh, I forgot to remove the #if OPENSSL_VERSION_NUMBER checks from lib-dcrypt. Will be removed in v2.2.29. Attached the planned patch that should do it. Well that didn't work with From aki.tuomi at dovecot.fi Sat Feb 25 21:40:44 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sat, 25 Feb 2017 23:40:44 +0200 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: <5FACDAE6-952E-42EF-8A7F-20C89B2B22BC@iki.fi> References: <5FACDAE6-952E-42EF-8A7F-20C89B2B22BC@iki.fi> Message-ID: t?st? j?i p?tsi =) Aki On 2017-02-25 22:08, Timo Sirainen wrote: > On 25 Feb 2017, at 21.54, Timo Sirainen wrote: >> Oh, I forgot to remove the #if OPENSSL_VERSION_NUMBER checks from lib-dcrypt. Will be removed in v2.2.29. Attached the planned patch that should do it. > Well that didn't work with > > > From aki.tuomi at dovecot.fi Sat Feb 25 21:42:40 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Sat, 25 Feb 2017 23:42:40 +0200 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: References: <5FACDAE6-952E-42EF-8A7F-20C89B2B22BC@iki.fi> Message-ID: Silly thunderbird, does not understand that "reply to sender" should reply to sender... Aki On 2017-02-25 23:40, Aki Tuomi wrote: > t?st? j?i p?tsi =) > > Aki > > > On 2017-02-25 22:08, Timo Sirainen wrote: >> On 25 Feb 2017, at 21.54, Timo Sirainen wrote: >>> Oh, I forgot to remove the #if OPENSSL_VERSION_NUMBER checks from >>> lib-dcrypt. Will be removed in v2.2.29. Attached the planned patch >>> that should do it. >> Well that didn't work with > >> >> >> From pch at myzel.net Sun Feb 26 10:47:25 2017 From: pch at myzel.net (Peter Chiochetti) Date: Sun, 26 Feb 2017 11:47:25 +0100 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: References: <5FACDAE6-952E-42EF-8A7F-20C89B2B22BC@iki.fi> Message-ID: <2f83008d-7d81-e40f-82f3-754a72649f49@myzel.net> Am 2017-02-25 um 22:42 schrieb Aki Tuomi: > Silly thunderbird, does not understand that "reply to sender" should > reply to sender... I think it is correct, as the sender explicitly stated: Reply-To: Dovecot Mailing List -- peter From ruga at protonmail.com Sun Feb 26 12:42:05 2017 From: ruga at protonmail.com (Ruga) Date: Sun, 26 Feb 2017 07:42:05 -0500 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: References: Message-ID: Timo, re: What OS is this? OS 10.12.3 with Xcode 8.2.1 and the official clang 3.9.0 re: test-time-util.c t_strftime and variants now .......................................... : ok Info: 'Thu, 08 Dec 2016 18:42:16 +0100' test-time-util.c:124: Assert failed: strcmp(t_strftime(RFC2822_FMT, gmtime(&ts)), exp) == 0 Info: 'Thu, 08 Dec 2016 18:42:16 +0100' test-time-util.c:126: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) == 0 t_strftime and variants fixed timestamp .............................. : FAILED timings 0 ............................................................ : ok re: Why do you want to disable /dev/urandom? https://github.com/libressl-portable/portable/issues/278 re: dcrypt-openssl.c, ssl.m4 and HMAC_CTX_free(NULL) Please discuss this with busterb at GitHub. -------- Original Message -------- Subject: Re: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) Local Time: 25 February 2017 8:54 PM UTC Time: 25 February 2017 19:54 From: tss at iki.fi To: Ruga Dovecot Mailing List On 25 Feb 2017, at 21.11, Ruga wrote: > > t_strftime and variants now .......................................... : ok > > > test-time-util.c:123: Assert failed: strcmp(t_strftime(RFC2822_FMT, gmtime(&ts)), exp) == 0 > > > test-time-util.c:124: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) == 0 > > > t_strftime and variants fixed timestamp .............................. : FAILED What OS is this? Could you try with the attached patch what it logs? > Why do you want to disable /dev/urandom? > Oh, I forgot to remove the #if OPENSSL_VERSION_NUMBER checks from lib-dcrypt. Will be removed in v2.2.29. Attached the planned patch that should do it. From ruga at protonmail.com Sun Feb 26 12:44:53 2017 From: ruga at protonmail.com (Ruga) Date: Sun, 26 Feb 2017 07:44:53 -0500 Subject: v2.2.28: var-expand.c Message-ID: var-expand.c:687:17: warning: passing an object that undergoes default argument promotion to 'va_start' has undefined behavior [-Wvarargs] va_start(args, key2); ^ var-expand.c:674:58: note: parameter of type 'char' is declared here var_expand_table_build(char key, const char *value, char key2, ...) ^ From tss at iki.fi Sun Feb 26 13:52:19 2017 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2017 15:52:19 +0200 Subject: v2.2.28: patches (to use libressl 2.4.5) and test error (strftime) In-Reply-To: References: Message-ID: On 26 Feb 2017, at 14.42, Ruga wrote: > > Timo, > > re: What OS is this? > OS 10.12.3 with Xcode 8.2.1 and the official clang 3.9.0 > > re: test-time-util.c > > > t_strftime and variants now .......................................... : ok > > > Info: 'Thu, 08 Dec 2016 18:42:16 +0100' > > > test-time-util.c:124: Assert failed: strcmp(t_strftime(RFC2822_FMT, gmtime(&ts)), exp) == 0 > > > Info: 'Thu, 08 Dec 2016 18:42:16 +0100' > > > test-time-util.c:126: Assert failed: strcmp(t_strfgmtime(RFC2822_FMT, ts), exp) == 0 > > > t_strftime and variants fixed timestamp .............................. : FAILED Fixed: https://github.com/dovecot/core/commit/3e6f1c0a999cc0abfc05d817dd89f39fb9337e76 > re: Why do you want to disable /dev/urandom? > https://github.com/libressl-portable/portable/issues/278 That is fixed in v2.2.28 already. > re: dcrypt-openssl.c, ssl.m4 and HMAC_CTX_free(NULL) > Please discuss this with busterb at GitHub. Added a comment there, but I guess the changes should work fine. From tss at iki.fi Sun Feb 26 13:54:46 2017 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2017 15:54:46 +0200 Subject: v2.2.28: var-expand.c In-Reply-To: References: Message-ID: On 26 Feb 2017, at 14.44, Ruga wrote: > > var-expand.c:687:17: warning: passing an object that undergoes default argument promotion to 'va_start' has undefined behavior [-Wvarargs] > > > va_start(args, key2); > > > ^ > > > var-expand.c:674:58: note: parameter of type 'char' is declared here > > > var_expand_table_build(char key, const char *value, char key2, ...) > > > ^ Yeah, the API is just bad. That function is removed from v2.3 already. I didn't want to break v2.2 backwards compatibility. From stephan at rename-it.nl Sun Feb 26 13:59:29 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 26 Feb 2017 14:59:29 +0100 Subject: Replication sieve scripts. In-Reply-To: References: Message-ID: Op 7/19/2016 om 11:30 AM schreef Luescher Claude: > Hello, > > Following up on old thread: > > http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html > > I have the exact same issue with 2.2.10: > > ii dovecot-antispam 2.0+20130822-2 > amd64 Dovecot plugins for training spam filters > ii dovecot-core 1:2.2.10-1 > amd64 secure POP3/IMAP server - core files > ii dovecot-imapd 1:2.2.10-1 > amd64 secure POP3/IMAP server - IMAP daemon > ii dovecot-ldap 1:2.2.10-1 > amd64 secure POP3/IMAP server - LDAP support > ii dovecot-lmtpd 1:2.2.10-1 > amd64 secure POP3/IMAP server - LMTP server > ii dovecot-managesieved 1:2.2.10-1 > amd64 secure POP3/IMAP server - ManageSieve server > ii dovecot-mysql 1:2.2.10-1 > amd64 secure POP3/IMAP server - MySQL support > ii dovecot-pop3d 1:2.2.10-1 > amd64 secure POP3/IMAP server - POP3 daemon > ii dovecot-sieve 1:2.2.10-1 > amd64 secure POP3/IMAP server - Sieve filters support > > As even his version should already support sieve replication through > dsync mine would have to support it for sure but I would like to hear > a confirmation from the developers. > > The mail sync just works perfectly between the 2 nodes, it is only the > sieve scripts which don't get replicated. > The following bugs were fixed recently: https://github.com/dovecot/core/commit/b4adb461ce12bf578d2d70806b205cf3cbf1a51d https://github.com/dovecot/core/commit/27ccbb0f36e07141785db94557afb63a2aa9eeba I wonder whether this also applies to your problem. Regards, Stephan. From stephan at rename-it.nl Sun Feb 26 13:59:35 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 26 Feb 2017 14:59:35 +0100 Subject: Sieve Script Replication Gliches (Report #2) In-Reply-To: <4ece61c7-5950-9231-7efe-cf2eb9e270b1@reub.net> References: <4ece61c7-5950-9231-7efe-cf2eb9e270b1@reub.net> Message-ID: <97768cf1-070e-8cc8-ff46-d2bac2df719f@rename-it.nl> Op 7/31/2016 om 4:27 AM schreef Reuben Farrelly: > Hi, > > I've observed some odd behaviour with dsync replication between two > hosts, specifically to do with sieve script replication. > > In short, I have two hosts which replicate in a master-master type > setup where almost all of the reads and writes happen to just one of > the two hosts. > > They are both running 2.2.devel (9dc6403), which is close to the > latest 2.2 -git . Pigeonhole is running master-0.4 . This is on > x86_64 Gentoo. > > Normal mail replication between Maildir's for all users works fine, > however it appears that something recently committed to the code has > broken sieve script replication between the two. I am sure this did > once work. Replication is via tcps: . > > Sieve scripts on the lesser-used host are not up to date by an order > of days/weeks with the main host and they don't seem to re-replicate - > even if the rules don't exist at all on the replica. > > The symptoms and effects look to be the same as this (unanswered) post > from December: > > http://dovecot.org/list/dovecot/2015-December/102690.html > > I am not sure how to view the transaction log files, but I am seeing > the same symptoms, ie no live replication, and on the lesser-used host > almost all the scripts were old and some had the 1970 date on them. > > Even after forcing a [dsync replication replicate '*'] the scripts are > not replicated. As it stands now there are no sieve scripts on one of > the two members and the system seems unable to replicate by itself. The following bugs were fixed recently: https://github.com/dovecot/core/commit/b4adb461ce12bf578d2d70806b205cf3cbf1a51d https://github.com/dovecot/core/commit/27ccbb0f36e07141785db94557afb63a2aa9eeba I wonder whether this also applies to your problem. Regards, Stephan. From stephan at rename-it.nl Sun Feb 26 14:30:44 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 26 Feb 2017 15:30:44 +0100 Subject: Sieve removeflag Action In-Reply-To: <20170224093057.GA57205@nihlus.leuxner.net> References: <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> <20170220115644.GA14379@nihlus.leuxner.net> <20170222194041.GA3860@nihlus.leuxner.net> <865f781e-1e1e-dc69-4f6b-2326a01df195@rename-it.nl> <20170224093057.GA57205@nihlus.leuxner.net> Message-ID: <2a5c1e3d-e4e9-190d-9ff4-4f2afe15f3d2@rename-it.nl> Op 2/24/2017 om 10:30 AM schreef Thomas Leuxner: > * Stephan Bosch 2017.02.24 10:20: > >> Could you show me your full configuration (`dovecot -n`)? >> >> Regards, >> >> Stephan > Live configuration and scripts sent off-list. Tried this with your full scripts, part of your public mailbox configuration and this test message: From: User To: stephan at rename-it.nl Subject: Frop! List-Id: Frop! Tried manual delivery several times, with equal results: stephan at drieka:~$ /usr/lib/dovecot/dovecot-lda -p ~/message.eml stephan at drieka:~$ doveadm -f table fetch "uid flags" mailbox ":public/Newsletters/Debian/Security" uid flags 1 \Recent 2 \Recent 3 \Recent stephan at drieka:~$ doveadm -f table fetch "uid flags" mailbox "INBOX" uid flags 14 \Flagged \Recent $MailFlagBit1 15 \Flagged \Recent $MailFlagBit1 16 \Flagged \Recent $MailFlagBit1 So, I cannot reproduce your problem. Could you try this at your end? Regards, Stephan. From stephan at rename-it.nl Sun Feb 26 18:49:03 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 26 Feb 2017 19:49:03 +0100 Subject: Sieve removeflag Action In-Reply-To: References: <20170113132219.GA62702@nihlus.leuxner.net> <20170119090706.GA24845@nihlus.leuxner.net> <0fa34b34-986f-7023-572d-202986ec7dc4@rename-it.nl> <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> <20170220115644.GA14379@nihlus.leuxner.net> <20170222194041.GA3860@nihlus.leuxner.net> <865f781e-1e1e-dc69-4f6b-2326a01df195@rename-it.nl> <20170224093057.GA57205@nihlus.leuxner.net> <2a5c1e3d-e4e9-190d-9ff4-4f2afe15f3d2@rename-it.nl> <695244E8-10E3-4355-B1A9-0C0976767FB4@leuxner.net> Message-ID: Op 2/26/2017 om 4:02 PM schreef Thomas Leuxner: >> Am 26.02.2017 um 15:52 schrieb Thomas Leuxner : >> >>> So, I cannot reproduce your problem. >>> >>> Could you try this at your end? >> Hi Stephan, >> >> Tried with: >> >> $ /usr/lib/dovecot/dovecot-lda -d tlx at leuxner.net -p ./message.eml >> >> From: User >> To: lists at example.org >> Subject: Frop! >> List-Id: >> >> Frop! >> >> It goes right to the INBOX with no flags: >> > This time *with sieve* enabled in mail_plugins for LDA. (I?m using LMTP). The mail gets processed correctly. Does it make a difference for LMTP from the code? That should not matter normally. I tested this at my end, but I saw no different behavior. I created the following file in ~/message-lmtp.eml (substitute MAIL FROM/RCPT TO with whatever you need): LHLO frop MAIL FROM: RCPT TO: DATA From: User To: stephan at rename-it.nl Subject: Frop! List-Id: Frop! . Then I invoked LMTP through its unix socket (using socat tool): stephan at drieka:~$ cat ~/message-lmtp.eml | socat gopen:/var/run/dovecot/lmtp stdio 220 drieka.tds Dovecot ready. 250-drieka.tds 250-8BITMIME 250-ENHANCEDSTATUSCODES 250 PIPELINING 250 2.1.0 OK 250 2.1.5 OK 354 OK 250 2.0.0 1D/3FPsfs1ge/QAASZWA2w Saved Evaluating the results shows that there is still no problem here, since the last delivery is fine (I initially forgot to enable Sieve for LMTP): stephan at drieka:~$ doveadm -f table fetch "uid flags" mailbox "INBOX" uid flags 14 \Flagged \Recent $MailFlagBit1 15 \Flagged \Recent $MailFlagBit1 16 \Flagged \Recent $MailFlagBit1 17 \Flagged \Recent $MailFlagBit1 18 \Flagged \Recent $MailFlagBit1 19 \Recent 20 \Flagged \Recent $MailFlagBit1 stephan at drieka:~$ doveadm -f table fetch "uid flags" mailbox ":public/Newsletters/Debian/Security" uid flags 1 \Recent 2 \Recent 3 \Recent 4 \Recent 5 \Recent 6 \Recent Regards, Stephan. From ygrishin-lists at mail2.ca Sun Feb 26 18:51:34 2017 From: ygrishin-lists at mail2.ca (ygrishin-lists at mail2.ca) Date: Sun, 26 Feb 2017 11:51:34 -0700 Subject: Dict quota calculation errors "remote disconnected"/"broken pipe" on 2.22. Message-ID: <4e2ef92843314fdd8aeca74160767ee3@mail2.ca> Solved the problem, reporting back to the community. /etc/dovecot/dovecot-dict-sql-user.conf had been lacking dovecot group permissions. It was 700/root:root. However why it wasn't reported by Dovecot explicitly in the log -- the greatest mystery to me. Now, after deleting dovecot and all its packages via 'apt' and installing afresh makes "lacking permissions for the file" entry to appear in the log. I can confirm that Dict-quota works perfectly well with Dovecot 2.2.22. From tlx at leuxner.net Sun Feb 26 19:51:17 2017 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 26 Feb 2017 20:51:17 +0100 Subject: Sieve removeflag Action In-Reply-To: References: <20170119094251.GA32291@nihlus.leuxner.net> <96c19bb2-1702-ed78-b4d8-d14e601fcfc4@rename-it.nl> <20170220115644.GA14379@nihlus.leuxner.net> <20170222194041.GA3860@nihlus.leuxner.net> <865f781e-1e1e-dc69-4f6b-2326a01df195@rename-it.nl> <20170224093057.GA57205@nihlus.leuxner.net> <2a5c1e3d-e4e9-190d-9ff4-4f2afe15f3d2@rename-it.nl> <695244E8-10E3-4355-B1A9-0C0976767FB4@leuxner.net> Message-ID: <20170226195032.GA29202@nihlus.leuxner.net> * Stephan Bosch 2017.02.26 19:49: > I created the following file in ~/message-lmtp.eml (substitute MAIL > FROM/RCPT TO with whatever you need): > > LHLO frop > MAIL FROM: > RCPT TO: > DATA > From: User > To: stephan at rename-it.nl > Subject: Frop! > List-Id: > > Frop! > . I tested a couple of messages via the socket and it worked, _even_ with a message that previously has been delivered without flags but should have - which is odd to say at least... Still can't find a scheme to narrow it down except it only happens to rules on the included script. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From stephan at rename-it.nl Sun Feb 26 22:36:12 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 26 Feb 2017 23:36:12 +0100 Subject: Released Pigeonhole v0.4.17 for Dovecot v2.2.28. Message-ID: <8cb94154-6627-1bc1-66f1-c5af0040ea5e@rename-it.nl> Hello Dovecot users, Here's the definitive 0.4.17 release. There were no changes since the release candidate. Changelog v0.4.17: - LDA Sieve plugin: Fixed handling of an early explicit keep during multiscript execution. Action side-effects and the message snapshot would be lost at the final stage where the implicit keep is evaluated. This could result in the IMAP flags assigned to the message to be forgotten or that headers modified by the "editheader" extension would revert to their original state. - file script storage: Amended the up-to-date time stamp comparison for on-disk binaries to include nanoseconds. This will fix problems occurring when both binary and script are saved within the same second. This fix is ineffective on older systems that have no support for nanoseconds in stat() timestamps, which should be pretty rare nowadays. - file script storage: Improve saving and listing permission error to include more details. - imapsieve plugin: Make sure "INBOX" is upper case in static mailbox rules. Otherwise, the mailbox name would never match, since matching is performed case-sensitively and Dovecot only returns the upper-cased "INBOX". - imapsieve plugin: Fixed assert failure occurring when used with virtual mailboxes. - doveadm sieve plugin: Fixed crash when setting Sieve script via attribute's string value. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.17.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.17.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From sami.ketola at dovecot.fi Mon Feb 27 09:40:14 2017 From: sami.ketola at dovecot.fi (Sami Ketola) Date: Mon, 27 Feb 2017 11:40:14 +0200 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: > On 24 Feb 2017, at 21.28, Mark Moseley wrote: > Attached. No claims are made on the quality of my code :) > With recent dovecots you probably should not use set_host_weight( server, '0? ) to mark backend down but instead should use director commands HOST-DOWN and HOST-UP in combination with HOST-FLUSH. Sami From mail at tomsommer.dk Mon Feb 27 10:20:37 2017 From: mail at tomsommer.dk (Tom Sommer) Date: Mon, 27 Feb 2017 11:20:37 +0100 Subject: Director+NFS Experiences In-Reply-To: References: Message-ID: <4562b72a5fabac378704333117342729@tomsommer.dk> On 2017-02-27 10:40, Sami Ketola wrote: >> On 24 Feb 2017, at 21.28, Mark Moseley wrote: >> Attached. No claims are made on the quality of my code :) >> > > > With recent dovecots you probably should not use set_host_weight( > server, '0? ) to mark backend > down but instead should use director commands HOST-DOWN and HOST-UP in > combination with HOST-FLUSH. This is already the case in the latest version of Poolmon From dluke at geeklair.net Mon Feb 27 16:52:09 2017 From: dluke at geeklair.net (Daniel J. Luke) Date: Mon, 27 Feb 2017 11:52:09 -0500 Subject: indexer-worker assert in v2.2.28 (fts-lucene) Message-ID: I upgraded to 2.2.28 and started seeing this logged: indexer-worker(dluke): Panic: file mailbox-list.c: line 1158 (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, strlen(root_dir)) == 0) indexer-worker(dluke): Error: Raw backtrace: 2 libdovecot.0.dylib 0x000000010ec63d24 default_fatal_finish + 36 -> 3 libdovecot.0.dylib 0x000000010ec64b5b i_internal_fatal_handler + 43 -> 4 libdovecot.0.dylib 0x000000010ec64039 i_panic + 169 -> 5 libdovecot-storage.0.dylib 0x000000010eac0950 mailbox_list_try_mkdir_root + 1248 -> 6 libdovecot-storage.0.dylib 0x000000010eac0a39 mailbox_list_mkdir_root + 25 -> 7 lib21_fts_lucene_plugin.so 0x0000000110adc949 fts_backend_lucene_update_set_build_key + 73 -> 8 lib20_fts_plugin.so 0x000000010ee736cf fts_backend_update_set_build_key + 79 -> 9 lib20_fts_plugin.so 0x000000010ee74317 fts_build_mail + 599 -> 10 lib20_fts_plugin.so 0x000000010ee7981a fts_mail_precache + 794 -> 11 libdovecot-storage.0.dylib 0x000000010eaa71e9 mail_precache + 25 -> 12 indexer-worker 0x000000010ea9e503 master_connection_input + 1523 -> 13 libdovecot.0.dylib 0x000000010ec78b89 io_loop_call_io + 89 -> 14 libdovecot.0.dylib 0x000000010ec7a96d io_loop_handler_run_internal + 269 -> 15 libdovecot.0.dylib 0x000000010ec7907f io_loop_handler_run + 303 -> 16 libdovecot.0.dylib 0x000000010ec78e58 io_loop_run + 88 -> 17 libdovecot.0.dylib 0x000000010ec03458 master_service_run + 24 -> 18 indexer-worker 0x000000010ea9ddb4 main + 340 -> 19 libdyld.dylib 0x00007fffba7df255 start + 1 Anyone else? -- Daniel J. Luke From stephan at rename-it.nl Mon Feb 27 22:03:44 2017 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 27 Feb 2017 23:03:44 +0100 Subject: Archive key for the Dovecot Automatic Debian Package Archive (Xi) updated Message-ID: <1ddc457c-9270-39b9-dce5-dacb72094e91@rename-it.nl> Hi, I've updated the archive key for the Xi archive. So, when updating, you will initially get a key error. To fix this, either upgrade the debian-dovecot-auto-keyring package (preferred), or update your key manually from the archive.key located in the repository root. Regards, Stephan. From me at christoph-kluge.eu Mon Feb 27 22:36:07 2017 From: me at christoph-kluge.eu (Christoph Kluge) Date: Mon, 27 Feb 2017 23:36:07 +0100 Subject: Fwd: Some mails do not get replicated anymore after memory-exhaust In-Reply-To: References: Message-ID: Hey guys, overall I have an working dovecot replication between 2 servers running on amazon cloud. Sadly I had some messages that my server ran out of memory. After investigating a little bit further I realized that some mails didn't got replicated, but I'm not sure if this was related to the memory exhaust. I was expecting that the full-sync would catch them up but sadly it's not. Attached I'm adding: * /etc/dovecot/dovecot.conf from both servers * one sample of my memory-exhaust exception * maildir directory listing of one mailbox on both servers * commands + outpot of manual attempt for full-replication * grep information of missing mail inside Maildir on both servers Here is my configuration from both servers. The configugration is 1:1 the same except the mail_replica server. Please note one server runs on debian 8.7 and the other one on 7.11. ---- SERVER A > # dovecot -n > # 2.2.13: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 8.7 > ---- SERVER B > # dovecot -n > # 2.2.13: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-34-pve i686 Debian 7.11 > auth_mechanisms = plain login > disable_plaintext_auth = no > doveadm_password = **** > doveadm_port = 12345 > listen = *,[::] > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_max_userip_connections = 100 > mail_plugins = notify replication quota > mail_privileged_group = vmail > passdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > plugin { > mail_replica = tcp:*.****.de > quota = dict:user::file:/var/vmail/%d/%n/.quotausage > replication_full_sync_interval = 1 hours > sieve = /var/vmail/%d/%n/.sieve > sieve_max_redirects = 25 > } > protocols = imap > replication_max_conns = 2 > service aggregator { > fifo_listener replication-notify-fifo { > mode = 0666 > user = vmail > } > unix_listener replication-notify { > mode = 0666 > user = vmail > } > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = root > } > service config { > unix_listener config { > user = vmail > } > } > service doveadm { > inet_listener { > port = 12345 > } > user = vmail > } > service imap-login { > client_limit = 1000 > process_limit = 512 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > service replicator { > process_min_avail = 1 > unix_listener replicator-doveadm { > mode = 0666 > } > } > ssl_cert = ssl_key = ssl_protocols = !SSLv2 !SSLv3 > userdb { > driver = prefetch > } > userdb { > args = /etc/dovecot/dovecot-sql.conf > driver = sql > } > protocol imap { > mail_plugins = notify replication quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = notify replication quota sieve > postmaster_address = webmaster at localhost > } > protocol lmtp { > mail_plugins = notify replication quota sieve > postmaster_address = webmaster at localhost > } This is the exception which I got several times: Feb 26 16:16:39 mx dovecot: replicator: Panic: data stack: Out of memory > when allocating 268435496 bytes > Feb 26 16:16:39 mx dovecot: replicator: Error: Raw backtrace: > /usr/lib/dovecot/libdovecot.so.0(+0x6b6fe) [0x7f7ca2b0a6fe] -> > /usr/lib/dovecot/libdovecot.so.0(+0x6b7ec) [0x7f7ca2b0a7ec] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7ca2ac18fb] -> > /usr/lib/dovecot/libdovecot.so.0(+0x6977e) [0x7f7ca2b0877e] -> > /usr/lib/dovecot/libdovecot.so.0(+0x699db) [0x7f7ca2b089db] -> > /usr/lib/dovecot/libdovecot.so.0(+0x82198) [0x7f7ca2b21198] -> > /usr/lib/dovecot/libdovecot.so.0(+0x6776d) [0x7f7ca2b0676d] -> > /usr/lib/dovecot/libdovecot.so.0(buffer_write+0x6c) [0x7f7ca2b069dc] -> > dovecot/replicator(replicator_queue_push+0x14e) [0x7f7ca2fa17ae] -> > dovecot/replicator(+0x4f9e) [0x7f7ca2fa0f9e] -> dovecot/replicator(+0x4618) > [0x7f7ca2fa0618] -> dovecot/replicator(+0x4805) [0x7f7ca2fa0805] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x3f) [0x7f7ca2b1bd0f] > -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xf9) > [0x7f7ca2b1cd09] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) > [0x7f7ca2b1bd79] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f7ca2b1bdf8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f7ca2ac6dc3] -> dovecot/replicator(main+0x195) [0x7f7ca2f9f8b5] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7ca2715b45] > -> dovecot/replicator(+0x395d) [0x7f7ca2f9f95d] > Feb 26 16:16:39 mx dovecot: imap(***.com): Warning: replication(***.com): > Sync failure: > Feb 26 16:16:39 mx dovecot: replicator: Fatal: master: > service(replicator): child 24012 killed with signal 6 (core dumps disabled) This is the current maildir listing on Server A # ls -la /var/vmail/*.eu/*h/Maildir/new/ > total 24 > drwx------ 2 vmail vmail 4096 Feb 27 18:12 . > drwx------ 15 vmail vmail 4096 Feb 27 21:47 .. > -rw------- 1 vmail vmail 3600 Feb 27 14:49 1488206976.M277562P25620.mail, > S=3600,W=3671 > -rw------- 1 vmail vmail 4390 Feb 27 15:17 1488208642.M513542P27111.mail, > S=4390,W=4478:2,S > -rw------- 1 vmail vmail 3577 Feb 27 16:32 1488213157.M307300P30773.mail, > S=3577,W=3648:2,S This is the current maildir listing on Server B # ls -la /var/vmail/*.eu/*h/Maildir/new/ > total 16 > drwx------ 2 vmail vmail 12288 Feb 27 16:45 . > drwx------ 15 vmail vmail 4096 Feb 27 21:47 .. This is how I tried to manually sync it doveadm -v sync -u *h@*.eu -f tcp:mx.***.de:12345 This is the users sync status # doveadm replicator status 'cheecoh at ragequit.eu' > username priority fast sync full sync failed > *h@*.eu none 00:24:47 10:57:04 - Then I tried to lookup for the mail-id which is also the same on both servers # grep -ri "M277562P25620" /var/vmail/*.eu/*h/ > /var/vmail/*.eu/*h/Maildir/dovecot-uidlist:493 :1488206976.M277562P25620. > mail,S=3600,W=3671 I have no idea what else I could do. I could also pass a "doveadm -Dv sync" output but this one is really huge.. Best Regards Christoph Kluge From dovecot at allycomm.com Mon Feb 27 22:38:20 2017 From: dovecot at allycomm.com (Jeff Kletsky) Date: Mon, 27 Feb 2017 14:38:20 -0800 Subject: Replacement for antispam plugin In-Reply-To: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> References: <711244819.1410.1486907575875@appsuite-dev.open-xchange.com> Message-ID: <4ae98de8-6ea5-cb1b-2de6-6bebb7cf965e@wagsky.com> Glad I poked around on the list today! Thanks to all for the suggestions about integration with dspam. I'll definitely have to look into this, as I rely on moving messages to a specific folder with various IMAP clients to retrain dspam false positives and negatives. A quick pair of questions: * Does Dovecot support the IMAP "MOVE" command at this time? * If so, what is the syntax for "COPY or MOVE" for the _causes variables? I did see messages from 2011 discussing it, but nothing since. While the script looks like it could be modified for use with dspam (with the great suggestions from others on the list), it has the same problem as "antispam" with bulk moves being serialized and tying of the client until they complete. I'll probably have to break down and look into using FreeBSD's auditd to trigger the actions and then de-queue the successfully processed messages. Sieve doesn't look like it can handle asynchronous processing, but I'd certainly be interested if I'm missing something there. One less thing to configure and maintain! Jeff On 2/12/17 5:52 AM, Aki Tuomi wrote: >> On February 10, 2017 at 10:06 AM Aki Tuomi wrote: >> >> >> Hi! >> Since antispam plugin is deprecated and we would really prefer people >> not to use it, we wrote instructions on how to replace it with >> IMAPSieve. Comments and suggestions are most welcome. >> >> https://wiki.dovecot.org/HowTo/AntispamWithSieve >> >> --- >> Aki Tuomi >> Dovecot oy > Hi everyone, > > thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script. > > Aki > From peter at pajamian.dhs.org Tue Feb 28 04:14:51 2017 From: peter at pajamian.dhs.org (Peter Ajamian) Date: Tue, 28 Feb 2017 17:14:51 +1300 Subject: make check failing in CentOS 6 Message-ID: <5ddf52d6-9906-d56d-a2c5-2d584bf753b5@pajamian.dhs.org> Dovecot builds just fine, but fails the tests in src/lib-index. Note that reverting this commit fixes the issue: https://github.com/dovecot/core/commit/dfa4b048ec9a174a42d6668e94501db2fb70793a $ make check for bin in test-mail-index-map test-mail-index-modseq test-mail-index-sync-ext test-mail-index-transaction-finish test-mail-index-transaction-update test-mail-transaction-log-append test-mail-transaction-log-view; do \ if ! ./$bin; then exit 1; fi; \ done mail index map lookup seq range ...................................... : ok 0 / 1 tests failed mail_transaction_log_file_get_modseq_next_offset() ................... : ok 0 / 1 tests failed mail index sync ext atomic inc ....................................... : ok 0 / 1 tests failed mail index transaction finish flag updates n_so_far=0 ................ : ok mail index transaction finish flag updates n_so_far=1 ................ : ok mail index transaction finish flag updates n_so_far=2 ................ : ok mail index transaction finish check conflicts n_so_far=0 ............. : ok mail index transaction finish check conflicts n_so_far=1 ............. : ok mail index transaction finish check conflicts n_so_far=2 ............. : ok mail index transaction finish modseq updates n_so_far=0 .............. : ok mail index transaction finish modseq updates n_so_far=1 .............. : ok mail index transaction finish modseq updates n_so_far=2 .............. : ok mail index transaction finish expunges n_so_far=0 .................... : ok mail index transaction finish expunges n_so_far=1 .................... : ok mail index transaction finish expunges n_so_far=2 .................... : ok 0 / 12 tests failed mail index append .................................................... : ok mail index append with uids .......................................... : ok mail index flag update fast paths .................................... : ok mail index flag update simple merges ................................. : ok mail index flag update complex merges ................................ : ok mail index flag update random ........................................ : ok mail index flag update appends ....................................... : ok mail index cancel flag updates ....................................... : ok mail index transaction get flag update pos ........................... : ok mail index modseq update ............................................. : ok mail index expunge ................................................... : ok test-mail-index-transaction-update.c:649: Assert(#1) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#1) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#3) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#3) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#4) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:649: Assert(#5) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#5) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#6) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#6) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#7) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#7) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#8) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#8) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#9) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#9) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#10) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:652: Assert(#10) failed: memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, sizeof(uint32_t) * 8) == 0 test-mail-index-transaction-update.c:649: Assert(#11) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone test-mail-index-transaction-update.c:649: Assert(#12) failed: new_hdr.day_stamp == tests[i].new_day_stamp + timezone mail index update day first uid ...................................... : FAILED test: random seed #1 was 1488809888 1 / 12 tests failed make: *** [check-test] Error 1 From dmiller at amfes.com Tue Feb 28 04:07:13 2017 From: dmiller at amfes.com (Daniel Miller) Date: Mon, 27 Feb 2017 20:07:13 -0800 Subject: Solr 6.4.1 config In-Reply-To: <5544df07-8816-2d22-bd51-48de0dcf63f8@amfes.com> References: <91215f50-fff4-b8ad-fa9e-031d18b52d53@amfes.com> <192b8a17-598a-9ea6-8557-3ff0572bbe6f@dovecot.fi> <36bf499b-9e55-d437-1faa-3ce2ff1531a9@dovecot.fi> <5544df07-8816-2d22-bd51-48de0dcf63f8@amfes.com> Message-ID: Since I was recently asked off-list - thought I'd make this available if it will help anyone else. Attached is my (I hope) working 6.4.1 config. Also some instructions: First, you have to decide if you're running in standalone or cloud mode. Standalone is simpler, but I seem to be having better luck running in cloud mode - even though I'm only using a single server/core/shard/collection/whatever. I've attached my config. Unzip this somewhere convenient - I will assume "/tmp/solr-dovecot". You may want to tweak it - this is adapted from my version 4 config and is set for English. If you look in solrbasedir/example/files/conf/lang or /solrbasedir/server/solr/configsets/basic_configs/conf/lang you'll find alternatives for "stopwords.txt" - I simply copied the "stopwords_en.txt" file directly to "conf/stopwords.txt". Assuming you extracted the Solr distribution to a base folder, such as /opt/solr-6.4.1, change to that folder. Then execute (again, /tmp/solr-dovecot is the path you extracted my config into): bin/solr create_collection -c dovecot -d /tmp/solr-dovecot That should create a minimal cloud instance. Then start it: bin/solr start -c And hopefully...you'll have a running Solr server. Assuming that's the case - since you've now created a "named" collection you have to specify that for Dovecot. So the fts_solr parameter in 90-plugin.conf needs to be something like: fts_solr = break-imap-search url=http://ser.ver.ip:8983/solr/dovecot/ Restart Dovecot...and with luck you're searching now. Hope this helps somebody! Daniel On 2/10/2017 3:15 PM, Daniel Miller wrote: > 2.2.18 > > On 2/9/2017 11:13 PM, Aki Tuomi wrote: >> What dovecot version are you using? >> >> >> On 10.02.2017 09:12, Aki Tuomi wrote: >>> Yeah, thanks. >>> >>> It seems that there indeed is content-type header (there should not >>> be). >>> We'll look into it. >>> >>> Aki >>> >>> On 10.02.2017 01:44, Daniel Miller wrote: >>>> Does this work (pcap attached)? >>>> >>>> Daniel >>>> >>>> On 2/8/2017 10:57 PM, Aki Tuomi wrote: >>>>> On 09.02.2017 07:54, Daniel Miller wrote: >>>>>> I've been running Solr for a while (4.10.3) - wanted to make the >>>>>> jump >>>>>> to the latest & greatest. I installed 6.4.1, copied over my >>>>>> schema.xml - and after a couple false starts where I needed to tweak >>>>>> it work with the new version...it works! I did not copy the >>>>>> database, >>>>>> started from scratch, and executed a "doveadm fts rescan -A". >>>>>> But... >>>>>> >>>>>> Judging solely from at least one client - it's fine. But looking in >>>>>> the logs I see: >>>>>> 1. The first scan of a mailbox dovecot's error log gives: >>>>>> dovecot: imap(dmiller at amfes.com): Error: fts_solr: Lookup >>>>>> failed: >>>>>> Bad Request >>>>>> >>>>>> 2. Subsequent scans do not appear to generate any dovecot error >>>>>> logs >>>>>> - but I'm not certain. Each new mailbox/subfolder scanned will each >>>>>> have one error on the initial scan. >>>>>> >>>>>> 3. Solr's log gives me the following - on every search. >>>>>> 2017-02-09 05:50:12.412 ERROR (qtp205125520-15) [ x:dovecot] >>>>>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Bad >>>>>> contentType for search handler :text/xml >>>>>> request={q=from:"test"+OR+to:"test"+OR+cc:"test"+OR+subject:"test"+OR+body:"test"&fl=uid,score&sort=uid+asc&fq=%2Bbox:c1af150abfc9df4d7f7a00003bc41c5f+%2Buser:"dmiller at amfes.com"&rows=67135} >>>>>> >>>>>> >>>>> Hi! >>>>> >>>>> can you please use tcpdump or wireshark to capture the actual HTTP >>>>> request causing this exception? >>>>> >>>>> Aki -------------- next part -------------- A non-text attachment was scrubbed... Name: solrconf.zip Type: application/octet-stream Size: 41041 bytes Desc: not available URL: From azurit at pobox.sk Tue Feb 28 08:11:53 2017 From: azurit at pobox.sk (azurit at pobox.sk) Date: Tue, 28 Feb 2017 09:11:53 +0100 Subject: Index queue Message-ID: <20170228091153.Horde.QjLRajsSJlKg5W_JVdCMQ0a@webmail.inetadmin.eu> Hi, can i, somehow, list mailboxes which are to be indexed by indexer-worker (=index queue?) ? How can i know what part of all mailboxes was indexed so far? Are there any statistics about Solr data dir size based on emails amount? For example, we have about 5TB of emails, what should i except about index size in Solr? Thank you. azur From aki.tuomi at dovecot.fi Tue Feb 28 08:59:21 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 28 Feb 2017 10:59:21 +0200 Subject: make check failing in CentOS 6 In-Reply-To: <5ddf52d6-9906-d56d-a2c5-2d584bf753b5@pajamian.dhs.org> References: <5ddf52d6-9906-d56d-a2c5-2d584bf753b5@pajamian.dhs.org> Message-ID: <13415324-46fd-5265-586b-370948668a8f@dovecot.fi> On 28.02.2017 06:14, Peter Ajamian wrote: > Dovecot builds just fine, but fails the tests in src/lib-index. > > Note that reverting this commit fixes the issue: > https://github.com/dovecot/core/commit/dfa4b048ec9a174a42d6668e94501db2fb70793a > > $ make check > for bin in test-mail-index-map test-mail-index-modseq > test-mail-index-sync-ext test-mail-index-transaction-finish > test-mail-index-transaction-update test-mail-transaction-log-append > test-mail-transaction-log-view; do \ > if ! ./$bin; then exit 1; fi; \ > done > mail index map lookup seq range ...................................... : ok > 0 / 1 tests failed > mail_transaction_log_file_get_modseq_next_offset() ................... : ok > 0 / 1 tests failed > mail index sync ext atomic inc ....................................... : ok > 0 / 1 tests failed > mail index transaction finish flag updates n_so_far=0 ................ : ok > mail index transaction finish flag updates n_so_far=1 ................ : ok > mail index transaction finish flag updates n_so_far=2 ................ : ok > mail index transaction finish check conflicts n_so_far=0 ............. : ok > mail index transaction finish check conflicts n_so_far=1 ............. : ok > mail index transaction finish check conflicts n_so_far=2 ............. : ok > mail index transaction finish modseq updates n_so_far=0 .............. : ok > mail index transaction finish modseq updates n_so_far=1 .............. : ok > mail index transaction finish modseq updates n_so_far=2 .............. : ok > mail index transaction finish expunges n_so_far=0 .................... : ok > mail index transaction finish expunges n_so_far=1 .................... : ok > mail index transaction finish expunges n_so_far=2 .................... : ok > 0 / 12 tests failed > mail index append .................................................... : ok > mail index append with uids .......................................... : ok > mail index flag update fast paths .................................... : ok > mail index flag update simple merges ................................. : ok > mail index flag update complex merges ................................ : ok > mail index flag update random ........................................ : ok > mail index flag update appends ....................................... : ok > mail index cancel flag updates ....................................... : ok > mail index transaction get flag update pos ........................... : ok > mail index modseq update ............................................. : ok > mail index expunge ................................................... : ok > test-mail-index-transaction-update.c:649: Assert(#1) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#1) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#3) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#3) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#4) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:649: Assert(#5) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#5) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#6) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#6) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#7) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#7) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#8) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#8) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#9) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#9) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#10) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:652: Assert(#10) failed: > memcmp(new_hdr.day_first_uid, tests[i].new_day_first_uid, > sizeof(uint32_t) * 8) == 0 > test-mail-index-transaction-update.c:649: Assert(#11) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > test-mail-index-transaction-update.c:649: Assert(#12) failed: > new_hdr.day_stamp == tests[i].new_day_stamp + timezone > mail index update day first uid ...................................... : > FAILED > test: random seed #1 was 1488809888 > 1 / 12 tests failed > make: *** [check-test] Error 1 Hi! We are aware of this and working on fix. Aki From max at kostikov.co Tue Feb 28 11:16:49 2017 From: max at kostikov.co (Max Kostikov) Date: Tue, 28 Feb 2017 13:16:49 +0200 Subject: dovecot-lda crash after upgrade to 2.2.28 Message-ID: Hi! I posted this problem few days ago in FreeBSD bugtracker https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217364 So, the problem is in crash of dovecot-lda client while local mailbox delivery with this message in log > dovecot: lda(my at email): Panic: file mail-namespace.c: line 709 > (mail_namespace_find): assertion failed: (ns != NULL) My Dovecot config stills the same from previous 2.2.27 installation to I believe that problem in new Dovecot code related namespace. System: > # uname -v > FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017 > root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC Options: > root at beta:/usr/ports/mail/dovecot2 # make showconfig | grep =on > DOCS=on: Build and/or install documentation > EXAMPLES=on: Build and/or install examples > KQUEUE=on: kqueue(2) support > LIBWRAP=on: TCP wrapper support > LZ4=on: LZ4 compression support > MYSQL=on: MySQL database support > GSSAPI_BASE=on: Use GSSAPI from base Config: # doveconf -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 11.0-RELEASE-p8 amd64 ufs auth_mechanisms = plain login default_login_user = dovecot dict { sqluserquota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql-user.conf } disable_plaintext_auth = no first_valid_gid = 0 first_valid_uid = 25 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_access_sockets = tcpwrap login_greeting = Dovecot ready! login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_access_groups = mail mail_gid = 6 mail_location = maildir:/var/mail/%d/%n mail_plugins = acl quota trash mail_privileged_group = mail mail_uid = 26 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { location = maildir:/var/mail/%d/.public:INDEXPVT=/var/mail/%d/%n/public prefix = public. separator = . subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile::cache_secs=300 antispam_backend = mailtrain antispam_mail_notspam = %u-revoke antispam_mail_sendmail = /usr/local/etc/dovecot/move-cmd.sh antispam_mail_sendmail_args = antispam_mail_spam = %u-report antispam_spam = Junk antispam_trash = Trash quota = dict:user_quota::proxy::sqluserquota quota_grace = 10%% quota_rule2 = Trash:storage=+10%% quota_rule3 = Junk:storage=+10%% quota_warning = storage=100%% quota-exceeded 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=75%% quota-warning 75 %u sieve = ~/dovecot.sieve sieve_before = /usr/local/etc/dovecot/default.sieve sieve_dir = ~/sieve sieve_global_path = /usr/local/etc/dovecot/default.sieve trash = /usr/local/etc/dovecot/dovecot-trash.conf } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-client { mode = 0660 user = mailnull } unix_listener auth-master { mode = 0660 user = mailnull } user = root } service dict { unix_listener dict { mode = 0660 user = mailnull } } service imap-login { chroot = login client_limit = 64 executable = /usr/local/libexec/dovecot/imap-login inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 32 process_min_avail = 8 service_count = 1 user = $default_login_user vsz_limit = 64 M } service imap { executable = /usr/local/libexec/dovecot/imap } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 1 service_count = 1 user = $default_login_user vsz_limit = 64 M } service managesieve { process_limit = 10 } service pop3-login { chroot = login client_limit = 64 executable = /usr/local/libexec/dovecot/pop3-login inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } process_limit = 32 process_min_avail = 8 service_count = 1 user = $default_login_user vsz_limit = 64 M } service pop3 { executable = /usr/local/libexec/dovecot/pop3 } service quota-warning { executable = script /usr/local/etc/dovecot/quota_warning.sh unix_listener quota-warning { mode = 0660 user = mailnull } } service tcpwrap { unix_listener login/tcpwrap { mode = 0600 user = $default_login_user } } ssl_ca = References: Message-ID: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> On 28.02.2017 13:16, Max Kostikov wrote: > Hi! > > I posted this problem few days ago in FreeBSD bugtracker > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217364 > So, the problem is in crash of dovecot-lda client while > local mailbox delivery with this message in log >> dovecot: lda(my at email): Panic: file mail-namespace.c: line 709 >> (mail_namespace_find): assertion failed: (ns != NULL) > Core backtrace would help alot! Aki Tuomi Dovecot oy From umutkacar at gmail.com Tue Feb 28 13:33:50 2017 From: umutkacar at gmail.com (=?UTF-8?Q?Umut_Erol_Ka=C3=A7ar?=) Date: Tue, 28 Feb 2017 13:33:50 +0000 Subject: Quota usage value shows 140% of actual disk usage In-Reply-To: <1d27e33b-b014-aff4-ef82-d668a213c0b0@luis.uni-hannover.de> References: <1d27e33b-b014-aff4-ef82-d668a213c0b0@luis.uni-hannover.de> Message-ID: You're right, it's the zlib compression. It works as described here: http://wiki2.dovecot.org/Plugins/Zlib It appends the before-compression size to the file name like S= and uses them for quota usage calculation. So, when I run the following command on a Maildir, the result matches the value on the database. (size value on the quota table) find . -type f | grep S= | awk -F, '{ gsub("S=","",$2); SUM+=$2 } END { print SUM }' And the compression ratios relate to the difference between actual disk space usage and calculated quota usage values when I check like this, on a Maildir: find . -type f -ls | grep S= | awk -F, '{ gsub("S=","",$2); print $2, $1 }' | awk '{ print "file: "$12,"\t orig: ",$1," comp.: ",$8," ratio: ",$1 / $8}' Thanks a lot! 25 ?ub 2017 Cmt, 00:07 tarihinde, Karsten Heiken < heiken at luis.uni-hannover.de> ?unu yazd?: > Am 24.02.2017 um 16:00 schrieb Steffen Kaiser: > > > > Quota does not count physical useage, but the amount of bytes allocated > by the messages. Maildir may hardlink messages, hence, they count multiple > times for the quota, but once for du. > > And in your case dovecot even compressed the mails: > According to your doveconf, you are using mail_plugins = [...] zlib. > > Dovecot's quota is calculated using the uncompressed size, whereas du > shows you the space actually allocated. > > From max at kostikov.co Tue Feb 28 14:32:38 2017 From: max at kostikov.co (Max Kostikov) Date: Tue, 28 Feb 2017 16:32:38 +0200 Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> Message-ID: <7a4b6acb95d5183179266446789602ab@kostikov.co> I just recreated configuration of Dovecot 2.2.28 and got the same error but in imap service (it was there in logs too as for dovecot-lda but I don't saw it at time). > Feb 27 20:09:41 beta dovecot: imap(postmaster at peek.ru): Panic: file > mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns > != NULL) > Feb 27 20:09:47 beta dovecot: lda(my at domain.ru): Panic: file > mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns > != NULL) So I show you only imap backtrace below. # gdb /usr/local/libexec/dovecot/imap 1 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Attaching to program: /usr/local/libexec/dovecot/imap, process 1 ptrace: Invalid argument. Core was generated by `imap'. Program terminated with signal 6, Aborted. Reading symbols from /usr/local/lib/dovecot/libdovecot-lda.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/local/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/libkrb5.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libkrb5.so.11 Reading symbols from /usr/lib/libgssapi.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgssapi.so.10 Reading symbols from /usr/lib/libgssapi_krb5.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.10 Reading symbols from /usr/lib/libasn1.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libasn1.so.11 Reading symbols from /usr/lib/libcom_err.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcom_err.so.5 Reading symbols from /lib/libcrypt.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypt.so.5 Reading symbols from /lib/libcrypto.so.8...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypto.so.8 Reading symbols from /usr/lib/libhx509.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libhx509.so.11 Reading symbols from /usr/lib/libroken.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libroken.so.11 Reading symbols from /usr/lib/libwind.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libwind.so.11 Reading symbols from /usr/lib/libheimbase.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libheimbase.so.11 Reading symbols from /usr/lib/libprivateheimipcc.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libprivateheimipcc.so.11 Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/local/lib/dovecot/lib01_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/lib/dovecot/lib02_imap_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/local/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/local/lib/dovecot/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/local/lib/dovecot/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/local/lib/dovecot/lib90_antispam_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /usr/local/lib/compat/pkg/nss_winbind.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/compat/pkg/nss_winbind.so.1 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x000000001117655a in thr_kill () from /lib/libc.so.7 [New Thread 13616000 (LWP 100455/)] (gdb) quit Aki Tuomi ????? 2017-02-28 13:20: > On 28.02.2017 13:16, Max Kostikov wrote: >> Hi! >> >> I posted this problem few days ago in FreeBSD bugtracker >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217364 >> So, the problem is in crash of dovecot-lda client while >> local mailbox delivery with this message in log >>> dovecot: lda(my at email): Panic: file mail-namespace.c: line 709 >>> (mail_namespace_find): assertion failed: (ns != NULL) >> > > Core backtrace would help alot! -- With best regards, Max Kostikov BBM: 24CA5DF8 | W: https://kostikov.co From aki.tuomi at dovecot.fi Tue Feb 28 15:42:22 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 28 Feb 2017 17:42:22 +0200 (EET) Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <7a4b6acb95d5183179266446789602ab@kostikov.co> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> Message-ID: <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> > On February 28, 2017 at 4:32 PM Max Kostikov wrote: > > > I just recreated configuration of Dovecot 2.2.28 and got the same error > but in imap service (it was there in logs too as for dovecot-lda but I > don't saw it at time). > > Feb 27 20:09:41 beta dovecot: imap(postmaster at peek.ru): Panic: file > > mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns > > != NULL) > > Feb 27 20:09:47 beta dovecot: lda(my at domain.ru): Panic: file > > mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns > > != NULL) > > So I show you only imap backtrace below. Can you please issue bt full in gdb and post the output here? Aki From larryrtx at gmail.com Tue Feb 28 15:47:34 2017 From: larryrtx at gmail.com (Larry Rosenman) Date: Tue, 28 Feb 2017 09:47:34 -0600 Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> Message-ID: <1AF13156-0B17-45A8-B888-5A290158B16D@gmail.com> We (FreeBSD) had a user report similar. On 2/28/17, 9:42 AM, "dovecot on behalf of Aki Tuomi" wrote: > On February 28, 2017 at 4:32 PM Max Kostikov wrote: > > > I just recreated configuration of Dovecot 2.2.28 and got the same error > but in imap service (it was there in logs too as for dovecot-lda but I > don't saw it at time). > > Feb 27 20:09:41 beta dovecot: imap(postmaster at peek.ru): Panic: file > > mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns > > != NULL) > > Feb 27 20:09:47 beta dovecot: lda(my at domain.ru): Panic: file > > mail-namespace.c: line 709 (mail_namespace_find): assertion failed: (ns > > != NULL) > > So I show you only imap backtrace below. Can you please issue bt full in gdb and post the output here? Aki From max at kostikov.co Tue Feb 28 15:55:17 2017 From: max at kostikov.co (Max Kostikov) Date: Tue, 28 Feb 2017 17:55:17 +0200 Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> Message-ID: Probably I need to compile 2.2.28 from sources with debug flags. Can you tell me wich flags I need to set for more informative backtrace in gdb? Aki Tuomi ????? 2017-02-28 17:42: > Can you please issue > bt full > in gdb and post the output here? -- With best regards, Max Kostikov BBM: 24CA5DF8 | W: https://kostikov.co -- With best regards, Max Kostikov BBM: 24CA5DF8 | W: https://kostikov.co From bra at fsn.hu Tue Feb 28 15:59:10 2017 From: bra at fsn.hu (Nagy, Attila) Date: Tue, 28 Feb 2017 16:59:10 +0100 Subject: Dict protocol changes string In-Reply-To: <4ee8769a-6462-eac2-5ab2-2887a2aeebe7@dovecot.fi> References: <09ccc308-80b2-f245-00f4-8134e7f7ab96@fsn.hu> <4ee8769a-6462-eac2-5ab2-2887a2aeebe7@dovecot.fi> Message-ID: On 09/23/2016 08:05 AM, Aki Tuomi wrote: > On 29.07.2016 15:35, Nagy, Attila wrote: >> I use pass and userdb with dict protocol in a similar way: >> >> key passdb { >> key = passdb^MAuth-User: %u^MAuth-Pass: %w^MAuth-Protocol: >> %s^MClient-IP: %r >> format = json >> } >> >> (^M is an \r character, inserted with vi CTRL-v + enter) >> >> Until 2.2.24 this has worked, but 2.2.25 seems to convert that ASCII >> 13 into an ASCII 1 and an "r". >> >> Python printout from what I get with 2.2.25: >> >> 'Lshared/passdb\x01rAuth-User: user\x01rAuth-Pass: >> pass\x01rAuth-Protocol: pop3\x01rClient-IP: 1.2.3.4' >> >> Is this change intentional? Why? > Hi! > > Dict protocol escapes you newlines. You are expected to de-escape them > yourself. > > Following escapes are done, you can de-escape them with your client. > > \x00 => \x10 > \x01 => \x11 > \t => \x1t > \r => \x1r > \n => \x1n > > Following up on this: dovecot 2.2.27 and 2.2.28 goes even further (2.2.25 was OK). If a user specifies a password with a % in it, dovecot silently truncates it. So for example if I specify (just to check this simple example is also bad): key passdb { key = %w format = json } and a user tries to log in with the password 'Lofasznehogyma%', dovecot sends the following into the dict socket: 'Lshared/Lofasznehogyma' According to user reports, other characters may also be affected. Could you please fix this? From aki.tuomi at dovecot.fi Tue Feb 28 16:34:17 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 28 Feb 2017 18:34:17 +0200 (EET) Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> Message-ID: <722518410.1399.1488299658357@appsuite-dev.open-xchange.com> Your backtrace would've been informative already if you had issued bt full. Just opening it with gdb is not sufficient. Aki > On February 28, 2017 at 5:55 PM Max Kostikov wrote: > > > Probably I need to compile 2.2.28 from sources with debug flags. > Can you tell me wich flags I need to set for more informative backtrace > in gdb? > > Aki Tuomi ????? 2017-02-28 17:42: > > > Can you please issue > > bt full > > in gdb and post the output here? > > -- > With best regards, > Max Kostikov > > BBM: 24CA5DF8 | W: https://kostikov.co > -- > With best regards, > Max Kostikov > > BBM: 24CA5DF8 | W: https://kostikov.co From tanstaafl at libertytrek.org Wed Feb 22 20:58:08 2017 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Wed, 22 Feb 2017 15:58:08 -0500 Subject: Scaling to 10 Million IMAP sessions on a single server In-Reply-To: <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> References: <0B71DF39-543D-4684-B77B-872B905D0660@my.walr.us> <20170222131249.3f6d7e42@batzmaru.gol.ad.jp> <31B86D98-EC8F-44DE-973C-6DF9126F204B@my.walr.us> <0C8BA615-95F3-425A-B1B1-4F9CFF022239@iki.fi> <2D8578FF-4DBE-45ED-8096-9B7A26FE26D7@my.walr.us> Message-ID: On 2/22/2017, 3:46:08 PM, KT Walrus wrote: > I want to use mdbox format but I have heard that these index files do > get corrupted occasionally and have to be rebuilt (possibly using an > older version of the index file to construct a new one). I worry that > using mdbox might cause my users to see the IMAP flags suddenly reset > back to a previous state (like seeing previously read messages > becoming unread in their mail clients). This is the only reason I haven't moved to mdbox myself. I really, really wish there was a way to not have to worry about losing flags. From max at kostikov.co Tue Feb 28 17:43:29 2017 From: max at kostikov.co (Max Kostikov) Date: Tue, 28 Feb 2017 19:43:29 +0200 Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <722518410.1399.1488299658357@appsuite-dev.open-xchange.com> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> <722518410.1399.1488299658357@appsuite-dev.open-xchange.com> Message-ID: <8443a611e041ce749e6eb903a04e782c@kostikov.co> Got it. Here is full backtrace output. # gdb /usr/local/libexec/dovecot/imap 1 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Attaching to program: /usr/local/libexec/dovecot/imap, process 1 ptrace: Invalid argument. Core was generated by `imap'. Program terminated with signal 6, Aborted. Reading symbols from /usr/local/lib/dovecot/libdovecot-lda.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/local/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/libkrb5.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libkrb5.so.11 Reading symbols from /usr/lib/libgssapi.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgssapi.so.10 Reading symbols from /usr/lib/libgssapi_krb5.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.10 Reading symbols from /usr/lib/libasn1.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libasn1.so.11 Reading symbols from /usr/lib/libcom_err.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcom_err.so.5 Reading symbols from /lib/libcrypt.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypt.so.5 Reading symbols from /lib/libcrypto.so.8...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypto.so.8 Reading symbols from /usr/lib/libhx509.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libhx509.so.11 Reading symbols from /usr/lib/libroken.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libroken.so.11 Reading symbols from /usr/lib/libwind.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libwind.so.11 Reading symbols from /usr/lib/libheimbase.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libheimbase.so.11 Reading symbols from /usr/lib/libprivateheimipcc.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libprivateheimipcc.so.11 Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/local/lib/dovecot/lib01_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/local/lib/dovecot/lib02_imap_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/local/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/local/lib/dovecot/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/local/lib/dovecot/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/local/lib/dovecot/lib90_antispam_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /usr/local/lib/compat/pkg/nss_winbind.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/compat/pkg/nss_winbind.so.1 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x000000001117655a in thr_kill () from /lib/libc.so.7 [New Thread 13616000 (LWP 100216/)] (gdb) bt #0 0x000000001117655a in thr_kill () from /lib/libc.so.7 #1 0x000000001117652b in raise () from /lib/libc.so.7 #2 0x0000000011176499 in abort () from /lib/libc.so.7 #3 0x0000000010e2af54 in default_error_handler () from /usr/local/lib/dovecot/libdovecot.so.0 #4 0x0000000010e2bc5b in i_set_failure_ignore_errors () from /usr/local/lib/dovecot/libdovecot.so.0 #5 0x0000000010e2b1f9 in i_fatal () from /usr/local/lib/dovecot/libdovecot.so.0 #6 0x0000000010aa190a in mail_namespace_find () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #7 0x000000001402e7e2 in trash_plugin_deinit () from /usr/local/lib/dovecot/lib11_trash_plugin.so #8 0x0000000010aaf20b in hook_mail_user_created () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #9 0x0000000010ab374f in mail_user_init () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #10 0x0000000010ab1c52 in mail_storage_service_next () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #11 0x0000000010ab2435 in mail_storage_service_lookup_next () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #12 0x0000000000425896 in client_create_from_input () #13 0x0000000000425fb2 in main () (gdb) bt full #0 0x000000001117655a in thr_kill () from /lib/libc.so.7 No symbol table info available. #1 0x000000001117652b in raise () from /lib/libc.so.7 No symbol table info available. #2 0x0000000011176499 in abort () from /lib/libc.so.7 No symbol table info available. #3 0x0000000010e2af54 in default_error_handler () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x0000000010e2bc5b in i_set_failure_ignore_errors () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x0000000010e2b1f9 in i_fatal () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x0000000010aa190a in mail_namespace_find () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #7 0x000000001402e7e2 in trash_plugin_deinit () from /usr/local/lib/dovecot/lib11_trash_plugin.so No symbol table info available. #8 0x0000000010aaf20b in hook_mail_user_created () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #9 0x0000000010ab374f in mail_user_init () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #10 0x0000000010ab1c52 in mail_storage_service_next () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0000000010ab2435 in mail_storage_service_lookup_next () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x0000000000425896 in client_create_from_input () No symbol table info available. #13 0x0000000000425fb2 in main () No symbol table info available. (gdb) quit Aki Tuomi ????? 2017-02-28 18:34: > Your backtrace would've been informative already if you had issued bt > full. Just opening it with gdb is not sufficient. -- With best regards, Max Kostikov BBM: 24CA5DF8 | W: https://kostikov.co From aki.tuomi at dovecot.fi Tue Feb 28 18:34:24 2017 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Tue, 28 Feb 2017 20:34:24 +0200 (EET) Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <8443a611e041ce749e6eb903a04e782c@kostikov.co> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> <722518410.1399.1488299658357@appsuite-dev.open-xchange.com> <8443a611e041ce749e6eb903a04e782c@kostikov.co> Message-ID: <1406481107.1657.1488306865726@appsuite-dev.open-xchange.com> > On February 28, 2017 at 7:43 PM Max Kostikov wrote: > > > Got it. > Here is full backtrace output. > Would appear the bug is in 'Trash' plugin. We'll open an issue about this, thank you for reporting this. Aki From max at kostikov.co Tue Feb 28 18:40:21 2017 From: max at kostikov.co (Max Kostikov) Date: Tue, 28 Feb 2017 20:40:21 +0200 Subject: dovecot-lda crash after upgrade to 2.2.28 In-Reply-To: <1406481107.1657.1488306865726@appsuite-dev.open-xchange.com> References: <285066cb-bed7-5927-f247-2fe1108140cd@dovecot.fi> <7a4b6acb95d5183179266446789602ab@kostikov.co> <1116891553.1309.1488296543757@appsuite-dev.open-xchange.com> <722518410.1399.1488299658357@appsuite-dev.open-xchange.com> <8443a611e041ce749e6eb903a04e782c@kostikov.co> <1406481107.1657.1488306865726@appsuite-dev.open-xchange.com> Message-ID: <902c89e2bd536b31ee1b8c653dd88ff8@kostikov.co> Thank you. Will be waiting for Dovecot update. Aki Tuomi ????? 2017-02-28 20:34: >> On February 28, 2017 at 7:43 PM Max Kostikov wrote: >> >> >> Got it. >> Here is full backtrace output. >> > > Would appear the bug is in 'Trash' plugin. We'll open an issue about > this, thank you for reporting this. > > Aki -- With best regards, Max Kostikov BBM: 24CA5DF8 | W: https://kostikov.co From spinner.dc at delphinidae.org.uk Tue Feb 28 22:42:21 2017 From: spinner.dc at delphinidae.org.uk (Andy R) Date: Tue, 28 Feb 2017 23:42:21 +0100 Subject: Replacement for antispam plugin In-Reply-To: References: Message-ID: <1754e048-40f0-a242-6769-332547e95e8c@delphinidae.org.uk> Greetings to the list :) I've been meaning to ask a couple of questions about the imapsieve for antispam. Firstly, I guess that the example at the bottom of the page "https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve" is meant to link to the new example at "https://wiki2.dovecot.org/HowTo/AntispamWithSieve" now? Also... in "imapsieve_mailboxX_name = ${directory}" is the string a literal, or is it working with the 'special use' flagged directories from the xx-mailboxes.conf? I'm asking as though I only have a few users to worry about, different mail clients each have their favourite special folders. IE thunderbird uses "junk" but one Outlook express version likes "Junk E-mail" and I have a different favourite from another version of OE too. So in mailboxes.conf I just added extra special use lines which seemed to make things happy. Otherwise I ended up with the server configured 'spam' folder showing as basic folder, and then each client adding it's own chosen spamfolder each time too. But, if the imapsieve is only matching to literal foldernames, should I just duplicate the trigger lines for each type of junk folder or is there a method to have the sieve script enumerate all the options listed by 'special use' or is there a better method for this? I want to put the spam-mail-filing script as a global sieve script as all users will need it, rather than duplicating out for each user. Many thanks , Andy R On 10/02/2017 09:06, Aki Tuomi wrote: > Hi! > Since antispam plugin is deprecated and we would really prefer people > not to use it, we wrote instructions on how to replace it with > IMAPSieve. Comments and suggestions are most welcome. > > https://wiki.dovecot.org/HowTo/AntispamWithSieve > > --- > Aki Tuomi > Dovecot oy > From spinner.dc at delphinidae.org.uk Tue Feb 28 22:58:55 2017 From: spinner.dc at delphinidae.org.uk (Andy R) Date: Tue, 28 Feb 2017 23:58:55 +0100 Subject: Query about list policy? Message-ID: <9c0ae949-1f28-76fe-09e3-ca890ff9a313@delphinidae.org.uk> Greetings for a second time today, As a quick query, what is the policy about linking to user-scripts and small addons ? I've used a couple of perl scripts to help me with dspam training from dovecot-antispam and imapsieve triggering for sorting and feeding messages to dspam in the background. They are most likely not the greatest but seem to work at least. I've posted them to github, but just wanted to ask whether it was an issue if i linked them here, or if they would need any review before posting or any other answer? Many thanks, and to Aki Tuomi and the dovecot team also. Andy R.