From gedalya at gedalya.net Sun May 1 01:02:49 2016 From: gedalya at gedalya.net (Gedalya) Date: Sat, 30 Apr 2016 21:02:49 -0400 Subject: Changing Password Schemes In-Reply-To: <5724FEDD.1070705@gmail.com> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> Message-ID: <572555B9.9030606@gedalya.net> First of all, you can probably go online before you convert all passwords. You can modify your query in dovecot-sql.conf.ext to something like the following: SELECT IF(crypt_pass IS NULL OR crypt_pass='', CONCAT('{PLAIN}',plain_pass), crypt_pass) as password FROM mailuser .. This is assuming that: * for incoming users, you have a plain_pass column containing just the plaintext password, without a {PLAIN} prefix, which we are adding in the query, letting dovecot process it correctly * for these users, your other password column, "crypt_pass" in this example, is either NULL or an empty string. * once crypt_pass is populated, it will contain a usable value, and this value will be returned by the query. Now, as for converting your database, try this, after adjusting the queries to fit your schema: #!/usr/bin/perl use strict; use warnings; use DBI; use MIME::Base64 'encode_base64'; my $dbtype = 'mysql'; my $dbhost = 'localhost'; my $dbname = 'maildb'; my $dbuser = 'dbuser'; my $dbpass = 'password'; my $dbh = DBI->connect("DBI:$dbtype:host=$dbhost;database=$dbname", $dbuser, $dbpass) or die "Could not connect to database: " . $DBI::errstr . "\n"; my $selectsth = $dbh->prepare('SELECT localpart, domain, plain_pass FROM mailuser where crypt_pass IS NULL OR crypt_pass=""'); my $updatesth = $dbh->prepare('UPDATE mailuser SET crypt_pass=? where localpart=? and domain=?'); $selectsth->execute; while (my $row = $selectsth->fetchrow_hashref) { open my $urand, '<', '/dev/urandom'; read $urand, my $salt, 12; close $urand; $salt = encode_base64($salt); $salt =~ s/\+/\./g; $salt =~ s/[^0-9a-z\.\/]//ig; #this shouldn't be needed my $cryptpw = '{SHA512-CRYPT}' . crypt $row->{plain_pass}, '$6$'.$salt; print "$row->{localpart}\@$row->{domain}: $cryptpw\n"; # uncomment this when you feel comfortable #$updatesth->execute($cryptpw, $row->{localpart}, $row->{domain}); } You can run this safely with the last line commended out, and review the output. Perhaps try to test by manually updating one user with the displayed output. If everything seems sane, uncomment the line and run again. On 04/30/2016 02:52 PM, Carl A Jeptha wrote: > Sorry not truncated: > {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI02QWAQNNfY5.Rk9zcSetYTgRfo4SPKf8qzMXsruvvS8uaSUidlvwDTLLSr3cVsQx2e6cu2/ > > ------------ > You have a good day now, en mag jou m?re ook so wees, > > Carl A Jeptha > > On 2016-04-30 14:58, Patrick Domack wrote: >> This looks good, except it is truncated, it should be something like 95chars long, Is your hash column set to 128 or up around there or larger? >> >> >> Quoting Carl A Jeptha : >> >>> Sorry for double reply, but this what a password looks like in the "hashed" password column: >>> {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI2 >>> >>> ------------ >>> You have a good day now, en mag jou m?re ook so wees, >>> >>> On 2016-04-30 01:14, Gedalya wrote: >>>> That's not SHA512-CRYPT. That's just a simple sha512 of the password, without salt. >>>> >>>> A SHA512-CRYPT password will be generated with: >>>> >>>> printf "1234\n1234" | doveadm pw -s SHA512-CRYPT >>>> >>>> or: >>>> >>>> doveadm pw -s SHA512-CRYPT -p 1234 >>>> >>>> or: >>>> >>>> mkpasswd -m sha-512 1234 >>>> >>>> (without the "{SHA512-CRYPT}" prefix) >>>> >>>> What exactly is the difficulty you are having with converting the passwords? >>>> What database engine are you using? >>>> >>>> >>>> On 04/29/2016 03:20 PM, Bill Shirley wrote: >>>>> Looks like an SQL update would do this: >>>>> UPDATE `users` >>>>> SET `passwd_SHA512` = SHA2(`passwd_clear`, 512); >>>>> >>>>> Bill >>>>> >>>>> On 4/29/2016 9:07 AM, Carl A Jeptha wrote: >>>>>> converting the passwords in the database from clear/plain text to SHA512-CRYPT From stephan at rename-it.nl Sun May 1 09:04:45 2016 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 1 May 2016 11:04:45 +0200 Subject: IX Out of Order? In-Reply-To: References: <93768d0a-6fe7-4971-6734-b0bf41468824@itronic.at> <571E18C7.1030500@rename-it.nl> Message-ID: <5725C6AD.5060606@rename-it.nl> Op 4/27/2016 om 4:46 PM schreef Markus Sch?nhaber: > Stephan Bosch, Mon, 25 Apr 2016 15:16:55 +0200: > >> Oops. Turned off package composer for maintenance, but didn't turn it >> back on. >> >> It is building now..... > Is there something to turn on to build packages in > /debian/pool/wheezy-auto/dovecot-2.2 > ? Or don't you provide packages for wheezy anymore? Looks like my recent merge with the official Debian Dovecot packages broke oldstable. The package is heavily restructured, making it very difficult to retain compatibility with oldstable. No new oldstable packages will be available for the time being, until I have time to look at this more. Regards, Stephan. From nospam.list at unclassified.de Sun May 1 09:10:40 2016 From: nospam.list at unclassified.de (Yves Goergen) Date: Sun, 1 May 2016 11:10:40 +0200 Subject: Long delays when deleting a message in inbox In-Reply-To: <5722F9AC.6030706@dovecot.fi> References: <5aa5ebb5-9adf-a5b2-3938-477f717c5cc7@unclassified.de> <5722F9AC.6030706@dovecot.fi> Message-ID: <293030d9-f723-f29b-1a1b-456d43dfe6bf@unclassified.de> Which log would you suggest? There's nothing for these times in mail.log, messages and syslog. Yves Goergen http://unclassified.software ________________________________________ Von: Aki Tuomi Gesendet: Fr, 2016-04-29 08:05 +0200 On 28.04.2016 20:54, Yves Goergen wrote: > Hi, > > Since recently, there's a long delay when deleting any message from my > inbox folder. This affects deleting and moving messages, but only the > inbox of a single mailbox. The delay is a few seconds. Other > operations work instantly. > > The server is Ubuntu 14.04 running Dovecot version "1:2.2.9-1ubuntu2". > The filesystem is local ext4. My MUA is Thunderbird on Windows, latest > version, connecting with IMAP. > > What could be the cause for this extreme delay? > > Yves Goergen > http://unclassified.software Have you looked at your logs? Aki From dovecot at list-post.mks-mail.de Sun May 1 11:08:40 2016 From: dovecot at list-post.mks-mail.de (=?UTF-8?Q?Markus_Sch=c3=b6nhaber?=) Date: Sun, 1 May 2016 13:08:40 +0200 Subject: IX Out of Order? In-Reply-To: <5725C6AD.5060606@rename-it.nl> References: <93768d0a-6fe7-4971-6734-b0bf41468824@itronic.at> <571E18C7.1030500@rename-it.nl> <5725C6AD.5060606@rename-it.nl> Message-ID: <3343fc7b-16c4-e92f-dab3-7b3c7ac6dc6f@list-post.mks-mail.de> Stephan Bosch, Sun, 1 May 2016 11:04:45 +0200: > No new oldstable packages will be available for the time being, until I > have time to look at this more. OK, thanks for the info (and for providing Debian packages in the first place!). -- Regards mks From jerry at seibercom.net Sun May 1 13:59:20 2016 From: jerry at seibercom.net (Jerry) Date: Sun, 1 May 2016 09:59:20 -0400 Subject: Sieve + Vacation Message-ID: <20160501095920.000027dc@seibercom.net> I am not sure if this is possible. When I use "vacation" in a sieve script, it always sends the response using "<>". I need it to send the response using a specific address. Is this possible? Thanks :) -- Jerry From cajeptha at gmail.com Sun May 1 15:27:14 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Sun, 1 May 2016 17:27:14 +0200 Subject: Changing Password Schemes In-Reply-To: <572555B9.9030606@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> Message-ID: Hi, Was testing your solution and was receiving: May 1 11:10:03 mail2 dovecot: message repeated 5 times: [ auth-worker(24202): Error: sql(user at domain.com,xxx.xxx.xxx.xxx): Password query returned multiple matches] Here is my dovecot-sql.conf.ext file: driver = mysql connect = host=127.0.0.1 dbname=vmail user=********* password=************* default_pass_scheme = SHA512-CRYPT password_query = SELECT IF(cryptpwd IS NULL OR cryptpwd='',CONCAT('{PLAIN}',clearpwd),cryptpwd)as password FROM mailbox user_query = SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' as mail, 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota FROM mailbox WHERE username = '%u' AND active = '1' ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On Sun, May 1, 2016 at 3:02 AM, Gedalya wrote: > First of all, you can probably go online before you convert all passwords. > You can modify your query in dovecot-sql.conf.ext to something like the > following: > > SELECT IF(crypt_pass IS NULL OR crypt_pass='', > CONCAT('{PLAIN}',plain_pass), crypt_pass) as password FROM mailuser .. > > This is assuming that: > > * for incoming users, you have a plain_pass column containing just the > plaintext password, without a {PLAIN} prefix, which we are adding in the > query, letting dovecot process it correctly > * for these users, your other password column, "crypt_pass" in this > example, is either NULL or an empty string. > * once crypt_pass is populated, it will contain a usable value, and this > value will be returned by the query. > > > Now, as for converting your database, try this, after adjusting the > queries to fit your schema: > > #!/usr/bin/perl > use strict; > use warnings; > use DBI; > use MIME::Base64 'encode_base64'; > > my $dbtype = 'mysql'; > my $dbhost = 'localhost'; > my $dbname = 'maildb'; > my $dbuser = 'dbuser'; > my $dbpass = 'password'; > > my $dbh = DBI->connect("DBI:$dbtype:host=$dbhost;database=$dbname", > $dbuser, $dbpass) > or die "Could not connect to database: " . $DBI::errstr . "\n"; > my $selectsth = $dbh->prepare('SELECT localpart, domain, plain_pass FROM > mailuser where crypt_pass IS NULL OR crypt_pass=""'); > my $updatesth = $dbh->prepare('UPDATE mailuser SET crypt_pass=? where > localpart=? and domain=?'); > $selectsth->execute; > while (my $row = $selectsth->fetchrow_hashref) { > open my $urand, '<', '/dev/urandom'; > read $urand, my $salt, 12; > close $urand; > $salt = encode_base64($salt); > $salt =~ s/\+/\./g; > $salt =~ s/[^0-9a-z\.\/]//ig; #this shouldn't be needed > my $cryptpw = '{SHA512-CRYPT}' . crypt $row->{plain_pass}, '$6$'.$salt; > print "$row->{localpart}\@$row->{domain}: $cryptpw\n"; > # uncomment this when you feel comfortable > #$updatesth->execute($cryptpw, $row->{localpart}, $row->{domain}); > } > > > You can run this safely with the last line commended out, and review the > output. Perhaps try to test by manually updating one user with the > displayed output. If everything seems sane, uncomment the line and run > again. > > > On 04/30/2016 02:52 PM, Carl A Jeptha wrote: > > Sorry not truncated: > > > {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI02QWAQNNfY5.Rk9zcSetYTgRfo4SPKf8qzMXsruvvS8uaSUidlvwDTLLSr3cVsQx2e6cu2/ > > > > ------------ > > You have a good day now, en mag jou m?re ook so wees, > > > > Carl A Jeptha > > > > On 2016-04-30 14:58, Patrick Domack wrote: > >> This looks good, except it is truncated, it should be something like > 95chars long, Is your hash column set to 128 or up around there or larger? > >> > >> > >> Quoting Carl A Jeptha : > >> > >>> Sorry for double reply, but this what a password looks like in the > "hashed" password column: > >>> {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI2 > >>> > >>> ------------ > >>> You have a good day now, en mag jou m?re ook so wees, > >>> > >>> On 2016-04-30 01:14, Gedalya wrote: > >>>> That's not SHA512-CRYPT. That's just a simple sha512 of the password, > without salt. > >>>> > >>>> A SHA512-CRYPT password will be generated with: > >>>> > >>>> printf "1234\n1234" | doveadm pw -s SHA512-CRYPT > >>>> > >>>> or: > >>>> > >>>> doveadm pw -s SHA512-CRYPT -p 1234 > >>>> > >>>> or: > >>>> > >>>> mkpasswd -m sha-512 1234 > >>>> > >>>> (without the "{SHA512-CRYPT}" prefix) > >>>> > >>>> What exactly is the difficulty you are having with converting the > passwords? > >>>> What database engine are you using? > >>>> > >>>> > >>>> On 04/29/2016 03:20 PM, Bill Shirley wrote: > >>>>> Looks like an SQL update would do this: > >>>>> UPDATE `users` > >>>>> SET `passwd_SHA512` = SHA2(`passwd_clear`, 512); > >>>>> > >>>>> Bill > >>>>> > >>>>> On 4/29/2016 9:07 AM, Carl A Jeptha wrote: > >>>>>> converting the passwords in the database from clear/plain text to > SHA512-CRYPT > From gedalya at gedalya.net Sun May 1 15:40:18 2016 From: gedalya at gedalya.net (Gedalya) Date: Sun, 1 May 2016 11:40:18 -0400 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> Message-ID: <57262362.8060700@gedalya.net> You do need to complete the query. Don't just replace your query with the one I wrote. You have to have a WHERE clause, and you might need to return other fields. Keep the password query you had before, just replace the 'password' column with "IF( ... ) as password" The query as you have it now simply returns all the passwords for all the users, because you don't have a WHERE clause. On 05/01/2016 11:27 AM, Carl Jeptha wrote: > Hi, > Was testing your solution and was receiving: > > May 1 11:10:03 mail2 dovecot: message repeated 5 times: [ > auth-worker(24202): Error: sql(user at domain.com,xxx.xxx.xxx.xxx): > Password query returned multiple matches] > > Here is my dovecot-sql.conf.ext file: > > driver = mysql > connect = host=127.0.0.1 dbname=vmail user=********* password=************* > default_pass_scheme = SHA512-CRYPT > password_query = SELECT IF(cryptpwd IS NULL OR > cryptpwd='',CONCAT('{PLAIN}',clearpwd),cryptpwd)as password FROM mailbox > user_query = SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' > as mail, 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota > FROM mailbox WHERE username = '%u' AND active = '1' > > ------------ > You have a good day now, en mag jou m?re ook so wees, > > Carl A Jeptha > > > On Sun, May 1, 2016 at 3:02 AM, Gedalya wrote: > >> First of all, you can probably go online before you convert all passwords. >> You can modify your query in dovecot-sql.conf.ext to something like the >> following: >> >> SELECT IF(crypt_pass IS NULL OR crypt_pass='', >> CONCAT('{PLAIN}',plain_pass), crypt_pass) as password FROM mailuser .. >> >> This is assuming that: >> >> * for incoming users, you have a plain_pass column containing just the >> plaintext password, without a {PLAIN} prefix, which we are adding in the >> query, letting dovecot process it correctly >> * for these users, your other password column, "crypt_pass" in this >> example, is either NULL or an empty string. >> * once crypt_pass is populated, it will contain a usable value, and this >> value will be returned by the query. >> >> >> Now, as for converting your database, try this, after adjusting the >> queries to fit your schema: >> >> #!/usr/bin/perl >> use strict; >> use warnings; >> use DBI; >> use MIME::Base64 'encode_base64'; >> >> my $dbtype = 'mysql'; >> my $dbhost = 'localhost'; >> my $dbname = 'maildb'; >> my $dbuser = 'dbuser'; >> my $dbpass = 'password'; >> >> my $dbh = DBI->connect("DBI:$dbtype:host=$dbhost;database=$dbname", >> $dbuser, $dbpass) >> or die "Could not connect to database: " . $DBI::errstr . "\n"; >> my $selectsth = $dbh->prepare('SELECT localpart, domain, plain_pass FROM >> mailuser where crypt_pass IS NULL OR crypt_pass=""'); >> my $updatesth = $dbh->prepare('UPDATE mailuser SET crypt_pass=? where >> localpart=? and domain=?'); >> $selectsth->execute; >> while (my $row = $selectsth->fetchrow_hashref) { >> open my $urand, '<', '/dev/urandom'; >> read $urand, my $salt, 12; >> close $urand; >> $salt = encode_base64($salt); >> $salt =~ s/\+/\./g; >> $salt =~ s/[^0-9a-z\.\/]//ig; #this shouldn't be needed >> my $cryptpw = '{SHA512-CRYPT}' . crypt $row->{plain_pass}, '$6$'.$salt; >> print "$row->{localpart}\@$row->{domain}: $cryptpw\n"; >> # uncomment this when you feel comfortable >> #$updatesth->execute($cryptpw, $row->{localpart}, $row->{domain}); >> } >> >> >> You can run this safely with the last line commended out, and review the >> output. Perhaps try to test by manually updating one user with the >> displayed output. If everything seems sane, uncomment the line and run >> again. >> >> >> On 04/30/2016 02:52 PM, Carl A Jeptha wrote: >>> Sorry not truncated: >>> >> {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI02QWAQNNfY5.Rk9zcSetYTgRfo4SPKf8qzMXsruvvS8uaSUidlvwDTLLSr3cVsQx2e6cu2/ >>> ------------ >>> You have a good day now, en mag jou m?re ook so wees, >>> >>> Carl A Jeptha >>> >>> On 2016-04-30 14:58, Patrick Domack wrote: >>>> This looks good, except it is truncated, it should be something like >> 95chars long, Is your hash column set to 128 or up around there or larger? >>>> >>>> Quoting Carl A Jeptha : >>>> >>>>> Sorry for double reply, but this what a password looks like in the >> "hashed" password column: >>>>> {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI2 >>>>> >>>>> ------------ >>>>> You have a good day now, en mag jou m?re ook so wees, >>>>> >>>>> On 2016-04-30 01:14, Gedalya wrote: >>>>>> That's not SHA512-CRYPT. That's just a simple sha512 of the password, >> without salt. >>>>>> A SHA512-CRYPT password will be generated with: >>>>>> >>>>>> printf "1234\n1234" | doveadm pw -s SHA512-CRYPT >>>>>> >>>>>> or: >>>>>> >>>>>> doveadm pw -s SHA512-CRYPT -p 1234 >>>>>> >>>>>> or: >>>>>> >>>>>> mkpasswd -m sha-512 1234 >>>>>> >>>>>> (without the "{SHA512-CRYPT}" prefix) >>>>>> >>>>>> What exactly is the difficulty you are having with converting the >> passwords? >>>>>> What database engine are you using? >>>>>> >>>>>> >>>>>> On 04/29/2016 03:20 PM, Bill Shirley wrote: >>>>>>> Looks like an SQL update would do this: >>>>>>> UPDATE `users` >>>>>>> SET `passwd_SHA512` = SHA2(`passwd_clear`, 512); >>>>>>> >>>>>>> Bill >>>>>>> >>>>>>> On 4/29/2016 9:07 AM, Carl A Jeptha wrote: >>>>>>>> converting the passwords in the database from clear/plain text to >> SHA512-CRYPT >> From stephan at rename-it.nl Sun May 1 21:07:42 2016 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 1 May 2016 23:07:42 +0200 Subject: Sieve + Vacation In-Reply-To: <20160501095920.000027dc@seibercom.net> References: <20160501095920.000027dc@seibercom.net> Message-ID: <5726701E.6090006@rename-it.nl> Op 5/1/2016 om 3:59 PM schreef Jerry: > I am not sure if this is possible. When I use "vacation" in a sieve > script, it always sends the response using "<>". I need it to send the > response using a specific address. Is this possible? > > Thanks :) This is currently the only option: http://wiki2.dovecot.org/Pigeonhole/Sieve/Extensions/Vacation#Configuration sieve_vacation_send_from_recipient=yes I can add some more flexibility to the next version. This will be a setting similar to "sieve_redirect_envelope_from". Regards, Stephan. From hakon at alstadheim.priv.no Sun May 1 22:38:09 2016 From: hakon at alstadheim.priv.no (=?UTF-8?Q?H=c3=a5kon_Alstadheim?=) Date: Mon, 2 May 2016 00:38:09 +0200 Subject: deb repo with dovecot-antispam ? Message-ID: <57268551.4000705@alstadheim.priv.no> There are .debs available for version 2.2 both from backports and ix.dovecot.fi, but neither seem to have the antispam plugin built for version 2.2. Anybody know why, and (if there is no particular reason) how to get dovecot-antispam in a proper .deb package for dovecot-2.2.* ? From stephan at rename-it.nl Mon May 2 00:39:13 2016 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 2 May 2016 02:39:13 +0200 Subject: deb repo with dovecot-antispam ? In-Reply-To: <57268551.4000705@alstadheim.priv.no> References: <57268551.4000705@alstadheim.priv.no> Message-ID: <5726A1B1.8020409@rename-it.nl> Op 5/2/2016 om 12:38 AM schreef H?kon Alstadheim: > There are .debs available for version 2.2 both from backports and > ix.dovecot.fi, but neither seem to have the antispam plugin built for > version 2.2. Anybody know why, and (if there is no particular reason) > how to get dovecot-antispam in a proper .deb package for dovecot-2.2.* ? Including it in the current Xi build process isn't trivial. I haven't needed it so far, so it never got included. I played with this a bit. The attached crude shell script will create a ./build directory in which a fresh debian package is built. It needs at least git and devscripts packages installed. Adjust the script as needed. Clear the build directory manually if you need to start over. You can also use this shell script as a step-by-step guide on how to make a fresh Debian package for it: - It clones git repositories for dovecot-antispam and its debian package. - From dovecot-antispam it creates the required orig.tar.gz file and unpacks it. - The debian directory is copied from the official Debian package. This will only work if the latest dovecot-antispam is still compatible. Currently, it is. - Its changelog is updated for the new version. - The package is built. At my end it builds successfully, but I haven't actually tried the resulting package. Regards, Stephan. -------------- next part -------------- #!/bin/sh # Used in Debian changelog export DEBEMAIL="stephan at rename-it.nl" export DEBFULLNAME="Stephan Bosch" # Date-based versioning scheme; identical to official packages BASE_VERSION="2.0" DATE=`date +%Y%m%d` VERSION="$BASE_VERSION+$DATE" REVISION="1" # Increment if multiple versions on one day mkdir build cd build git clone git://git.sipsolutions.net/dovecot-antispam.git git clone git://anonscm.debian.org/users/ron/dovecot-antispam.git dovecot-antispam-debian cd dovecot-antispam git archive \ --format=tar \ --prefix="dovecot-antispam-$VERSION/" \ master > "../dovecot-antispam_$VERSION.orig.tar" cd .. gzip "dovecot-antispam_$VERSION.orig.tar" tar -xzf "dovecot-antispam_$VERSION.orig.tar.gz" cp -ar dovecot-antispam-debian/debian "dovecot-antispam-$VERSION" # Requires devscripts cd "dovecot-antispam-$VERSION" dch -v "$VERSION-$REVISION" "Updated to the latest dovecot-antispam Git revision" # Requires devscripts debuild -us -uc From leon at f-m.fm Mon May 2 00:47:40 2016 From: leon at f-m.fm (Leon Kyneur) Date: Mon, 2 May 2016 08:47:40 +0800 Subject: autoexpunge wildcard mailbox from userdb Message-ID: <5726A3AC.6050508@f-m.fm> Was excited to see that wildcards are now possible on autoexpunge in the 2.2.24 release. However I'm trying to configure these from the userdb lookup and it doesn't appear to be working. if I set in 15-mailboxes.conf mailbox Trash/* { ... autoexpunge = 10m } everything works.. however if I return =namespace/inbox/mailbox/Trash/*/autoexpunge=10m via userdb - it does not. Logs say: Debug: Unknown userdb setting: plugin/namespace/inbox/mailbox/Trash/*/autoexpunge=10m Is what I am trying to do even possible? Thanks Leon From dogz.tw at gmail.com Mon May 2 01:14:54 2016 From: dogz.tw at gmail.com (Dogz) Date: Mon, 2 May 2016 09:14:54 +0800 Subject: Dovecot (LDAP) get the quota limitation from windows Active directory In-Reply-To: References: Message-ID: Hi, Finally I found the root cause due to some specific fields are not in windows GC. I was use port:3268 to access windows active directory. For example, while I choose maxStorage to limit users' quota, it can't find the value from GC, but I changed to port:389, it works. Sorry to bother all of you. -- Best Regards, Dogz From skdovecot at smail.inf.fh-brs.de Mon May 2 06:39:04 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 May 2016 08:39:04 +0200 (CEST) Subject: Long delays when deleting a message in inbox In-Reply-To: <293030d9-f723-f29b-1a1b-456d43dfe6bf@unclassified.de> References: <5aa5ebb5-9adf-a5b2-3938-477f717c5cc7@unclassified.de> <5722F9AC.6030706@dovecot.fi> <293030d9-f723-f29b-1a1b-456d43dfe6bf@unclassified.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 1 May 2016, Yves Goergen wrote: > Which log would you suggest? There's nothing for these times in mail.log, > messages and syslog. Dovecot logs. > > Yves Goergen > http://unclassified.software > > ________________________________________ > Von: Aki Tuomi > Gesendet: Fr, 2016-04-29 08:05 +0200 > > > On 28.04.2016 20:54, Yves Goergen wrote: >> Hi, >> >> Since recently, there's a long delay when deleting any message from my >> inbox folder. This affects deleting and moving messages, but only the >> inbox of a single mailbox. The delay is a few seconds. Other >> operations work instantly. >> >> The server is Ubuntu 14.04 running Dovecot version "1:2.2.9-1ubuntu2". >> The filesystem is local ext4. My MUA is Thunderbird on Windows, latest >> version, connecting with IMAP. >> >> What could be the cause for this extreme delay? >> >> Yves Goergen >> http://unclassified.software > > Have you looked at your logs? > > Aki > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVyb2CHz1H7kL/d9rAQJqAgf/f6g4VHhqxm14yGMobQyahQegdSqTTHso l2hfAZoxKfUmF6UliL1c1WZ7uWWOZas6ne9IFk6TJEV80iWCb0JNgE1f5/nUm99J 2PCwcPLp9iN/b1W/hZZ8RvUiAsaCD5Je6ghSZX6PQCUGUPhr0CHrRN/nSOhWA/nE sJ398k+CCXty09bmB+BkUXAerYocbWCweBK1rVVYZeBvGHdQ2C1GgapysE/15hGL MaxIH5bdegsNc5ooiGnF24c/Ro88ZuhklRYnYj+1b19hJKN8pOh+RHs69ls5P9R0 oGiE2Hya0NmyGh0kRmWEP4Uu8pvqCA7JEc2xtOxoQgTtY2UgKkjoWA== =OoL4 -----END PGP SIGNATURE----- From hakon at alstadheim.priv.no Mon May 2 06:50:52 2016 From: hakon at alstadheim.priv.no (=?UTF-8?Q?H=c3=a5kon_Alstadheim?=) Date: Mon, 2 May 2016 08:50:52 +0200 Subject: deb repo with dovecot-antispam ? In-Reply-To: <5726A1B1.8020409@rename-it.nl> References: <57268551.4000705@alstadheim.priv.no> <5726A1B1.8020409@rename-it.nl> Message-ID: <5726F8CC.1030907@alstadheim.priv.no> Den 02. mai 2016 02:39, skrev Stephan Bosch: > Op 5/2/2016 om 12:38 AM schreef H?kon Alstadheim: >> There are .debs available for version 2.2 both from backports and >> ix.dovecot.fi, but neither seem to have the antispam plugin built for >> version 2.2. Anybody know why, and (if there is no particular reason) >> how to get dovecot-antispam in a proper .deb package for dovecot-2.2.* ? > Including it in the current Xi build process isn't trivial. I haven't > needed it so far, so it never got included. > > I played with this a bit. The attached crude shell script will create a > ./build directory in which a fresh debian package is built. It needs at > least git and devscripts packages installed. Adjust the script as needed. > > Clear the build directory manually if you need to start over. > > You can also use this shell script as a step-by-step guide on how to > make a fresh Debian package for it: > > - It clones git repositories for dovecot-antispam and its debian package. > - From dovecot-antispam it creates the required orig.tar.gz file and > unpacks it. > - The debian directory is copied from the official Debian package. This > will only work if the latest dovecot-antispam is still compatible. > Currently, it is. > - Its changelog is updated for the new version. > - The package is built. > > At my end it builds successfully, but I haven't actually tried the > resulting package. > > Regards, > > Stephan. > > Thanks! Don't have time to try it now, but I have a script which will unhold dovecot, update and then build dovecot-antispam iff there was a new version of dovecot, and then hold dovectot. I'll try plugging your script in there, and see how It goes. From mail at tomsommer.dk Mon May 2 06:56:08 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Mon, 02 May 2016 08:56:08 +0200 Subject: more info about deleting users In-Reply-To: <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> Message-ID: <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> On 2016-04-29 16:25, Matteo Gaito Test Account wrote: > i've search a lot in order to find a way to delete account by dovecot > but i > found only this way > > http://dovecot.org/list/dovecot/2011-November/079273.html > > It's possible today delete a maildir remotely with doveadm or with > dovecot api? Would love this as well // Tom From skdovecot at smail.inf.fh-brs.de Mon May 2 07:19:28 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 2 May 2016 09:19:28 +0200 (CEST) Subject: more info about deleting users In-Reply-To: <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 2 May 2016, Tom Sommer wrote: > On 2016-04-29 16:25, Matteo Gaito Test Account wrote: > >> i've search a lot in order to find a way to delete account by dovecot but i >> found only this way >> >> http://dovecot.org/list/dovecot/2011-November/079273.html >> >> It's possible today delete a maildir remotely with doveadm or with dovecot >> api? > > Would love this as well In fact, I would like to have: + full user backup + full user removal + restore of backup'ed user incl. all data, Dovecot creates / maintains by itself, such as Quota, ACLs (of deleted user to other mailboxes), messages and all settings of mailboxes, indexes etc.pp. however, part of the user data comes from my LDAP, there are some symlink tricks for some accounts, and some user data is faked depending on the protocol used. Because Dovecot is such flexible, I guess, there are some pitfalls, Timo & company cannot make a reliable code for. Timo pointed out, that there is no internal "vision" of backup / removal in the current API and therefore one has to extend the API and check every backend driver / plugin, if it stores user information and how to backup / delete / restore it. To make this reliable as Time wants Dovecot to be it will take some resources. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVyb/gHz1H7kL/d9rAQLSCQgAs2P+QMUf082HHfjUaxGyd9B7/3FSeixh 63Pucgr+x8tJKH+BRkN4sEonXbHuQ7pkfmmA+2KughhAgCWhxlX0ZqsuXFvFSxS4 z9bwckx/Y1/FHZTKGLwlVQ/8b1HR3AkCeWv9hvw1bxBIa2JhPhe0SkVKwnF0yT/m mxmOKws7TxXT+jRedA3WXUvKbXM5ntgZi5s+y+Wb466uKNzdTw2GZZyHbg5ppgL2 eAr6pS0ZT5cgT83dN3/a23cpldX+0gKvyXWI/cv+baMUiKJ3xoTXfNsSvHGBIWsV mCUXuVZhGzBjPRqOvtExcXDuR9qFE6EMMMmPngYtghpz8y+juWBWAw== =dbJ8 -----END PGP SIGNATURE----- From tss at iki.fi Mon May 2 08:51:57 2016 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 May 2016 11:51:57 +0300 Subject: autoexpunge wildcard mailbox from userdb In-Reply-To: <5726A3AC.6050508@f-m.fm> References: <5726A3AC.6050508@f-m.fm> Message-ID: <6D916BF5-2E84-45E4-810E-BBEBE4E92A92@iki.fi> On 02 May 2016, at 03:47, Leon Kyneur wrote: > > Was excited to see that wildcards are now possible on autoexpunge in the 2.2.24 release. > > However I'm trying to configure these from the userdb lookup and it doesn't appear to be working. > > if I set in 15-mailboxes.conf > mailbox Trash/* { > ... > autoexpunge = 10m > } > > everything works.. > > however if I return > > =namespace/inbox/mailbox/Trash/*/autoexpunge=10m Using "/" in the namespace name conflicts with the "/" separator, so it needs to be escaped. You can use "doveconf -n -S" to find out what the escaping needs to look like ("/" = "\s"). From cajeptha at gmail.com Mon May 2 09:32:18 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Mon, 2 May 2016 11:32:18 +0200 Subject: Changing Password Schemes In-Reply-To: <57262362.8060700@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> Message-ID: driver = mysql connect = host=127.0.0.1 dbname=********* user=*********** password=****************** default_pass_scheme = SHA512-CRYPT password_query = \ SELECT username AS USER, \ IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), cryptpwd) AS PASSWORD, \ '/var/vmail/%d/%n' as userdb_home, \ 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid \ FROM mailbox \ WHERE username = '%u' AND active = '1' user_query = \ SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' as mail, \ 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota \ FROM mailbox WHERE username = '%u' AND active = '1' Above is what I have done, but still getting an error: May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a field named 'password' For testing purposes I put the query in PHPMyAdmin and it complains this (notice it drops "PASSWORD", but shows it in the query: #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '\ IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), cryptpwd) as ' at line 1 ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On Sun, May 1, 2016 at 5:40 PM, Gedalya wrote: > You do need to complete the query. Don't just replace your query with the > one I wrote. You have to have a WHERE clause, and you might need to return > other fields. > Keep the password query you had before, just replace the 'password' column > with "IF( ... ) as password" > The query as you have it now simply returns all the passwords for all the > users, because you don't have a WHERE clause. > > On 05/01/2016 11:27 AM, Carl Jeptha wrote: > > Hi, > > Was testing your solution and was receiving: > > > > May 1 11:10:03 mail2 dovecot: message repeated 5 times: [ > > auth-worker(24202): Error: sql(user at domain.com,xxx.xxx.xxx.xxx): > > Password query returned multiple matches] > > > > Here is my dovecot-sql.conf.ext file: > > > > driver = mysql > > connect = host=127.0.0.1 dbname=vmail user=********* > password=************* > > default_pass_scheme = SHA512-CRYPT > > password_query = SELECT IF(cryptpwd IS NULL OR > > cryptpwd='',CONCAT('{PLAIN}',clearpwd),cryptpwd)as password FROM mailbox > > user_query = SELECT '/var/vmail/%d/%n' as home, > 'maildir:/var/vmail/%d/%n' > > as mail, 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota > > FROM mailbox WHERE username = '%u' AND active = '1' > > > > ------------ > > You have a good day now, en mag jou m?re ook so wees, > > > > Carl A Jeptha > > > > > > On Sun, May 1, 2016 at 3:02 AM, Gedalya wrote: > > > >> First of all, you can probably go online before you convert all > passwords. > >> You can modify your query in dovecot-sql.conf.ext to something like the > >> following: > >> > >> SELECT IF(crypt_pass IS NULL OR crypt_pass='', > >> CONCAT('{PLAIN}',plain_pass), crypt_pass) as password FROM mailuser .. > >> > >> This is assuming that: > >> > >> * for incoming users, you have a plain_pass column containing just the > >> plaintext password, without a {PLAIN} prefix, which we are adding in the > >> query, letting dovecot process it correctly > >> * for these users, your other password column, "crypt_pass" in this > >> example, is either NULL or an empty string. > >> * once crypt_pass is populated, it will contain a usable value, and this > >> value will be returned by the query. > >> > >> > >> Now, as for converting your database, try this, after adjusting the > >> queries to fit your schema: > >> > >> #!/usr/bin/perl > >> use strict; > >> use warnings; > >> use DBI; > >> use MIME::Base64 'encode_base64'; > >> > >> my $dbtype = 'mysql'; > >> my $dbhost = 'localhost'; > >> my $dbname = 'maildb'; > >> my $dbuser = 'dbuser'; > >> my $dbpass = 'password'; > >> > >> my $dbh = DBI->connect("DBI:$dbtype:host=$dbhost;database=$dbname", > >> $dbuser, $dbpass) > >> or die "Could not connect to database: " . $DBI::errstr . "\n"; > >> my $selectsth = $dbh->prepare('SELECT localpart, domain, plain_pass FROM > >> mailuser where crypt_pass IS NULL OR crypt_pass=""'); > >> my $updatesth = $dbh->prepare('UPDATE mailuser SET crypt_pass=? where > >> localpart=? and domain=?'); > >> $selectsth->execute; > >> while (my $row = $selectsth->fetchrow_hashref) { > >> open my $urand, '<', '/dev/urandom'; > >> read $urand, my $salt, 12; > >> close $urand; > >> $salt = encode_base64($salt); > >> $salt =~ s/\+/\./g; > >> $salt =~ s/[^0-9a-z\.\/]//ig; #this shouldn't be needed > >> my $cryptpw = '{SHA512-CRYPT}' . crypt $row->{plain_pass}, > '$6$'.$salt; > >> print "$row->{localpart}\@$row->{domain}: $cryptpw\n"; > >> # uncomment this when you feel comfortable > >> #$updatesth->execute($cryptpw, $row->{localpart}, $row->{domain}); > >> } > >> > >> > >> You can run this safely with the last line commended out, and review the > >> output. Perhaps try to test by manually updating one user with the > >> displayed output. If everything seems sane, uncomment the line and run > >> again. > >> > >> > >> On 04/30/2016 02:52 PM, Carl A Jeptha wrote: > >>> Sorry not truncated: > >>> > >> > {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI02QWAQNNfY5.Rk9zcSetYTgRfo4SPKf8qzMXsruvvS8uaSUidlvwDTLLSr3cVsQx2e6cu2/ > >>> ------------ > >>> You have a good day now, en mag jou m?re ook so wees, > >>> > >>> Carl A Jeptha > >>> > >>> On 2016-04-30 14:58, Patrick Domack wrote: > >>>> This looks good, except it is truncated, it should be something like > >> 95chars long, Is your hash column set to 128 or up around there or > larger? > >>>> > >>>> Quoting Carl A Jeptha : > >>>> > >>>>> Sorry for double reply, but this what a password looks like in the > >> "hashed" password column: > >>>>> {SHA512-CRYPT}$6$wEn1UFuiMzl9OSjd$Vh/PZ95WDID1GwI2 > >>>>> > >>>>> ------------ > >>>>> You have a good day now, en mag jou m?re ook so wees, > >>>>> > >>>>> On 2016-04-30 01:14, Gedalya wrote: > >>>>>> That's not SHA512-CRYPT. That's just a simple sha512 of the > password, > >> without salt. > >>>>>> A SHA512-CRYPT password will be generated with: > >>>>>> > >>>>>> printf "1234\n1234" | doveadm pw -s SHA512-CRYPT > >>>>>> > >>>>>> or: > >>>>>> > >>>>>> doveadm pw -s SHA512-CRYPT -p 1234 > >>>>>> > >>>>>> or: > >>>>>> > >>>>>> mkpasswd -m sha-512 1234 > >>>>>> > >>>>>> (without the "{SHA512-CRYPT}" prefix) > >>>>>> > >>>>>> What exactly is the difficulty you are having with converting the > >> passwords? > >>>>>> What database engine are you using? > >>>>>> > >>>>>> > >>>>>> On 04/29/2016 03:20 PM, Bill Shirley wrote: > >>>>>>> Looks like an SQL update would do this: > >>>>>>> UPDATE `users` > >>>>>>> SET `passwd_SHA512` = SHA2(`passwd_clear`, 512); > >>>>>>> > >>>>>>> Bill > >>>>>>> > >>>>>>> On 4/29/2016 9:07 AM, Carl A Jeptha wrote: > >>>>>>>> converting the passwords in the database from clear/plain text to > >> SHA512-CRYPT > >> > From ml+dovecot at valo.at Mon May 2 09:58:57 2016 From: ml+dovecot at valo.at (Christian Kivalo) Date: Mon, 02 May 2016 11:58:57 +0200 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> Message-ID: <6adc6bc0665aa01a7f79398c91903ab4@valo.at> On 2016-05-02 11:32, Carl Jeptha wrote: > password_query = \ > SELECT username AS USER, \ > IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), > cryptpwd) AS PASSWORD, \ > '/var/vmail/%d/%n' as userdb_home, \ > 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 > as > userdb_gid \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' You have a right parenthesis after clearpwd in your sql statement CONCAT('{PLAIN}',clearpwd),cryptpwd) > user_query = \ > SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' as > mail, \ > 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota \ > FROM mailbox WHERE username = '%u' AND active = '1' > > Above is what I have done, but still getting an error: > > May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: > sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a > field named 'password' > > For testing purposes I put the query in PHPMyAdmin and it complains > this > (notice it drops "PASSWORD", but shows it in the query: > #1064 - You have an error in your SQL syntax; check the manual that > corresponds to your MySQL server version for the right syntax to use > near '\ > IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), > cryptpwd) as ' at line 1 The PHPMyAdmin error message shows the relevant part to inspect. -- Christian Kivalo From cajeptha at gmail.com Mon May 2 10:07:04 2016 From: cajeptha at gmail.com (cajeptha) Date: Mon, 02 May 2016 12:07:04 +0200 Subject: Changing Password Schemes Message-ID: That one is there to close the left one after IF Sent from my Samsung device -------- Original message -------- From: Christian Kivalo Date: 2016-05-02 11:58 (GMT+02:00) To: dovecot at dovecot.org Subject: Re: Changing Password Schemes On 2016-05-02 11:32, Carl Jeptha wrote: > password_query = \ >?????? SELECT username AS USER, \ >???? IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), > cryptpwd) AS PASSWORD, \ >???? '/var/vmail/%d/%n' as userdb_home, \ >?????? 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 > as > userdb_gid \ >?????? FROM mailbox \ >?????? WHERE username = '%u' AND active = '1' You have a right parenthesis after clearpwd in your sql statement ?? CONCAT('{PLAIN}',clearpwd),cryptpwd) >?? user_query = \ >?? SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' as > mail, \ >?? 150 AS uid, 8 AS gid, concat('dirsize:storage=', quota) AS quota \ >?? FROM mailbox WHERE username = '%u' AND active = '1' > > Above is what I have done, but still getting an error: > > May? 2 05:26:03 |****** dovecot: auth-worker(3442): Error: > sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a > field named 'password' > > For testing purposes I put the query in PHPMyAdmin and it complains > this > (notice it drops "PASSWORD", but shows it in the query: > #1064 - You have an error in your SQL syntax; check the manual that > corresponds to your MySQL server version for the right syntax to use > near '\ >???? IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), > cryptpwd) as ' at line 1 The PHPMyAdmin error message shows the relevant part to inspect. -- Christian Kivalo From gedalya at gedalya.net Mon May 2 12:07:10 2016 From: gedalya at gedalya.net (Gedalya) Date: Mon, 2 May 2016 08:07:10 -0400 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> Message-ID: <572742EE.7080001@gedalya.net> On 05/02/2016 05:32 AM, Carl Jeptha wrote: > May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: > sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a > field named 'password' I'm not sure, maybe it's checking case-sensitive. Your query returns PASSWORD. Make it lowercase. > > For testing purposes I put the query in PHPMyAdmin and it complains this > (notice it drops "PASSWORD", but shows it in the query: > #1064 - You have an error in your SQL syntax; check the manual that > corresponds to your MySQL server version for the right syntax to use near '\ > IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), > cryptpwd) as ' at line 1 > > It also sarts with a \ ... did you leave that in? That is specific to the dovecot config file. In PHPMyAdmin you should remove the line-continuation backslashes. Actually if you use the mysql command-line client, you would be able to paste that in with the backlashes. Make sure to put in a real value in WHERE username = '%u' <<< From leon at f-m.fm Mon May 2 13:40:49 2016 From: leon at f-m.fm (Leon Kyneur) Date: Mon, 2 May 2016 21:40:49 +0800 Subject: autoexpunge wildcard mailbox from userdb In-Reply-To: <6D916BF5-2E84-45E4-810E-BBEBE4E92A92@iki.fi> References: <5726A3AC.6050508@f-m.fm> <6D916BF5-2E84-45E4-810E-BBEBE4E92A92@iki.fi> Message-ID: <572758E1.1080108@f-m.fm> On 02/05/16 16:51, Timo Sirainen wrote: > On 02 May 2016, at 03:47, Leon Kyneur wrote: >> Was excited to see that wildcards are now possible on autoexpunge in the 2.2.24 release. >> >> However I'm trying to configure these from the userdb lookup and it doesn't appear to be working. >> >> if I set in 15-mailboxes.conf >> mailbox Trash/* { >> ... >> autoexpunge = 10m >> } >> >> everything works.. >> >> however if I return >> >> =namespace/inbox/mailbox/Trash/*/autoexpunge=10m > Using "/" in the namespace name conflicts with the "/" separator, so it needs to be escaped. You can use "doveconf -n -S" to find out what the escaping needs to look like ("/" = "\s"). > Thanks Timo - that's what I was missing. Another thing - will it ever be possible to wildcard *all* mailboxes? My setup doesn't specify prefix = INBOX/ on "namespace inbox"... without knowing all of the root mailboxes users have created is it possible to autoexpunge these? This obviously does not work =namespace/inbox/mailbox/\s*/autoexpunge=90d From tss at iki.fi Mon May 2 13:43:48 2016 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 May 2016 16:43:48 +0300 Subject: autoexpunge wildcard mailbox from userdb In-Reply-To: <572758E1.1080108@f-m.fm> References: <5726A3AC.6050508@f-m.fm> <6D916BF5-2E84-45E4-810E-BBEBE4E92A92@iki.fi> <572758E1.1080108@f-m.fm> Message-ID: On 02 May 2016, at 16:40, Leon Kyneur wrote: > > > > On 02/05/16 16:51, Timo Sirainen wrote: >> On 02 May 2016, at 03:47, Leon Kyneur wrote: >>> Was excited to see that wildcards are now possible on autoexpunge in the 2.2.24 release. >>> >>> However I'm trying to configure these from the userdb lookup and it doesn't appear to be working. >>> >>> if I set in 15-mailboxes.conf >>> mailbox Trash/* { >>> ... >>> autoexpunge = 10m >>> } >>> >>> everything works.. >>> >>> however if I return >>> >>> =namespace/inbox/mailbox/Trash/*/autoexpunge=10m >> Using "/" in the namespace name conflicts with the "/" separator, so it needs to be escaped. You can use "doveconf -n -S" to find out what the escaping needs to look like ("/" = "\s"). >> > > Thanks Timo - that's what I was missing. > > Another thing - will it ever be possible to wildcard *all* mailboxes? > > My setup doesn't specify prefix = INBOX/ on "namespace inbox"... without knowing all of the root mailboxes users have created is it possible to autoexpunge these? > > This obviously does not work > =namespace/inbox/mailbox/\s*/autoexpunge=90d Why? I would expect mailbox * { autoexpunge } to work. From leon at f-m.fm Mon May 2 13:51:49 2016 From: leon at f-m.fm (Leon Kyneur) Date: Mon, 2 May 2016 21:51:49 +0800 Subject: autoexpunge wildcard mailbox from userdb In-Reply-To: References: <5726A3AC.6050508@f-m.fm> <6D916BF5-2E84-45E4-810E-BBEBE4E92A92@iki.fi> <572758E1.1080108@f-m.fm> Message-ID: <57275B75.7000404@f-m.fm> On 02/05/16 21:43, Timo Sirainen wrote: > On 02 May 2016, at 16:40, Leon Kyneur wrote: >> >> >> On 02/05/16 16:51, Timo Sirainen wrote: >>> On 02 May 2016, at 03:47, Leon Kyneur wrote: >>>> Was excited to see that wildcards are now possible on autoexpunge in the 2.2.24 release. >>>> >>>> However I'm trying to configure these from the userdb lookup and it doesn't appear to be working. >>>> >>>> if I set in 15-mailboxes.conf >>>> mailbox Trash/* { >>>> ... >>>> autoexpunge = 10m >>>> } >>>> >>>> everything works.. >>>> >>>> however if I return >>>> >>>> =namespace/inbox/mailbox/Trash/*/autoexpunge=10m >>> Using "/" in the namespace name conflicts with the "/" separator, so it needs to be escaped. You can use "doveconf -n -S" to find out what the escaping needs to look like ("/" = "\s"). >>> >> Thanks Timo - that's what I was missing. >> >> Another thing - will it ever be possible to wildcard *all* mailboxes? >> >> My setup doesn't specify prefix = INBOX/ on "namespace inbox"... without knowing all of the root mailboxes users have created is it possible to autoexpunge these? >> >> This obviously does not work >> =namespace/inbox/mailbox/\s*/autoexpunge=90d > Why? I would expect mailbox * { autoexpunge } to work. > Ahh it does! my syntax was just off.. Thanks again Timo! From me at junc.eu Mon May 2 16:15:11 2016 From: me at junc.eu (Benny Pedersen) Date: Mon, 02 May 2016 18:15:11 +0200 Subject: Sieve + Vacation In-Reply-To: <20160501095920.000027dc@seibercom.net> References: <20160501095920.000027dc@seibercom.net> Message-ID: <20160502161603.5B711239C3@talvi.dovecot.org> On May 1, 2016 4:00:15 PM Jerry wrote: > I am not sure if this is possible. When I use "vacation" in a sieve > script, it always sends the response using "<>". I need it to send the > response using a specific address. Is this possible? > > Thanks :) Why not just talk with Eliza abort it ?, maybe she know Siri usefullness aswell?, sorry could not resist here The resonans for not using another email is to avoid loop auto replies From michael at orlitzky.com Mon May 2 16:44:13 2016 From: michael at orlitzky.com (Michael Orlitzky) Date: Mon, 2 May 2016 12:44:13 -0400 Subject: more info about deleting users In-Reply-To: References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> Message-ID: <572783DD.9060209@orlitzky.com> On 05/02/2016 03:19 AM, Steffen Kaiser wrote: > On Mon, 2 May 2016, Tom Sommer wrote: >> On 2016-04-29 16:25, Matteo Gaito Test Account wrote: > > > In fact, I would like to have: > > + full user backup > + full user removal > + restore of backup'ed user > > incl. all data, Dovecot creates / maintains by itself, such as Quota, > ACLs (of deleted user to other mailboxes), messages and all settings of > mailboxes, indexes etc.pp. > You also need to delete any aliases that are associated with that user in your MTA. And any contacts associated with that address in your webmail/CardDAV system. And any calendars associated with it in your CalDAV system. And so on. Moreover, unless you plan to give root CLI access to your support people, there needs to be a way to figure out who was removed from SQL and then perform the necessary cleanup (as root) later on. I solved this for my own exact use case with mailshears: http://michael.orlitzky.com/code/mailshears.php If you use Dovecot with the PostfixAdmin schema in PostgreSQL, and your filesystem layout is /, then it will work for you out-of-the-box. It also supports Roundcube, AgenDAV, and DAViCal. Any user accounts for Dovecot, Postfix, AgenDAV, DAViCal, or Roundcube can be, * Pruned (cleaned up after someone removes them from PostfixAdmin). * Removed directly. * And even renamed! Yes, this works: # mailshears mv michael at orlitzky.com mjo at orlitzky.com There are examples of everything in the man page. Like I said, right now it's tailored exactly for my mail system. But, everything is well-designed and implemented by plugins. If you want to support LDAP or anything else, it can probably be done with a plugin. I would also happily accept patches for MySQL support, or to make the filesystem layout configurable, or anything like that. From jerry at seibercom.net Mon May 2 17:35:45 2016 From: jerry at seibercom.net (Jerry) Date: Mon, 2 May 2016 13:35:45 -0400 Subject: Sieve + Vacation In-Reply-To: <20160502161603.5B711239C3@talvi.dovecot.org> References: <20160501095920.000027dc@seibercom.net> <20160502161603.5B711239C3@talvi.dovecot.org> Message-ID: <20160502133545.0000172f@seibercom.net> On Mon, 02 May 2016 18:15:11 +0200, Benny Pedersen stated: >On May 1, 2016 4:00:15 PM Jerry wrote: > >> I am not sure if this is possible. When I use "vacation" in a sieve >> script, it always sends the response using "<>". I need it to send >> the response using a specific address. Is this possible? >> >> Thanks :) > >Why not just talk with Eliza abort it ?, maybe she know Siri >usefullness aswell?, sorry could not resist here I have no idea what you are talking about. -- Jerry From dovecot at netnerdz.se Mon May 2 18:15:48 2016 From: dovecot at netnerdz.se (Robban) Date: Mon, 2 May 2016 20:15:48 +0200 Subject: %{domain} (or %d) is empty Message-ID: <20160502181546.GW14657@shell01.saturnus.netnerdz.se> Hi all! I have two userdb's, for one of them it's ldap based and the 2nd one is static. I want mail to ldap-users to be saved in /home/vmail//Maildir So far so good. Now I want mail to non-existent users to be placed in /home/vmail//Maildir (notice the double-slashes that indicate that %d is empty): # doveadm user publicfolder at vendelso.se rs at netnerdz.se field value uid 500 gid 500 home /home/vmail// mail maildir:/home/vmail//Maildir/ uid vmail gid vmail home /home/vmail/rs mail maildir:/home/vmail/rs uidNumber 500 gidNumber 500 And here's my configuration files: # cat /etc/dovecot/dovecot-ldap.conf.ext hosts = 172.20.202.60 dn=uid=dovecot,cn=sysaccounts,cn=etc,dc=netnerdz,dc=se dnpass= auth_bind = yes auth_bind_userdn = uid=%u,cn=users,cn=accounts,dc=netnerdz,dc=se ldap_version = 3 base = cn=accounts,dc=netnerdz,dc=se deref = never scope = subtree user_filter = (&(objectClass=posixAccount)(mail=%u@*)) user_attrs = \ =user=%{ldap:uid}, \ =home=/home/vmail/%{ldap:uid}, \ =mail=maildir:/home/vmail/%{ldap:uid}, \ =uidNumber=500, \ =gidNumber=500 pass_filter = (&(objectClass=posixAccount)(mail=%u@*)) pass_attrs = uid=user,userPassword=password # dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.10.1.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_mechanisms = login gssapi plain auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %n auth_username_translation = - at .@/@_@ auth_verbose = yes base_dir = /var/run/dovecot/ imap_id_log = * imap_id_send = name imap_urlauth_host = * import_environment = TZ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = X-Original-To lmtp_save_to_detail_mailbox = yes login_greeting = IMAPd ready. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_access_groups = vmail mail_attachment_hash = %{sha256} mail_debug = yes mail_gid = vmail mail_location = maildir:~/Maildir mail_log_prefix = "%s(%u)::%n: " mail_plugins = acl virtual mail_privileged_group = vmail mail_shared_explicit_inbox = yes mail_uid = vmail mailbox_list_index = yes maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace NetnerdzSE { list = yes location = maildir:/home/vmail/netnerdz.se/:INDEX=/home/vmail/%u/public prefix = NetnerdzSE/ separator = / subscriptions = yes type = public } namespace Shared { list = children location = maildir:/home/vmail/%%n/Maildir/:INDEX=/home/vmail/%n/shared/%%u prefix = Shared/%%n/ separator = / subscriptions = yes type = shared } namespace Vendels?SE { list = yes location = maildir:/home/vmail/xn--vendels-g1a.se/:INDEX=/home/vmail/%u/public prefix = Vendels?SE/ separator = / subscriptions = yes type = public } namespace VendelsoSE { list = yes location = maildir:/home/vmail/vendelso.se/:INDEX=/home/vmail/%u/public prefix = VendelsoSE/ separator = / subscriptions = yes type = public } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/home/vmail/dovecot-acl acl_shared_dict = file:/home/vmail/shared-mailboxes.db mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size sieve = /home/vmail/%u/dovecot.sieve sieve_default = /etc/dovecot/default.sieve sieve_default_name = roundcube sieve_dir = /home/vmail/%u/sieve sieve_global_dir = /etc/dovecot/sieve/global.sieve sieve_global_path = /etc/dovecot/sieve/global.sieve sieve_max_actions = 0 sieve_max_redirects = 4 sieve_max_script_size = 0 } postmaster_address = postmaster@%d protocols = imap lmtp sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { group = mode = 0666 user = } user = $default_internal_user } service dict { unix_listener dict { group = mail mode = 0600 user = mail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 0 service_count = 1 vsz_limit = 256 M } service imap { process_limit = 1024 vsz_limit = 256 M } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_ca = References: <20160501095920.000027dc@seibercom.net> <20160502161603.5B711239C3@talvi.dovecot.org> <20160502133545.0000172f@seibercom.net> Message-ID: <053FB547-47B6-41BC-ABD0-46B67171B580@kreme.com> On Mon May 02 2016 11:35:45 Jerry said: > > On Mon, 02 May 2016 18:15:11 +0200, Benny Pedersen stated: > >> On May 1, 2016 4:00:15 PM Jerry wrote: >> >>> I am not sure if this is possible. When I use "vacation" in a sieve >>> script, it always sends the response using "<>". I need it to send >>> the response using a specific address. Is this possible? >>> >>> Thanks :) >> >> Why not just talk with Eliza abort it ?, maybe she know Siri >> usefullness aswell?, sorry could not resist here > > I have no idea what you are talking about. -- Growing up leads to growing old, and then to dying/And dying to me don't sound like all that much fun. From dovecot at netnerdz.se Mon May 2 18:51:29 2016 From: dovecot at netnerdz.se (Robban) Date: Mon, 2 May 2016 20:51:29 +0200 Subject: %{domain} (or %d) is empty Message-ID: <20160502185129.GX14657@shell01.saturnus.netnerdz.se> Hi again! I manage to solve this by removing the option auth_username_translation. The reason I wanted to have auth_username_translation was to be able to let sieve handle Is this possible with other configuration options? //Robban ----- Forwarded message from Robban ----- Date: Mon, 2 May 2016 20:15:48 +0200 From: Robban To: dovecot at dovecot.org Subject: %{domain} (or %d) is empty User-Agent: Mutt/1.5.21 (2010-09-15) Hi all! I have two userdb's, for one of them it's ldap based and the 2nd one is static. I want mail to ldap-users to be saved in /home/vmail//Maildir So far so good. Now I want mail to non-existent users to be placed in /home/vmail//Maildir (notice the double-slashes that indicate that %d is empty): # doveadm user publicfolder at vendelso.se rs at netnerdz.se field value uid 500 gid 500 home /home/vmail// mail maildir:/home/vmail//Maildir/ uid vmail gid vmail home /home/vmail/rs mail maildir:/home/vmail/rs uidNumber 500 gidNumber 500 And here's my configuration files: # cat /etc/dovecot/dovecot-ldap.conf.ext hosts = 172.20.202.60 dn=uid=dovecot,cn=sysaccounts,cn=etc,dc=netnerdz,dc=se dnpass= auth_bind = yes auth_bind_userdn = uid=%u,cn=users,cn=accounts,dc=netnerdz,dc=se ldap_version = 3 base = cn=accounts,dc=netnerdz,dc=se deref = never scope = subtree user_filter = (&(objectClass=posixAccount)(mail=%u@*)) user_attrs = \ =user=%{ldap:uid}, \ =home=/home/vmail/%{ldap:uid}, \ =mail=maildir:/home/vmail/%{ldap:uid}, \ =uidNumber=500, \ =gidNumber=500 pass_filter = (&(objectClass=posixAccount)(mail=%u@*)) pass_attrs = uid=user,userPassword=password # dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.10.1.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_mechanisms = login gssapi plain auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %n auth_username_translation = - at .@/@_@ auth_verbose = yes base_dir = /var/run/dovecot/ imap_id_log = * imap_id_send = name imap_urlauth_host = * import_environment = TZ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = X-Original-To lmtp_save_to_detail_mailbox = yes login_greeting = IMAPd ready. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_access_groups = vmail mail_attachment_hash = %{sha256} mail_debug = yes mail_gid = vmail mail_location = maildir:~/Maildir mail_log_prefix = "%s(%u)::%n: " mail_plugins = acl virtual mail_privileged_group = vmail mail_shared_explicit_inbox = yes mail_uid = vmail mailbox_list_index = yes maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace NetnerdzSE { list = yes location = maildir:/home/vmail/netnerdz.se/:INDEX=/home/vmail/%u/public prefix = NetnerdzSE/ separator = / subscriptions = yes type = public } namespace Shared { list = children location = maildir:/home/vmail/%%n/Maildir/:INDEX=/home/vmail/%n/shared/%%u prefix = Shared/%%n/ separator = / subscriptions = yes type = shared } namespace Vendels?SE { list = yes location = maildir:/home/vmail/xn--vendels-g1a.se/:INDEX=/home/vmail/%u/public prefix = Vendels?SE/ separator = / subscriptions = yes type = public } namespace VendelsoSE { list = yes location = maildir:/home/vmail/vendelso.se/:INDEX=/home/vmail/%u/public prefix = VendelsoSE/ separator = / subscriptions = yes type = public } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/home/vmail/dovecot-acl acl_shared_dict = file:/home/vmail/shared-mailboxes.db mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size sieve = /home/vmail/%u/dovecot.sieve sieve_default = /etc/dovecot/default.sieve sieve_default_name = roundcube sieve_dir = /home/vmail/%u/sieve sieve_global_dir = /etc/dovecot/sieve/global.sieve sieve_global_path = /etc/dovecot/sieve/global.sieve sieve_max_actions = 0 sieve_max_redirects = 4 sieve_max_script_size = 0 } postmaster_address = postmaster@%d protocols = imap lmtp sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { group = mode = 0666 user = } user = $default_internal_user } service dict { unix_listener dict { group = mail mode = 0600 user = mail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 0 service_count = 1 vsz_limit = 256 M } service imap { process_limit = 1024 vsz_limit = 256 M } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_ca = References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: Here is what is in phpmyadmin: password_query = SELECT username as user, SELECT IF( cryptpwd IS NULL OR cryptpwd = '', CONCAT('{PLAIN}', clearpwd), cryptpwd ) as password, '/var/vmail/%d/%n' as userdb_home, 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = '%u' AND active = '1' and the error now: #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'password_query = SELECT username as user, SELECT IF( cryptpwd IS NULL ' at line 1 On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: > On 05/02/2016 05:32 AM, Carl Jeptha wrote: > > May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: > > sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a > > field named 'password' > I'm not sure, maybe it's checking case-sensitive. Your query returns > PASSWORD. Make it lowercase. > > > > > For testing purposes I put the query in PHPMyAdmin and it complains this > > (notice it drops "PASSWORD", but shows it in the query: > > #1064 - You have an error in your SQL syntax; check the manual that > > corresponds to your MySQL server version for the right syntax to use > near '\ > > IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), > > cryptpwd) as ' at line 1 > > > > > It also sarts with a \ ... did you leave that in? That is specific to the > dovecot config file. In PHPMyAdmin you should remove the line-continuation > backslashes. > > Actually if you use the mysql command-line client, you would be able to > paste that in with the backlashes. > > Make sure to put in a real value in WHERE username = '%u' <<< > From cajeptha at gmail.com Tue May 3 10:10:59 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 3 May 2016 12:10:59 +0200 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: OK QUERY is WORKING ("password_query" relies on having a field/column "password', hence the addition under WHERE): password_query = \ SELECT username AS USER, \ IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), cryptpwd) AS PASSWORD, \ '/var/vmail/%d/%n' as userdb_home, \ 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid \ FROM mailbox \ WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') But still no happy dance, we now have a new error: dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 secs): user=, method=PLAIN, rip=165.255.109.89, lip=10.0.0.12, TLS, session= On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: > Here is what is in phpmyadmin: > password_query = > SELECT > username as user, > SELECT > IF( > cryptpwd IS NULL > OR cryptpwd = '', > CONCAT('{PLAIN}', clearpwd), > cryptpwd > ) as password, > '/var/vmail/%d/%n' as userdb_home, > 'maildir:/var/vmail/%d/%n' as userdb_mail, > 150 as userdb_uid, > 8 as userdb_gid > FROM > mailbox > WHERE > username = '%u' > AND active = '1' > > and the error now: > #1064 - You have an error in your SQL syntax; check the manual that > corresponds to your MySQL server version for the right syntax to use near > 'password_query = > SELECT > username as user, > SELECT > IF( > cryptpwd IS NULL > ' at line 1 > > On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: > >> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >> > May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >> > sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >> > field named 'password' >> I'm not sure, maybe it's checking case-sensitive. Your query returns >> PASSWORD. Make it lowercase. >> >> > >> > For testing purposes I put the query in PHPMyAdmin and it complains this >> > (notice it drops "PASSWORD", but shows it in the query: >> > #1064 - You have an error in your SQL syntax; check the manual that >> > corresponds to your MySQL server version for the right syntax to use >> near '\ >> > IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >> > cryptpwd) as ' at line 1 >> > >> > >> It also sarts with a \ ... did you leave that in? That is specific to the >> dovecot config file. In PHPMyAdmin you should remove the line-continuation >> backslashes. >> >> Actually if you use the mysql command-line client, you would be able to >> paste that in with the backlashes. >> >> Make sure to put in a real value in WHERE username = '%u' <<< >> > > From skdovecot at smail.inf.fh-brs.de Tue May 3 11:02:09 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 3 May 2016 13:02:09 +0200 (CEST) Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 3 May 2016, Carl Jeptha wrote: > OK QUERY is WORKING ("password_query" relies on having a field/column > "password', hence the addition under WHERE): > password_query = \ > SELECT username AS USER, \ > IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), > cryptpwd) AS PASSWORD, \ > '/var/vmail/%d/%n' as userdb_home, \ > 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as > userdb_gid \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') > > But still no happy dance, we now have a new error: > > dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 > secs): user=, method=PLAIN, rip=165.255.109.89, > lip=10.0.0.12, TLS, session= 1st) You should also enable auth debugging. 2nd) You are poking in the dark with SQL without understanding it, WHERE ... cryptpwd = password ('%w') ???? 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a post login script. Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. > > > > On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: > >> Here is what is in phpmyadmin: >> password_query = >> SELECT >> username as user, >> SELECT >> IF( >> cryptpwd IS NULL >> OR cryptpwd = '', >> CONCAT('{PLAIN}', clearpwd), >> cryptpwd >> ) as password, >> '/var/vmail/%d/%n' as userdb_home, >> 'maildir:/var/vmail/%d/%n' as userdb_mail, >> 150 as userdb_uid, >> 8 as userdb_gid >> FROM >> mailbox >> WHERE >> username = '%u' >> AND active = '1' >> >> and the error now: >> #1064 - You have an error in your SQL syntax; check the manual that >> corresponds to your MySQL server version for the right syntax to use near >> 'password_query = >> SELECT >> username as user, >> SELECT >> IF( >> cryptpwd IS NULL >> ' at line 1 >> >> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >> >>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>> field named 'password' >>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>> PASSWORD. Make it lowercase. >>> >>>> >>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>> (notice it drops "PASSWORD", but shows it in the query: >>>> #1064 - You have an error in your SQL syntax; check the manual that >>>> corresponds to your MySQL server version for the right syntax to use >>> near '\ >>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>> cryptpwd) as ' at line 1 >>>> >>>> >>> It also sarts with a \ ... did you leave that in? That is specific to the >>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>> backslashes. >>> >>> Actually if you use the mysql command-line client, you would be able to >>> paste that in with the backlashes. >>> >>> Make sure to put in a real value in WHERE username = '%u' <<< >>> >> >> > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== =sXel -----END PGP SIGNATURE----- From mail at tomsommer.dk Tue May 3 11:03:09 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Tue, 03 May 2016 13:03:09 +0200 Subject: Director ignores nologin, reason etc. Message-ID: <0d9188c41350e07a06ee6b8a060a6d62@tomsommer.dk> I want my director to do all authentication, but it ignores the nologin and reason etc. fields returned by my director password_query I have my director password_query returning proxy=y, password=X, master=XXXX, pass=XXXX It works fine, but even if nologin is Y and 'reason' is set, the user still gets proxied to the backend and gets to login? -- Tom From cajeptha at gmail.com Tue May 3 12:08:19 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 3 May 2016 14:08:19 +0200 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: 1. Auth debug turned on, - nothing 2. cryptpwd is the name of my "password" column, have to specify that if you want to run password_query as it relies on a field "password" to work. 3. I have access to the "clear passwords" but none of my google searches worked for converting them to SHA512_CRYPT On Tue, May 3, 2016 at 1:02 PM, Steffen Kaiser < skdovecot at smail.inf.fh-brs.de> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 3 May 2016, Carl Jeptha wrote: > > OK QUERY is WORKING ("password_query" relies on having a field/column >> "password', hence the addition under WHERE): >> password_query = \ >> SELECT username AS USER, \ >> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >> cryptpwd) AS PASSWORD, \ >> '/var/vmail/%d/%n' as userdb_home, \ >> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >> userdb_gid \ >> FROM mailbox \ >> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >> >> But still no happy dance, we now have a new error: >> >> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >> secs): user=, method=PLAIN, rip=165.255.109.89, >> lip=10.0.0.12, TLS, session= >> > > 1st) You should also enable auth debugging. > > 2nd) You are poking in the dark with SQL without understanding it, > > WHERE ... cryptpwd = password ('%w') > > ???? > > 3rd) I had the impression that you want to upgrade lower hashed passwords > into stronger hashed ones with a specific scheme and that you therefore > need to authentificate against two columns, but update the strong hashes > from the entered plain text password if missing. > > If you already have access to the clear/text passwords, hash them, put the > hashes into the database and be fine. No need for different columns and a > post login script. > > Otherwise: Nobody answered this particular question. And I see no > evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD > along. I've read the Wiki, but I see nothing like that in the code. Did > you've verified that the post login script gets the plain password? > > If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. > > > >> >> >> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >> >> Here is what is in phpmyadmin: >>> password_query = >>> SELECT >>> username as user, >>> SELECT >>> IF( >>> cryptpwd IS NULL >>> OR cryptpwd = '', >>> CONCAT('{PLAIN}', clearpwd), >>> cryptpwd >>> ) as password, >>> '/var/vmail/%d/%n' as userdb_home, >>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>> 150 as userdb_uid, >>> 8 as userdb_gid >>> FROM >>> mailbox >>> WHERE >>> username = '%u' >>> AND active = '1' >>> >>> and the error now: >>> #1064 - You have an error in your SQL syntax; check the manual that >>> corresponds to your MySQL server version for the right syntax to use near >>> 'password_query = >>> SELECT >>> username as user, >>> SELECT >>> IF( >>> cryptpwd IS NULL >>> ' at line 1 >>> >>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>> >>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>> >>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>> field named 'password' >>>>> >>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>> PASSWORD. Make it lowercase. >>>> >>>> >>>>> For testing purposes I put the query in PHPMyAdmin and it complains >>>>> this >>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>> corresponds to your MySQL server version for the right syntax to use >>>>> >>>> near '\ >>>> >>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>> cryptpwd) as ' at line 1 >>>>> >>>>> >>>>> It also sarts with a \ ... did you leave that in? That is specific to >>>> the >>>> dovecot config file. In PHPMyAdmin you should remove the >>>> line-continuation >>>> backslashes. >>>> >>>> Actually if you use the mysql command-line client, you would be able to >>>> paste that in with the backlashes. >>>> >>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>> >>>> >>> >>> >> > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH > 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd > +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW > +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG > LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG > 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== > =sXel > -----END PGP SIGNATURE----- > From gedalya at gedalya.net Tue May 3 14:24:25 2016 From: gedalya at gedalya.net (Gedalya) Date: Tue, 3 May 2016 10:24:25 -0400 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: <5728B499.7040209@gedalya.net> Drop this from the end of your query: AND cryptpwd = password ('%w') and Steffen is right, it wouldn't hurt you to get a better understanding of the principles at work here. Nothing in this thread has had anything to do with dovecot so far. On 05/03/2016 08:08 AM, Carl Jeptha wrote: > 1. Auth debug turned on, - nothing > 2. cryptpwd is the name of my "password" column, have to specify that if > you want to run password_query as it relies on a field "password" to work. > 3. I have access to the "clear passwords" but none of my google searches > worked for converting them to SHA512_CRYPT > > On Tue, May 3, 2016 at 1:02 PM, Steffen Kaiser < > skdovecot at smail.inf.fh-brs.de> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Tue, 3 May 2016, Carl Jeptha wrote: >> >> OK QUERY is WORKING ("password_query" relies on having a field/column >>> "password', hence the addition under WHERE): >>> password_query = \ >>> SELECT username AS USER, \ >>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>> cryptpwd) AS PASSWORD, \ >>> '/var/vmail/%d/%n' as userdb_home, \ >>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>> userdb_gid \ >>> FROM mailbox \ >>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>> >>> But still no happy dance, we now have a new error: >>> >>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>> secs): user=, method=PLAIN, rip=165.255.109.89, >>> lip=10.0.0.12, TLS, session= >>> >> 1st) You should also enable auth debugging. >> >> 2nd) You are poking in the dark with SQL without understanding it, >> >> WHERE ... cryptpwd = password ('%w') >> >> ???? >> >> 3rd) I had the impression that you want to upgrade lower hashed passwords >> into stronger hashed ones with a specific scheme and that you therefore >> need to authentificate against two columns, but update the strong hashes >> from the entered plain text password if missing. >> >> If you already have access to the clear/text passwords, hash them, put the >> hashes into the database and be fine. No need for different columns and a >> post login script. >> >> Otherwise: Nobody answered this particular question. And I see no >> evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD >> along. I've read the Wiki, but I see nothing like that in the code. Did >> you've verified that the post login script gets the plain password? >> >> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >> >> >> >>> >>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>> >>> Here is what is in phpmyadmin: >>>> password_query = >>>> SELECT >>>> username as user, >>>> SELECT >>>> IF( >>>> cryptpwd IS NULL >>>> OR cryptpwd = '', >>>> CONCAT('{PLAIN}', clearpwd), >>>> cryptpwd >>>> ) as password, >>>> '/var/vmail/%d/%n' as userdb_home, >>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>> 150 as userdb_uid, >>>> 8 as userdb_gid >>>> FROM >>>> mailbox >>>> WHERE >>>> username = '%u' >>>> AND active = '1' >>>> >>>> and the error now: >>>> #1064 - You have an error in your SQL syntax; check the manual that >>>> corresponds to your MySQL server version for the right syntax to use near >>>> 'password_query = >>>> SELECT >>>> username as user, >>>> SELECT >>>> IF( >>>> cryptpwd IS NULL >>>> ' at line 1 >>>> >>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>> >>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>> field named 'password' >>>>>> >>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>> PASSWORD. Make it lowercase. >>>>> >>>>> >>>>>> For testing purposes I put the query in PHPMyAdmin and it complains >>>>>> this >>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>> >>>>> near '\ >>>>> >>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>> cryptpwd) as ' at line 1 >>>>>> >>>>>> >>>>>> It also sarts with a \ ... did you leave that in? That is specific to >>>>> the >>>>> dovecot config file. In PHPMyAdmin you should remove the >>>>> line-continuation >>>>> backslashes. >>>>> >>>>> Actually if you use the mysql command-line client, you would be able to >>>>> paste that in with the backlashes. >>>>> >>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>> >>>>> >>>> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >> =sXel >> -----END PGP SIGNATURE----- >> From gedalya at gedalya.net Tue May 3 14:26:11 2016 From: gedalya at gedalya.net (Gedalya) Date: Tue, 3 May 2016 10:26:11 -0400 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: <5728B503.1040303@gedalya.net> Oh, you uppercased PASSWORD again. Change: IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), cryptpwd) AS PASSWORD To: IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), cryptpwd) AS password and again, try to understand what's going on here. On 05/03/2016 08:08 AM, Carl Jeptha wrote: > 1. Auth debug turned on, - nothing > 2. cryptpwd is the name of my "password" column, have to specify that if > you want to run password_query as it relies on a field "password" to work. > 3. I have access to the "clear passwords" but none of my google searches > worked for converting them to SHA512_CRYPT > > On Tue, May 3, 2016 at 1:02 PM, Steffen Kaiser < > skdovecot at smail.inf.fh-brs.de> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Tue, 3 May 2016, Carl Jeptha wrote: >> >> OK QUERY is WORKING ("password_query" relies on having a field/column >>> "password', hence the addition under WHERE): >>> password_query = \ >>> SELECT username AS USER, \ >>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>> cryptpwd) AS PASSWORD, \ >>> '/var/vmail/%d/%n' as userdb_home, \ >>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>> userdb_gid \ >>> FROM mailbox \ >>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>> >>> But still no happy dance, we now have a new error: >>> >>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>> secs): user=, method=PLAIN, rip=165.255.109.89, >>> lip=10.0.0.12, TLS, session= >>> >> 1st) You should also enable auth debugging. >> >> 2nd) You are poking in the dark with SQL without understanding it, >> >> WHERE ... cryptpwd = password ('%w') >> >> ???? >> >> 3rd) I had the impression that you want to upgrade lower hashed passwords >> into stronger hashed ones with a specific scheme and that you therefore >> need to authentificate against two columns, but update the strong hashes >> from the entered plain text password if missing. >> >> If you already have access to the clear/text passwords, hash them, put the >> hashes into the database and be fine. No need for different columns and a >> post login script. >> >> Otherwise: Nobody answered this particular question. And I see no >> evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD >> along. I've read the Wiki, but I see nothing like that in the code. Did >> you've verified that the post login script gets the plain password? >> >> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >> >> >> >>> >>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>> >>> Here is what is in phpmyadmin: >>>> password_query = >>>> SELECT >>>> username as user, >>>> SELECT >>>> IF( >>>> cryptpwd IS NULL >>>> OR cryptpwd = '', >>>> CONCAT('{PLAIN}', clearpwd), >>>> cryptpwd >>>> ) as password, >>>> '/var/vmail/%d/%n' as userdb_home, >>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>> 150 as userdb_uid, >>>> 8 as userdb_gid >>>> FROM >>>> mailbox >>>> WHERE >>>> username = '%u' >>>> AND active = '1' >>>> >>>> and the error now: >>>> #1064 - You have an error in your SQL syntax; check the manual that >>>> corresponds to your MySQL server version for the right syntax to use near >>>> 'password_query = >>>> SELECT >>>> username as user, >>>> SELECT >>>> IF( >>>> cryptpwd IS NULL >>>> ' at line 1 >>>> >>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>> >>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>> field named 'password' >>>>>> >>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>> PASSWORD. Make it lowercase. >>>>> >>>>> >>>>>> For testing purposes I put the query in PHPMyAdmin and it complains >>>>>> this >>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>> >>>>> near '\ >>>>> >>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>> cryptpwd) as ' at line 1 >>>>>> >>>>>> >>>>>> It also sarts with a \ ... did you leave that in? That is specific to >>>>> the >>>>> dovecot config file. In PHPMyAdmin you should remove the >>>>> line-continuation >>>>> backslashes. >>>>> >>>>> Actually if you use the mysql command-line client, you would be able to >>>>> paste that in with the backlashes. >>>>> >>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>> >>>>> >>>> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >> =sXel >> -----END PGP SIGNATURE----- >> From cajeptha at gmail.com Tue May 3 15:52:16 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 11:52:16 -0400 Subject: Changing Password Schemes In-Reply-To: <5728B503.1040303@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728B503.1040303@gedalya.net> Message-ID: <5728C930.9040902@gmail.com> Here is what worked: password_query = \ SELECT username AS USER, \ IF(password IS NULL OR password='', CONCAT('{PLAIN}',clearpwd), PASSWORD) AS password, \ '/var/vmail/%d/%n' as userdb_home, \ 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid \ FROM mailbox \ WHERE username = '%u' AND active = '1' PLEASE NOTE THE FOLLOWING as per an earlier post: you MUST have field called "password" when using password_query, but under WHERE you may show what the "password" field is called, for example: WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') ALSO NOTE my original post was about following a dovecot wiki, I was told by Gedalya to do the above, now that I "understand" that "password_query" I will move on with the rest of the solution he gave me. ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 10:26, Gedalya wrote: > Oh, you uppercased PASSWORD again. > > Change: > > IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), cryptpwd) AS PASSWORD > > To: > > IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), cryptpwd) AS password > > and again, try to understand what's going on here. > > > On 05/03/2016 08:08 AM, Carl Jeptha wrote: >> 1. Auth debug turned on, - nothing >> 2. cryptpwd is the name of my "password" column, have to specify that if >> you want to run password_query as it relies on a field "password" to work. >> 3. I have access to the "clear passwords" but none of my google searches >> worked for converting them to SHA512_CRYPT >> >> On Tue, May 3, 2016 at 1:02 PM, Steffen Kaiser < >> skdovecot at smail.inf.fh-brs.de> wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On Tue, 3 May 2016, Carl Jeptha wrote: >>> >>> OK QUERY is WORKING ("password_query" relies on having a field/column >>>> "password', hence the addition under WHERE): >>>> password_query = \ >>>> SELECT username AS USER, \ >>>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>>> cryptpwd) AS PASSWORD, \ >>>> '/var/vmail/%d/%n' as userdb_home, \ >>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>>> userdb_gid \ >>>> FROM mailbox \ >>>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>>> >>>> But still no happy dance, we now have a new error: >>>> >>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>> lip=10.0.0.12, TLS, session= >>>> >>> 1st) You should also enable auth debugging. >>> >>> 2nd) You are poking in the dark with SQL without understanding it, >>> >>> WHERE ... cryptpwd = password ('%w') >>> >>> ???? >>> >>> 3rd) I had the impression that you want to upgrade lower hashed passwords >>> into stronger hashed ones with a specific scheme and that you therefore >>> need to authentificate against two columns, but update the strong hashes >>> from the entered plain text password if missing. >>> >>> If you already have access to the clear/text passwords, hash them, put the >>> hashes into the database and be fine. No need for different columns and a >>> post login script. >>> >>> Otherwise: Nobody answered this particular question. And I see no >>> evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD >>> along. I've read the Wiki, but I see nothing like that in the code. Did >>> you've verified that the post login script gets the plain password? >>> >>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>> >>> >>> >>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>>> >>>> Here is what is in phpmyadmin: >>>>> password_query = >>>>> SELECT >>>>> username as user, >>>>> SELECT >>>>> IF( >>>>> cryptpwd IS NULL >>>>> OR cryptpwd = '', >>>>> CONCAT('{PLAIN}', clearpwd), >>>>> cryptpwd >>>>> ) as password, >>>>> '/var/vmail/%d/%n' as userdb_home, >>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>> 150 as userdb_uid, >>>>> 8 as userdb_gid >>>>> FROM >>>>> mailbox >>>>> WHERE >>>>> username = '%u' >>>>> AND active = '1' >>>>> >>>>> and the error now: >>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>> corresponds to your MySQL server version for the right syntax to use near >>>>> 'password_query = >>>>> SELECT >>>>> username as user, >>>>> SELECT >>>>> IF( >>>>> cryptpwd IS NULL >>>>> ' at line 1 >>>>> >>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>>> >>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>>> field named 'password' >>>>>>> >>>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>>> PASSWORD. Make it lowercase. >>>>>> >>>>>> >>>>>>> For testing purposes I put the query in PHPMyAdmin and it complains >>>>>>> this >>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>>> >>>>>> near '\ >>>>>> >>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>>> cryptpwd) as ' at line 1 >>>>>>> >>>>>>> >>>>>>> It also sarts with a \ ... did you leave that in? That is specific to >>>>>> the >>>>>> dovecot config file. In PHPMyAdmin you should remove the >>>>>> line-continuation >>>>>> backslashes. >>>>>> >>>>>> Actually if you use the mysql command-line client, you would be able to >>>>>> paste that in with the backlashes. >>>>>> >>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>> >>>>>> >>> - -- Steffen Kaiser >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1 >>> >>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>> =sXel >>> -----END PGP SIGNATURE----- >>> From cajeptha at gmail.com Tue May 3 15:58:40 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 11:58:40 -0400 Subject: Changing Password Schemes In-Reply-To: References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> Message-ID: <5728CAB0.2080404@gmail.com> Steffen, If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. But otherwise, I will move on, and not waste anyone's time anymore. ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 07:02, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 3 May 2016, Carl Jeptha wrote: > >> OK QUERY is WORKING ("password_query" relies on having a field/column >> "password', hence the addition under WHERE): >> password_query = \ >> SELECT username AS USER, \ >> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >> cryptpwd) AS PASSWORD, \ >> '/var/vmail/%d/%n' as userdb_home, \ >> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >> userdb_gid \ >> FROM mailbox \ >> WHERE username = '%u' AND active = '1' AND cryptpwd = password >> ('%w') >> >> But still no happy dance, we now have a new error: >> >> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >> secs): user=, method=PLAIN, rip=165.255.109.89, >> lip=10.0.0.12, TLS, session= > > 1st) You should also enable auth debugging. > > 2nd) You are poking in the dark with SQL without understanding it, > > WHERE ... cryptpwd = password ('%w') > > ???? > > 3rd) I had the impression that you want to upgrade lower hashed > passwords into stronger hashed ones with a specific scheme and that > you therefore need to authentificate against two columns, but update > the strong hashes from the entered plain text password if missing. > > If you already have access to the clear/text passwords, hash them, put > the hashes into the database and be fine. No need for different > columns and a > post login script. > > Otherwise: Nobody answered this particular question. And I see no > evidance, that Dovecot passes an environment variable named > PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that > in the code. Did you've verified that the post login script gets the > plain password? > > If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. > >> >> >> >> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >> >>> Here is what is in phpmyadmin: >>> password_query = >>> SELECT >>> username as user, >>> SELECT >>> IF( >>> cryptpwd IS NULL >>> OR cryptpwd = '', >>> CONCAT('{PLAIN}', clearpwd), >>> cryptpwd >>> ) as password, >>> '/var/vmail/%d/%n' as userdb_home, >>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>> 150 as userdb_uid, >>> 8 as userdb_gid >>> FROM >>> mailbox >>> WHERE >>> username = '%u' >>> AND active = '1' >>> >>> and the error now: >>> #1064 - You have an error in your SQL syntax; check the manual that >>> corresponds to your MySQL server version for the right syntax to use >>> near >>> 'password_query = >>> SELECT >>> username as user, >>> SELECT >>> IF( >>> cryptpwd IS NULL >>> ' at line 1 >>> >>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>> >>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>> field named 'password' >>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>> PASSWORD. Make it lowercase. >>>> >>>>> >>>>> For testing purposes I put the query in PHPMyAdmin and it >>>>> complains this >>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>> corresponds to your MySQL server version for the right syntax to use >>>> near '\ >>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>> cryptpwd) as ' at line 1 >>>>> >>>>> >>>> It also sarts with a \ ... did you leave that in? That is specific >>>> to the >>>> dovecot config file. In PHPMyAdmin you should remove the >>>> line-continuation >>>> backslashes. >>>> >>>> Actually if you use the mysql command-line client, you would be >>>> able to >>>> paste that in with the backlashes. >>>> >>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>> >>> >>> >> > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH > 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd > +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW > +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG > LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG > 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== > =sXel > -----END PGP SIGNATURE----- From gedalya at gedalya.net Tue May 3 16:10:26 2016 From: gedalya at gedalya.net (Gedalya) Date: Tue, 3 May 2016 12:10:26 -0400 Subject: Changing Password Schemes In-Reply-To: <5728CAB0.2080404@gmail.com> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> Message-ID: <5728CD72.4070904@gedalya.net> The script I sent you should do the job of populating your cryptpwd column with a SHA512-CRYPT version of the clearpwd column. The only reason why you would bother with a perl script is to get a better quality salt from /dev/urandom If you don't care so much about the quality of the salt, you can just run this single query. Make a backup of your database first!! UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; Here you are using MySQL's RAND() function to generate salt. It will do the minimum job of making the resulting encrypted password not equal to a SHA512 of the password itself, but the salt isn't very random. So the perl script I sent you reads 12 bytes of better quality random data from /dev/urandom and uses that. This means that if your database gets stolen it will be harder to decrypt the passwords. On 05/03/2016 11:58 AM, Carl Jeptha wrote: > Steffen, > If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. > > But otherwise, I will move on, and not waste anyone's time anymore. > > ------------ > You have a good day now, en mag jou m?re ook so wees, > > > Carl A Jeptha > > On 2016-05-03 07:02, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Tue, 3 May 2016, Carl Jeptha wrote: >> >>> OK QUERY is WORKING ("password_query" relies on having a field/column >>> "password', hence the addition under WHERE): >>> password_query = \ >>> SELECT username AS USER, \ >>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>> cryptpwd) AS PASSWORD, \ >>> '/var/vmail/%d/%n' as userdb_home, \ >>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>> userdb_gid \ >>> FROM mailbox \ >>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>> >>> But still no happy dance, we now have a new error: >>> >>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>> secs): user=, method=PLAIN, rip=165.255.109.89, >>> lip=10.0.0.12, TLS, session= >> >> 1st) You should also enable auth debugging. >> >> 2nd) You are poking in the dark with SQL without understanding it, >> >> WHERE ... cryptpwd = password ('%w') >> >> ???? >> >> 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. >> >> If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a >> post login script. >> >> Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? >> >> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >> >>> >>> >>> >>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>> >>>> Here is what is in phpmyadmin: >>>> password_query = >>>> SELECT >>>> username as user, >>>> SELECT >>>> IF( >>>> cryptpwd IS NULL >>>> OR cryptpwd = '', >>>> CONCAT('{PLAIN}', clearpwd), >>>> cryptpwd >>>> ) as password, >>>> '/var/vmail/%d/%n' as userdb_home, >>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>> 150 as userdb_uid, >>>> 8 as userdb_gid >>>> FROM >>>> mailbox >>>> WHERE >>>> username = '%u' >>>> AND active = '1' >>>> >>>> and the error now: >>>> #1064 - You have an error in your SQL syntax; check the manual that >>>> corresponds to your MySQL server version for the right syntax to use near >>>> 'password_query = >>>> SELECT >>>> username as user, >>>> SELECT >>>> IF( >>>> cryptpwd IS NULL >>>> ' at line 1 >>>> >>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>> >>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>> field named 'password' >>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>> PASSWORD. Make it lowercase. >>>>> >>>>>> >>>>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>> corresponds to your MySQL server version for the right syntax to use >>>>> near '\ >>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>> cryptpwd) as ' at line 1 >>>>>> >>>>>> >>>>> It also sarts with a \ ... did you leave that in? That is specific to the >>>>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>>>> backslashes. >>>>> >>>>> Actually if you use the mysql command-line client, you would be able to >>>>> paste that in with the backlashes. >>>>> >>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>> >>>> >>>> >>> >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >> =sXel >> -----END PGP SIGNATURE----- From cajeptha at gmail.com Tue May 3 16:29:29 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 18:29:29 +0200 Subject: Changing Password Schemes In-Reply-To: <5728CD72.4070904@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> Message-ID: <5728D1E9.4080100@gmail.com> Thank you, Due to changes I had to make to let password_query work, I think your "quick" version should be like this my setup: UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 18:10, Gedalya wrote: > The script I sent you should do the job of populating your cryptpwd column with a SHA512-CRYPT version of the clearpwd column. > The only reason why you would bother with a perl script is to get a better quality salt from /dev/urandom > If you don't care so much about the quality of the salt, you can just run this single query. > Make a backup of your database first!! > > UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; > > Here you are using MySQL's RAND() function to generate salt. It will do the minimum job of making the resulting encrypted password not equal to a SHA512 of the password itself, but the salt isn't very random. So the perl script I sent you reads 12 bytes of better quality random data from /dev/urandom and uses that. This means that if your database gets stolen it will be harder to decrypt the passwords. > > > On 05/03/2016 11:58 AM, Carl Jeptha wrote: >> Steffen, >> If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. >> >> But otherwise, I will move on, and not waste anyone's time anymore. >> >> ------------ >> You have a good day now, en mag jou m?re ook so wees, >> >> >> Carl A Jeptha >> >> On 2016-05-03 07:02, Steffen Kaiser wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On Tue, 3 May 2016, Carl Jeptha wrote: >>> >>>> OK QUERY is WORKING ("password_query" relies on having a field/column >>>> "password', hence the addition under WHERE): >>>> password_query = \ >>>> SELECT username AS USER, \ >>>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>>> cryptpwd) AS PASSWORD, \ >>>> '/var/vmail/%d/%n' as userdb_home, \ >>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>>> userdb_gid \ >>>> FROM mailbox \ >>>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>>> >>>> But still no happy dance, we now have a new error: >>>> >>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>> lip=10.0.0.12, TLS, session= >>> 1st) You should also enable auth debugging. >>> >>> 2nd) You are poking in the dark with SQL without understanding it, >>> >>> WHERE ... cryptpwd = password ('%w') >>> >>> ???? >>> >>> 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. >>> >>> If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a >>> post login script. >>> >>> Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? >>> >>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>> >>>> >>>> >>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>>> >>>>> Here is what is in phpmyadmin: >>>>> password_query = >>>>> SELECT >>>>> username as user, >>>>> SELECT >>>>> IF( >>>>> cryptpwd IS NULL >>>>> OR cryptpwd = '', >>>>> CONCAT('{PLAIN}', clearpwd), >>>>> cryptpwd >>>>> ) as password, >>>>> '/var/vmail/%d/%n' as userdb_home, >>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>> 150 as userdb_uid, >>>>> 8 as userdb_gid >>>>> FROM >>>>> mailbox >>>>> WHERE >>>>> username = '%u' >>>>> AND active = '1' >>>>> >>>>> and the error now: >>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>> corresponds to your MySQL server version for the right syntax to use near >>>>> 'password_query = >>>>> SELECT >>>>> username as user, >>>>> SELECT >>>>> IF( >>>>> cryptpwd IS NULL >>>>> ' at line 1 >>>>> >>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>>> >>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>>> field named 'password' >>>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>>> PASSWORD. Make it lowercase. >>>>>> >>>>>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>> near '\ >>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>>> cryptpwd) as ' at line 1 >>>>>>> >>>>>>> >>>>>> It also sarts with a \ ... did you leave that in? That is specific to the >>>>>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>>>>> backslashes. >>>>>> >>>>>> Actually if you use the mysql command-line client, you would be able to >>>>>> paste that in with the backlashes. >>>>>> >>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>> >>>>> >>> - -- Steffen Kaiser >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1 >>> >>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>> =sXel >>> -----END PGP SIGNATURE----- From gedalya at gedalya.net Tue May 3 16:33:13 2016 From: gedalya at gedalya.net (Gedalya) Date: Tue, 3 May 2016 12:33:13 -0400 Subject: Changing Password Schemes In-Reply-To: <5728D1E9.4080100@gmail.com> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> Message-ID: <5728D2C9.6000109@gedalya.net> Just make sure it says: WHERE password IS NULL OR password=''; With no space between the quote marks, this way it matches an empty string On 05/03/2016 12:29 PM, Carl Jeptha wrote: > Thank you, > Due to changes I had to make to let password_query work, I think your "quick" version should be like this my setup: > > UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; > > ------------ > You have a good day now, en mag jou m?re ook so wees, > > Carl A Jeptha > > On 2016-05-03 18:10, Gedalya wrote: >> The script I sent you should do the job of populating your cryptpwd column with a SHA512-CRYPT version of the clearpwd column. >> The only reason why you would bother with a perl script is to get a better quality salt from /dev/urandom >> If you don't care so much about the quality of the salt, you can just run this single query. >> Make a backup of your database first!! >> >> UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; >> >> Here you are using MySQL's RAND() function to generate salt. It will do the minimum job of making the resulting encrypted password not equal to a SHA512 of the password itself, but the salt isn't very random. So the perl script I sent you reads 12 bytes of better quality random data from /dev/urandom and uses that. This means that if your database gets stolen it will be harder to decrypt the passwords. >> >> >> On 05/03/2016 11:58 AM, Carl Jeptha wrote: >>> Steffen, >>> If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. >>> >>> But otherwise, I will move on, and not waste anyone's time anymore. >>> >>> ------------ >>> You have a good day now, en mag jou m?re ook so wees, >>> >>> >>> Carl A Jeptha >>> >>> On 2016-05-03 07:02, Steffen Kaiser wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> On Tue, 3 May 2016, Carl Jeptha wrote: >>>> >>>>> OK QUERY is WORKING ("password_query" relies on having a field/column >>>>> "password', hence the addition under WHERE): >>>>> password_query = \ >>>>> SELECT username AS USER, \ >>>>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>>>> cryptpwd) AS PASSWORD, \ >>>>> '/var/vmail/%d/%n' as userdb_home, \ >>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>>>> userdb_gid \ >>>>> FROM mailbox \ >>>>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>>>> >>>>> But still no happy dance, we now have a new error: >>>>> >>>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>>> lip=10.0.0.12, TLS, session= >>>> 1st) You should also enable auth debugging. >>>> >>>> 2nd) You are poking in the dark with SQL without understanding it, >>>> >>>> WHERE ... cryptpwd = password ('%w') >>>> >>>> ???? >>>> >>>> 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. >>>> >>>> If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a >>>> post login script. >>>> >>>> Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? >>>> >>>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>>> >>>>> >>>>> >>>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>>>> >>>>>> Here is what is in phpmyadmin: >>>>>> password_query = >>>>>> SELECT >>>>>> username as user, >>>>>> SELECT >>>>>> IF( >>>>>> cryptpwd IS NULL >>>>>> OR cryptpwd = '', >>>>>> CONCAT('{PLAIN}', clearpwd), >>>>>> cryptpwd >>>>>> ) as password, >>>>>> '/var/vmail/%d/%n' as userdb_home, >>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>>> 150 as userdb_uid, >>>>>> 8 as userdb_gid >>>>>> FROM >>>>>> mailbox >>>>>> WHERE >>>>>> username = '%u' >>>>>> AND active = '1' >>>>>> >>>>>> and the error now: >>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>> corresponds to your MySQL server version for the right syntax to use near >>>>>> 'password_query = >>>>>> SELECT >>>>>> username as user, >>>>>> SELECT >>>>>> IF( >>>>>> cryptpwd IS NULL >>>>>> ' at line 1 >>>>>> >>>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>>>> >>>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>>>> field named 'password' >>>>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>>>> PASSWORD. Make it lowercase. >>>>>>> >>>>>>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>>> near '\ >>>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>>>> cryptpwd) as ' at line 1 >>>>>>>> >>>>>>>> >>>>>>> It also sarts with a \ ... did you leave that in? That is specific to the >>>>>>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>>>>>> backslashes. >>>>>>> >>>>>>> Actually if you use the mysql command-line client, you would be able to >>>>>>> paste that in with the backlashes. >>>>>>> >>>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>>> >>>>>> >>>> - -- Steffen Kaiser >>>> -----BEGIN PGP SIGNATURE----- >>>> Version: GnuPG v1 >>>> >>>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>>> =sXel >>>> -----END PGP SIGNATURE----- From cajeptha at gmail.com Tue May 3 16:49:16 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 18:49:16 +0200 Subject: Changing Password Schemes In-Reply-To: <5728D2C9.6000109@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> <5728D2C9.6000109@gedalya.net> Message-ID: <5728D68C.5000509@gmail.com> OK, I ran that code on a "backup" database using phpmyadmin and it ran the code: SQL query: UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password='' Matched rows: 0 ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 18:33, Gedalya wrote: > UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; From cajeptha at gmail.com Tue May 3 17:07:52 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 19:07:52 +0200 Subject: Changing Password Schemes In-Reply-To: <5728D2C9.6000109@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> <5728D2C9.6000109@gedalya.net> Message-ID: <5728DAE8.9050905@gmail.com> Just tried to run it on the "Live" database, the simulation found all the rows, but when I ran the query I got this error (still trying to see what mus be changed): |#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.sha(RAND()))) WHERE password IS NULL OR password=''' at line 1 | ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 18:33, Gedalya wrote: > Just make sure it says: > > WHERE password IS NULL OR password=''; > > With no space between the quote marks, this way it matches an empty string > > > On 05/03/2016 12:29 PM, Carl Jeptha wrote: >> Thank you, >> Due to changes I had to make to let password_query work, I think your "quick" version should be like this my setup: >> >> UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; >> >> ------------ >> You have a good day now, en mag jou m?re ook so wees, >> >> Carl A Jeptha >> >> On 2016-05-03 18:10, Gedalya wrote: >>> The script I sent you should do the job of populating your cryptpwd column with a SHA512-CRYPT version of the clearpwd column. >>> The only reason why you would bother with a perl script is to get a better quality salt from /dev/urandom >>> If you don't care so much about the quality of the salt, you can just run this single query. >>> Make a backup of your database first!! >>> >>> UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; >>> >>> Here you are using MySQL's RAND() function to generate salt. It will do the minimum job of making the resulting encrypted password not equal to a SHA512 of the password itself, but the salt isn't very random. So the perl script I sent you reads 12 bytes of better quality random data from /dev/urandom and uses that. This means that if your database gets stolen it will be harder to decrypt the passwords. >>> >>> >>> On 05/03/2016 11:58 AM, Carl Jeptha wrote: >>>> Steffen, >>>> If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. >>>> >>>> But otherwise, I will move on, and not waste anyone's time anymore. >>>> >>>> ------------ >>>> You have a good day now, en mag jou m?re ook so wees, >>>> >>>> >>>> Carl A Jeptha >>>> >>>> On 2016-05-03 07:02, Steffen Kaiser wrote: >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> On Tue, 3 May 2016, Carl Jeptha wrote: >>>>> >>>>>> OK QUERY is WORKING ("password_query" relies on having a field/column >>>>>> "password', hence the addition under WHERE): >>>>>> password_query = \ >>>>>> SELECT username AS USER, \ >>>>>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>>>>> cryptpwd) AS PASSWORD, \ >>>>>> '/var/vmail/%d/%n' as userdb_home, \ >>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>>>>> userdb_gid \ >>>>>> FROM mailbox \ >>>>>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>>>>> >>>>>> But still no happy dance, we now have a new error: >>>>>> >>>>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>>>> lip=10.0.0.12, TLS, session= >>>>> 1st) You should also enable auth debugging. >>>>> >>>>> 2nd) You are poking in the dark with SQL without understanding it, >>>>> >>>>> WHERE ... cryptpwd = password ('%w') >>>>> >>>>> ???? >>>>> >>>>> 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. >>>>> >>>>> If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a >>>>> post login script. >>>>> >>>>> Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? >>>>> >>>>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>>>> >>>>>> >>>>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>>>>> >>>>>>> Here is what is in phpmyadmin: >>>>>>> password_query = >>>>>>> SELECT >>>>>>> username as user, >>>>>>> SELECT >>>>>>> IF( >>>>>>> cryptpwd IS NULL >>>>>>> OR cryptpwd = '', >>>>>>> CONCAT('{PLAIN}', clearpwd), >>>>>>> cryptpwd >>>>>>> ) as password, >>>>>>> '/var/vmail/%d/%n' as userdb_home, >>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>>>> 150 as userdb_uid, >>>>>>> 8 as userdb_gid >>>>>>> FROM >>>>>>> mailbox >>>>>>> WHERE >>>>>>> username = '%u' >>>>>>> AND active = '1' >>>>>>> >>>>>>> and the error now: >>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>> corresponds to your MySQL server version for the right syntax to use near >>>>>>> 'password_query = >>>>>>> SELECT >>>>>>> username as user, >>>>>>> SELECT >>>>>>> IF( >>>>>>> cryptpwd IS NULL >>>>>>> ' at line 1 >>>>>>> >>>>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>>>>> >>>>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>>>>> field named 'password' >>>>>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>>>>> PASSWORD. Make it lowercase. >>>>>>>> >>>>>>>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>>>> near '\ >>>>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>>>>> cryptpwd) as ' at line 1 >>>>>>>>> >>>>>>>>> >>>>>>>> It also sarts with a \ ... did you leave that in? That is specific to the >>>>>>>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>>>>>>> backslashes. >>>>>>>> >>>>>>>> Actually if you use the mysql command-line client, you would be able to >>>>>>>> paste that in with the backlashes. >>>>>>>> >>>>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>>>> >>>>> - -- Steffen Kaiser >>>>> -----BEGIN PGP SIGNATURE----- >>>>> Version: GnuPG v1 >>>>> >>>>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>>>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>>>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>>>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>>>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>>>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>>>> =sXel >>>>> -----END PGP SIGNATURE----- From dovecot.mtoth at queldor.net Tue May 3 17:13:51 2016 From: dovecot.mtoth at queldor.net (Michael Toth) Date: Tue, 3 May 2016 13:13:51 -0400 Subject: [DOVECOT] Re: [DOVECOT] Re: Changing Password Schemes In-Reply-To: <5728DAE8.9050905@gmail.com> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> <5728D2C9.6000109@gedalya.net> <5728DAE8.9050905@gmail.com> Message-ID: <5499cecd-9e35-ff74-d45a-c676cc114bbb@queldor.net> You have a typo in your SQL statement it should be ,sha not .sha On 5/3/2016 1:07 PM, Carl Jeptha wrote: > Just tried to run it on the "Live" database, the simulation found all > the rows, but when I ran the query I got this error (still trying to see > what mus be changed): > |#1064 - You have an error in your SQL syntax; check the manual that > corresponds to your MySQL server version for the right syntax to use > near '.sha(RAND()))) WHERE password IS NULL OR password=''' at line 1 | > > ------------ > You have a good day now, en mag jou m?re ook so wees, > > Carl A Jeptha > > On 2016-05-03 18:33, Gedalya wrote: >> Just make sure it says: >> >> WHERE password IS NULL OR password=''; >> >> With no space between the quote marks, this way it matches an empty >> string >> >> >> On 05/03/2016 12:29 PM, Carl Jeptha wrote: >>> Thank you, >>> Due to changes I had to make to let password_query work, I think your >>> "quick" version should be like this my setup: >>> >>> UPDATE mailbox set password = ENCRYPT(clearpwd, >>> CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; >>> >>> ------------ >>> You have a good day now, en mag jou m?re ook so wees, >>> >>> Carl A Jeptha >>> >>> On 2016-05-03 18:10, Gedalya wrote: >>>> The script I sent you should do the job of populating your cryptpwd >>>> column with a SHA512-CRYPT version of the clearpwd column. >>>> The only reason why you would bother with a perl script is to get a >>>> better quality salt from /dev/urandom >>>> If you don't care so much about the quality of the salt, you can >>>> just run this single query. >>>> Make a backup of your database first!! >>>> >>>> UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, >>>> CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; >>>> >>>> Here you are using MySQL's RAND() function to generate salt. It will >>>> do the minimum job of making the resulting encrypted password not >>>> equal to a SHA512 of the password itself, but the salt isn't very >>>> random. So the perl script I sent you reads 12 bytes of better >>>> quality random data from /dev/urandom and uses that. This means that >>>> if your database gets stolen it will be harder to decrypt the >>>> passwords. >>>> >>>> >>>> On 05/03/2016 11:58 AM, Carl Jeptha wrote: >>>>> Steffen, >>>>> If you can point me in the direction as to how to convert a column >>>>> of clear text passwords to SHA512-CRYPT I will be happy to follow >>>>> it and close this query, I only came here because I had spent >>>>> almost two weeks trying to make the dovecot wiki work and thought >>>>> someone would point out the mistakes I had made. >>>>> >>>>> But otherwise, I will move on, and not waste anyone's time anymore. >>>>> >>>>> ------------ >>>>> You have a good day now, en mag jou m?re ook so wees, >>>>> >>>>> >>>>> Carl A Jeptha >>>>> >>>>> On 2016-05-03 07:02, Steffen Kaiser wrote: >>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>> Hash: SHA1 >>>>>> >>>>>> On Tue, 3 May 2016, Carl Jeptha wrote: >>>>>> >>>>>>> OK QUERY is WORKING ("password_query" relies on having a >>>>>>> field/column >>>>>>> "password', hence the addition under WHERE): >>>>>>> password_query = \ >>>>>>> SELECT username AS USER, \ >>>>>>> IF(cryptpwd IS NULL OR cryptpwd=' ', >>>>>>> CONCAT('{PLAIN}',clearpwd), >>>>>>> cryptpwd) AS PASSWORD, \ >>>>>>> '/var/vmail/%d/%n' as userdb_home, \ >>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as >>>>>>> userdb_uid, 8 as >>>>>>> userdb_gid \ >>>>>>> FROM mailbox \ >>>>>>> WHERE username = '%u' AND active = '1' AND cryptpwd = >>>>>>> password ('%w') >>>>>>> >>>>>>> But still no happy dance, we now have a new error: >>>>>>> >>>>>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>>>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>>>>> lip=10.0.0.12, TLS, session= >>>>>> 1st) You should also enable auth debugging. >>>>>> >>>>>> 2nd) You are poking in the dark with SQL without understanding it, >>>>>> >>>>>> WHERE ... cryptpwd = password ('%w') >>>>>> >>>>>> ???? >>>>>> >>>>>> 3rd) I had the impression that you want to upgrade lower hashed >>>>>> passwords into stronger hashed ones with a specific scheme and >>>>>> that you therefore need to authentificate against two columns, but >>>>>> update the strong hashes from the entered plain text password if >>>>>> missing. >>>>>> >>>>>> If you already have access to the clear/text passwords, hash them, >>>>>> put the hashes into the database and be fine. No need for >>>>>> different columns and a >>>>>> post login script. >>>>>> >>>>>> Otherwise: Nobody answered this particular question. And I see no >>>>>> evidance, that Dovecot passes an environment variable named >>>>>> PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like >>>>>> that in the code. Did you've verified that the post login script >>>>>> gets the plain password? >>>>>> >>>>>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>>>>> >>>>>>> >>>>>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha >>>>>>> wrote: >>>>>>> >>>>>>>> Here is what is in phpmyadmin: >>>>>>>> password_query = >>>>>>>> SELECT >>>>>>>> username as user, >>>>>>>> SELECT >>>>>>>> IF( >>>>>>>> cryptpwd IS NULL >>>>>>>> OR cryptpwd = '', >>>>>>>> CONCAT('{PLAIN}', clearpwd), >>>>>>>> cryptpwd >>>>>>>> ) as password, >>>>>>>> '/var/vmail/%d/%n' as userdb_home, >>>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>>>>> 150 as userdb_uid, >>>>>>>> 8 as userdb_gid >>>>>>>> FROM >>>>>>>> mailbox >>>>>>>> WHERE >>>>>>>> username = '%u' >>>>>>>> AND active = '1' >>>>>>>> >>>>>>>> and the error now: >>>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>>> corresponds to your MySQL server version for the right syntax to >>>>>>>> use near >>>>>>>> 'password_query = >>>>>>>> SELECT >>>>>>>> username as user, >>>>>>>> SELECT >>>>>>>> IF( >>>>>>>> cryptpwd IS NULL >>>>>>>> ' at line 1 >>>>>>>> >>>>>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must >>>>>>>>>> return a >>>>>>>>>> field named 'password' >>>>>>>>> I'm not sure, maybe it's checking case-sensitive. Your query >>>>>>>>> returns >>>>>>>>> PASSWORD. Make it lowercase. >>>>>>>>> >>>>>>>>>> For testing purposes I put the query in PHPMyAdmin and it >>>>>>>>>> complains this >>>>>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>>>>> #1064 - You have an error in your SQL syntax; check the manual >>>>>>>>>> that >>>>>>>>>> corresponds to your MySQL server version for the right syntax >>>>>>>>>> to use >>>>>>>>> near '\ >>>>>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', >>>>>>>>>> CONCAT('{PLAIN}',clearpwd), >>>>>>>>>> cryptpwd) as ' at line 1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>> It also sarts with a \ ... did you leave that in? That is >>>>>>>>> specific to the >>>>>>>>> dovecot config file. In PHPMyAdmin you should remove the >>>>>>>>> line-continuation >>>>>>>>> backslashes. >>>>>>>>> >>>>>>>>> Actually if you use the mysql command-line client, you would be >>>>>>>>> able to >>>>>>>>> paste that in with the backlashes. >>>>>>>>> >>>>>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>>>>> >>>>>> - -- Steffen Kaiser >>>>>> -----BEGIN PGP SIGNATURE----- >>>>>> Version: GnuPG v1 >>>>>> >>>>>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>>>>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>>>>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>>>>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>>>>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>>>>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>>>>> =sXel >>>>>> -----END PGP SIGNATURE----- From cajeptha at gmail.com Tue May 3 17:14:15 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 19:14:15 +0200 Subject: Changing Password Schemes In-Reply-To: <5728D2C9.6000109@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> <5728D2C9.6000109@gedalya.net> Message-ID: <5728DC67.3040003@gmail.com> Sorry was giving a rapid update, but there was an error, which I picked up immediately, ".sha" should have been ",sha". The query was successful, Thank you, and I did learn a lot. Now to transfer all the mailbox folders from the old server to the new server. ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 18:33, Gedalya wrote: > Just make sure it says: > > WHERE password IS NULL OR password=''; > > With no space between the quote marks, this way it matches an empty string > > > On 05/03/2016 12:29 PM, Carl Jeptha wrote: >> Thank you, >> Due to changes I had to make to let password_query work, I think your "quick" version should be like this my setup: >> >> UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; >> >> ------------ >> You have a good day now, en mag jou m?re ook so wees, >> >> Carl A Jeptha >> >> On 2016-05-03 18:10, Gedalya wrote: >>> The script I sent you should do the job of populating your cryptpwd column with a SHA512-CRYPT version of the clearpwd column. >>> The only reason why you would bother with a perl script is to get a better quality salt from /dev/urandom >>> If you don't care so much about the quality of the salt, you can just run this single query. >>> Make a backup of your database first!! >>> >>> UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; >>> >>> Here you are using MySQL's RAND() function to generate salt. It will do the minimum job of making the resulting encrypted password not equal to a SHA512 of the password itself, but the salt isn't very random. So the perl script I sent you reads 12 bytes of better quality random data from /dev/urandom and uses that. This means that if your database gets stolen it will be harder to decrypt the passwords. >>> >>> >>> On 05/03/2016 11:58 AM, Carl Jeptha wrote: >>>> Steffen, >>>> If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. >>>> >>>> But otherwise, I will move on, and not waste anyone's time anymore. >>>> >>>> ------------ >>>> You have a good day now, en mag jou m?re ook so wees, >>>> >>>> >>>> Carl A Jeptha >>>> >>>> On 2016-05-03 07:02, Steffen Kaiser wrote: >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> On Tue, 3 May 2016, Carl Jeptha wrote: >>>>> >>>>>> OK QUERY is WORKING ("password_query" relies on having a field/column >>>>>> "password', hence the addition under WHERE): >>>>>> password_query = \ >>>>>> SELECT username AS USER, \ >>>>>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>>>>> cryptpwd) AS PASSWORD, \ >>>>>> '/var/vmail/%d/%n' as userdb_home, \ >>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>>>>> userdb_gid \ >>>>>> FROM mailbox \ >>>>>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>>>>> >>>>>> But still no happy dance, we now have a new error: >>>>>> >>>>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>>>> lip=10.0.0.12, TLS, session= >>>>> 1st) You should also enable auth debugging. >>>>> >>>>> 2nd) You are poking in the dark with SQL without understanding it, >>>>> >>>>> WHERE ... cryptpwd = password ('%w') >>>>> >>>>> ???? >>>>> >>>>> 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. >>>>> >>>>> If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a >>>>> post login script. >>>>> >>>>> Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? >>>>> >>>>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>>>> >>>>>> >>>>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>>>>> >>>>>>> Here is what is in phpmyadmin: >>>>>>> password_query = >>>>>>> SELECT >>>>>>> username as user, >>>>>>> SELECT >>>>>>> IF( >>>>>>> cryptpwd IS NULL >>>>>>> OR cryptpwd = '', >>>>>>> CONCAT('{PLAIN}', clearpwd), >>>>>>> cryptpwd >>>>>>> ) as password, >>>>>>> '/var/vmail/%d/%n' as userdb_home, >>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>>>> 150 as userdb_uid, >>>>>>> 8 as userdb_gid >>>>>>> FROM >>>>>>> mailbox >>>>>>> WHERE >>>>>>> username = '%u' >>>>>>> AND active = '1' >>>>>>> >>>>>>> and the error now: >>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>> corresponds to your MySQL server version for the right syntax to use near >>>>>>> 'password_query = >>>>>>> SELECT >>>>>>> username as user, >>>>>>> SELECT >>>>>>> IF( >>>>>>> cryptpwd IS NULL >>>>>>> ' at line 1 >>>>>>> >>>>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>>>>> >>>>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>>>>> field named 'password' >>>>>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>>>>> PASSWORD. Make it lowercase. >>>>>>>> >>>>>>>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>>>> near '\ >>>>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>>>>> cryptpwd) as ' at line 1 >>>>>>>>> >>>>>>>>> >>>>>>>> It also sarts with a \ ... did you leave that in? That is specific to the >>>>>>>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>>>>>>> backslashes. >>>>>>>> >>>>>>>> Actually if you use the mysql command-line client, you would be able to >>>>>>>> paste that in with the backlashes. >>>>>>>> >>>>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>>>> >>>>> - -- Steffen Kaiser >>>>> -----BEGIN PGP SIGNATURE----- >>>>> Version: GnuPG v1 >>>>> >>>>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>>>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>>>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>>>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>>>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>>>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>>>> =sXel >>>>> -----END PGP SIGNATURE----- From cajeptha at gmail.com Tue May 3 17:16:18 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 19:16:18 +0200 Subject: [DOVECOT] Re: [DOVECOT] Re: Changing Password Schemes In-Reply-To: <5499cecd-9e35-ff74-d45a-c676cc114bbb@queldor.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> <5728D2C9.6000109@gedalya.net> <5728DAE8.9050905@gmail.com> <5499cecd-9e35-ff74-d45a-c676cc114bbb@queldor.net> Message-ID: <5728DCE2.2080105@gmail.com> You are to fast, see my "late" email :-[ I picked it up immediately after sending the update. ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 19:13, Michael Toth wrote: > You have a typo in your SQL statement it should be ,sha not .sha > > On 5/3/2016 1:07 PM, Carl Jeptha wrote: >> Just tried to run it on the "Live" database, the simulation found all >> the rows, but when I ran the query I got this error (still trying to see >> what mus be changed): >> |#1064 - You have an error in your SQL syntax; check the manual that >> corresponds to your MySQL server version for the right syntax to use >> near '.sha(RAND()))) WHERE password IS NULL OR password=''' at line 1 | >> >> ------------ >> You have a good day now, en mag jou m?re ook so wees, >> >> Carl A Jeptha >> >> On 2016-05-03 18:33, Gedalya wrote: >>> Just make sure it says: >>> >>> WHERE password IS NULL OR password=''; >>> >>> With no space between the quote marks, this way it matches an empty >>> string >>> >>> >>> On 05/03/2016 12:29 PM, Carl Jeptha wrote: >>>> Thank you, >>>> Due to changes I had to make to let password_query work, I think your >>>> "quick" version should be like this my setup: >>>> >>>> UPDATE mailbox set password = ENCRYPT(clearpwd, >>>> CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; >>>> >>>> ------------ >>>> You have a good day now, en mag jou m?re ook so wees, >>>> >>>> Carl A Jeptha >>>> >>>> On 2016-05-03 18:10, Gedalya wrote: >>>>> The script I sent you should do the job of populating your cryptpwd >>>>> column with a SHA512-CRYPT version of the clearpwd column. >>>>> The only reason why you would bother with a perl script is to get a >>>>> better quality salt from /dev/urandom >>>>> If you don't care so much about the quality of the salt, you can >>>>> just run this single query. >>>>> Make a backup of your database first!! >>>>> >>>>> UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, >>>>> CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; >>>>> >>>>> Here you are using MySQL's RAND() function to generate salt. It will >>>>> do the minimum job of making the resulting encrypted password not >>>>> equal to a SHA512 of the password itself, but the salt isn't very >>>>> random. So the perl script I sent you reads 12 bytes of better >>>>> quality random data from /dev/urandom and uses that. This means that >>>>> if your database gets stolen it will be harder to decrypt the >>>>> passwords. >>>>> >>>>> >>>>> On 05/03/2016 11:58 AM, Carl Jeptha wrote: >>>>>> Steffen, >>>>>> If you can point me in the direction as to how to convert a column >>>>>> of clear text passwords to SHA512-CRYPT I will be happy to follow >>>>>> it and close this query, I only came here because I had spent >>>>>> almost two weeks trying to make the dovecot wiki work and thought >>>>>> someone would point out the mistakes I had made. >>>>>> >>>>>> But otherwise, I will move on, and not waste anyone's time anymore. >>>>>> >>>>>> ------------ >>>>>> You have a good day now, en mag jou m?re ook so wees, >>>>>> >>>>>> >>>>>> Carl A Jeptha >>>>>> >>>>>> On 2016-05-03 07:02, Steffen Kaiser wrote: >>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>> Hash: SHA1 >>>>>>> >>>>>>> On Tue, 3 May 2016, Carl Jeptha wrote: >>>>>>> >>>>>>>> OK QUERY is WORKING ("password_query" relies on having a >>>>>>>> field/column >>>>>>>> "password', hence the addition under WHERE): >>>>>>>> password_query = \ >>>>>>>> SELECT username AS USER, \ >>>>>>>> IF(cryptpwd IS NULL OR cryptpwd=' ', >>>>>>>> CONCAT('{PLAIN}',clearpwd), >>>>>>>> cryptpwd) AS PASSWORD, \ >>>>>>>> '/var/vmail/%d/%n' as userdb_home, \ >>>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as >>>>>>>> userdb_uid, 8 as >>>>>>>> userdb_gid \ >>>>>>>> FROM mailbox \ >>>>>>>> WHERE username = '%u' AND active = '1' AND cryptpwd = >>>>>>>> password ('%w') >>>>>>>> >>>>>>>> But still no happy dance, we now have a new error: >>>>>>>> >>>>>>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>>>>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>>>>>> lip=10.0.0.12, TLS, session= >>>>>>> 1st) You should also enable auth debugging. >>>>>>> >>>>>>> 2nd) You are poking in the dark with SQL without understanding it, >>>>>>> >>>>>>> WHERE ... cryptpwd = password ('%w') >>>>>>> >>>>>>> ???? >>>>>>> >>>>>>> 3rd) I had the impression that you want to upgrade lower hashed >>>>>>> passwords into stronger hashed ones with a specific scheme and >>>>>>> that you therefore need to authentificate against two columns, but >>>>>>> update the strong hashes from the entered plain text password if >>>>>>> missing. >>>>>>> >>>>>>> If you already have access to the clear/text passwords, hash them, >>>>>>> put the hashes into the database and be fine. No need for >>>>>>> different columns and a >>>>>>> post login script. >>>>>>> >>>>>>> Otherwise: Nobody answered this particular question. And I see no >>>>>>> evidance, that Dovecot passes an environment variable named >>>>>>> PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like >>>>>>> that in the code. Did you've verified that the post login script >>>>>>> gets the plain password? >>>>>>> >>>>>>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is >>>>>>> nonsense. >>>>>>> >>>>>>>> >>>>>>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Here is what is in phpmyadmin: >>>>>>>>> password_query = >>>>>>>>> SELECT >>>>>>>>> username as user, >>>>>>>>> SELECT >>>>>>>>> IF( >>>>>>>>> cryptpwd IS NULL >>>>>>>>> OR cryptpwd = '', >>>>>>>>> CONCAT('{PLAIN}', clearpwd), >>>>>>>>> cryptpwd >>>>>>>>> ) as password, >>>>>>>>> '/var/vmail/%d/%n' as userdb_home, >>>>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>>>>>> 150 as userdb_uid, >>>>>>>>> 8 as userdb_gid >>>>>>>>> FROM >>>>>>>>> mailbox >>>>>>>>> WHERE >>>>>>>>> username = '%u' >>>>>>>>> AND active = '1' >>>>>>>>> >>>>>>>>> and the error now: >>>>>>>>> #1064 - You have an error in your SQL syntax; check the manual >>>>>>>>> that >>>>>>>>> corresponds to your MySQL server version for the right syntax to >>>>>>>>> use near >>>>>>>>> 'password_query = >>>>>>>>> SELECT >>>>>>>>> username as user, >>>>>>>>> SELECT >>>>>>>>> IF( >>>>>>>>> cryptpwd IS NULL >>>>>>>>> ' at line 1 >>>>>>>>> >>>>>>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must >>>>>>>>>>> return a >>>>>>>>>>> field named 'password' >>>>>>>>>> I'm not sure, maybe it's checking case-sensitive. Your query >>>>>>>>>> returns >>>>>>>>>> PASSWORD. Make it lowercase. >>>>>>>>>> >>>>>>>>>>> For testing purposes I put the query in PHPMyAdmin and it >>>>>>>>>>> complains this >>>>>>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>>>>>> #1064 - You have an error in your SQL syntax; check the manual >>>>>>>>>>> that >>>>>>>>>>> corresponds to your MySQL server version for the right syntax >>>>>>>>>>> to use >>>>>>>>>> near '\ >>>>>>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', >>>>>>>>>>> CONCAT('{PLAIN}',clearpwd), >>>>>>>>>>> cryptpwd) as ' at line 1 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> It also sarts with a \ ... did you leave that in? That is >>>>>>>>>> specific to the >>>>>>>>>> dovecot config file. In PHPMyAdmin you should remove the >>>>>>>>>> line-continuation >>>>>>>>>> backslashes. >>>>>>>>>> >>>>>>>>>> Actually if you use the mysql command-line client, you would be >>>>>>>>>> able to >>>>>>>>>> paste that in with the backlashes. >>>>>>>>>> >>>>>>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>>>>>> >>>>>>> - -- Steffen Kaiser >>>>>>> -----BEGIN PGP SIGNATURE----- >>>>>>> Version: GnuPG v1 >>>>>>> >>>>>>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>>>>>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>>>>>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>>>>>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>>>>>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>>>>>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>>>>>> =sXel >>>>>>> -----END PGP SIGNATURE----- > From cajeptha at gmail.com Tue May 3 17:30:03 2016 From: cajeptha at gmail.com (Carl Jeptha) Date: Tue, 03 May 2016 19:30:03 +0200 Subject: Changing Password Schemes In-Reply-To: <5728D2C9.6000109@gedalya.net> References: <57233028.7000201@gmail.com> <57235C97.3060008@gmail.com> <5723B3F3.2040307@KnoxvilleChristian.org> <5723EAED.2060007@gedalya.net> <5724673B.80108@gmail.com> <20160430085842.Horde.sQtUxgU5ss6qUhV_CpiayCe@mail.patrickdk.com> <5724FEDD.1070705@gmail.com> <572555B9.9030606@gedalya.net> <57262362.8060700@gedalya.net> <572742EE.7080001@gedalya.net> <5728CAB0.2080404@gmail.com> <5728CD72.4070904@gedalya.net> <5728D1E9.4080100@gmail.com> <5728D2C9.6000109@gedalya.net> Message-ID: <5728E01B.90802@gmail.com> So to Close off this, may I summarize what I did. Changed my password_query in dovecot-sql.conf.ext to: password_query = \ SELECT username AS USER, \ IF(password IS NULL OR password='', CONCAT('{PLAIN}',clearpwd), PASSWORD) AS password, \ '/var/vmail/%d/%n' as userdb_home, \ 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid \ FROM mailbox \ WHERE username = '%u' AND active = '1' This allowed HASHED and un-HASHED passwords to be used concurrently. I then ran the following in PhpMyAdmin, which populated all of my clear passwords with a hash into HASHED password column: UPDATE mailbox SET password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password='' I again thank Geldalya for his patience and understanding. ------------ You have a good day now, en mag jou m?re ook so wees, Carl A Jeptha On 2016-05-03 18:33, Gedalya wrote: > Just make sure it says: > > WHERE password IS NULL OR password=''; > > With no space between the quote marks, this way it matches an empty string > > > On 05/03/2016 12:29 PM, Carl Jeptha wrote: >> Thank you, >> Due to changes I had to make to let password_query work, I think your "quick" version should be like this my setup: >> >> UPDATE mailbox set password = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE password IS NULL OR password=' '; >> >> ------------ >> You have a good day now, en mag jou m?re ook so wees, >> >> Carl A Jeptha >> >> On 2016-05-03 18:10, Gedalya wrote: >>> The script I sent you should do the job of populating your cryptpwd column with a SHA512-CRYPT version of the clearpwd column. >>> The only reason why you would bother with a perl script is to get a better quality salt from /dev/urandom >>> If you don't care so much about the quality of the salt, you can just run this single query. >>> Make a backup of your database first!! >>> >>> UPDATE mailbox set cryptpwd = ENCRYPT(clearpwd, CONCAT('$6$',sha(RAND()))) WHERE cryptpwd IS NULL OR cryptpwd=' '; >>> >>> Here you are using MySQL's RAND() function to generate salt. It will do the minimum job of making the resulting encrypted password not equal to a SHA512 of the password itself, but the salt isn't very random. So the perl script I sent you reads 12 bytes of better quality random data from /dev/urandom and uses that. This means that if your database gets stolen it will be harder to decrypt the passwords. >>> >>> >>> On 05/03/2016 11:58 AM, Carl Jeptha wrote: >>>> Steffen, >>>> If you can point me in the direction as to how to convert a column of clear text passwords to SHA512-CRYPT I will be happy to follow it and close this query, I only came here because I had spent almost two weeks trying to make the dovecot wiki work and thought someone would point out the mistakes I had made. >>>> >>>> But otherwise, I will move on, and not waste anyone's time anymore. >>>> >>>> ------------ >>>> You have a good day now, en mag jou m?re ook so wees, >>>> >>>> >>>> Carl A Jeptha >>>> >>>> On 2016-05-03 07:02, Steffen Kaiser wrote: >>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>> Hash: SHA1 >>>>> >>>>> On Tue, 3 May 2016, Carl Jeptha wrote: >>>>> >>>>>> OK QUERY is WORKING ("password_query" relies on having a field/column >>>>>> "password', hence the addition under WHERE): >>>>>> password_query = \ >>>>>> SELECT username AS USER, \ >>>>>> IF(cryptpwd IS NULL OR cryptpwd=' ', CONCAT('{PLAIN}',clearpwd), >>>>>> cryptpwd) AS PASSWORD, \ >>>>>> '/var/vmail/%d/%n' as userdb_home, \ >>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as >>>>>> userdb_gid \ >>>>>> FROM mailbox \ >>>>>> WHERE username = '%u' AND active = '1' AND cryptpwd = password ('%w') >>>>>> >>>>>> But still no happy dance, we now have a new error: >>>>>> >>>>>> dovecot: imap-login: Disconnected (auth failed, 3 attempts in 15 >>>>>> secs): user=, method=PLAIN, rip=165.255.109.89, >>>>>> lip=10.0.0.12, TLS, session= >>>>> 1st) You should also enable auth debugging. >>>>> >>>>> 2nd) You are poking in the dark with SQL without understanding it, >>>>> >>>>> WHERE ... cryptpwd = password ('%w') >>>>> >>>>> ???? >>>>> >>>>> 3rd) I had the impression that you want to upgrade lower hashed passwords into stronger hashed ones with a specific scheme and that you therefore need to authentificate against two columns, but update the strong hashes from the entered plain text password if missing. >>>>> >>>>> If you already have access to the clear/text passwords, hash them, put the hashes into the database and be fine. No need for different columns and a >>>>> post login script. >>>>> >>>>> Otherwise: Nobody answered this particular question. And I see no evidance, that Dovecot passes an environment variable named PLAIN_PASSWORD along. I've read the Wiki, but I see nothing like that in the code. Did you've verified that the post login script gets the plain password? >>>>> >>>>> If you have hashed passwords, CONCAT('{PLAIN}',clearpwd) is nonsense. >>>>> >>>>>> >>>>>> On Tue, May 3, 2016 at 11:10 AM, Carl Jeptha wrote: >>>>>> >>>>>>> Here is what is in phpmyadmin: >>>>>>> password_query = >>>>>>> SELECT >>>>>>> username as user, >>>>>>> SELECT >>>>>>> IF( >>>>>>> cryptpwd IS NULL >>>>>>> OR cryptpwd = '', >>>>>>> CONCAT('{PLAIN}', clearpwd), >>>>>>> cryptpwd >>>>>>> ) as password, >>>>>>> '/var/vmail/%d/%n' as userdb_home, >>>>>>> 'maildir:/var/vmail/%d/%n' as userdb_mail, >>>>>>> 150 as userdb_uid, >>>>>>> 8 as userdb_gid >>>>>>> FROM >>>>>>> mailbox >>>>>>> WHERE >>>>>>> username = '%u' >>>>>>> AND active = '1' >>>>>>> >>>>>>> and the error now: >>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>> corresponds to your MySQL server version for the right syntax to use near >>>>>>> 'password_query = >>>>>>> SELECT >>>>>>> username as user, >>>>>>> SELECT >>>>>>> IF( >>>>>>> cryptpwd IS NULL >>>>>>> ' at line 1 >>>>>>> >>>>>>> On Mon, May 2, 2016 at 2:07 PM, Gedalya wrote: >>>>>>> >>>>>>>> On 05/02/2016 05:32 AM, Carl Jeptha wrote: >>>>>>>>> May 2 05:26:03 |****** dovecot: auth-worker(3442): Error: >>>>>>>>> sql(user at domain.tld,xxx.xxx.xxx.xxx): Password query must return a >>>>>>>>> field named 'password' >>>>>>>> I'm not sure, maybe it's checking case-sensitive. Your query returns >>>>>>>> PASSWORD. Make it lowercase. >>>>>>>> >>>>>>>>> For testing purposes I put the query in PHPMyAdmin and it complains this >>>>>>>>> (notice it drops "PASSWORD", but shows it in the query: >>>>>>>>> #1064 - You have an error in your SQL syntax; check the manual that >>>>>>>>> corresponds to your MySQL server version for the right syntax to use >>>>>>>> near '\ >>>>>>>>> IF(cryptpwd IS NULL OR cryptpwd='', CONCAT('{PLAIN}',clearpwd), >>>>>>>>> cryptpwd) as ' at line 1 >>>>>>>>> >>>>>>>>> >>>>>>>> It also sarts with a \ ... did you leave that in? That is specific to the >>>>>>>> dovecot config file. In PHPMyAdmin you should remove the line-continuation >>>>>>>> backslashes. >>>>>>>> >>>>>>>> Actually if you use the mysql command-line client, you would be able to >>>>>>>> paste that in with the backlashes. >>>>>>>> >>>>>>>> Make sure to put in a real value in WHERE username = '%u' <<< >>>>>>>> >>>>> - -- Steffen Kaiser >>>>> -----BEGIN PGP SIGNATURE----- >>>>> Version: GnuPG v1 >>>>> >>>>> iQEVAwUBVyiFMXz1H7kL/d9rAQKnRAgAuvDfoovuWo6Pe9K0xOL7P3EDzB2KNdMH >>>>> 8Wdno9O859LH9sBFIn3//WW2oQqgqOPCWfOnkUTG/w+l4yYHkFCeVmJgDoKlWGUd >>>>> +tNlpZjFvrqBKazKlTAaJ/WBiMkyDlT3qJzrIAGMaXZv+0ycUMTN3+ulrUceB4WW >>>>> +Uk5Cvt6LEq9wuqDABje4frIfQc9WVVxI69+z8bHnW6OIq2sL2DXFFRskPbdKFTG >>>>> LTUewcpZTzBKSYLtbFfseBXTCmLy2XPazziamDr9/GWE9yBUR8VhcaTlCp4aI9VG >>>>> 0vB4qCwHF5GNZ6740vYwkVWPFHNYaZW+xZ7v9GCY2mF71A2viCP+QA== >>>>> =sXel >>>>> -----END PGP SIGNATURE----- From aki.tuomi at dovecot.fi Wed May 4 18:33:16 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Wed, 4 May 2016 21:33:16 +0300 (EEST) Subject: more info about deleting users In-Reply-To: <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> Message-ID: <24963661.1711.1462386797729@appsuite-dev.open-xchange.com> > On May 2, 2016 at 9:56 AM Tom Sommer wrote: > > > > On 2016-04-29 16:25, Matteo Gaito Test Account wrote: > > > i've search a lot in order to find a way to delete account by dovecot > > but i > > found only this way > > > > http://dovecot.org/list/dovecot/2011-November/079273.html > > > > It's possible today delete a maildir remotely with doveadm or with > > dovecot api? > > Would love this as well > > // Tom You could try using fs commands that are available in doveadm HTTP API. --- Aki Tuomi From alavarre at gmail.com Thu May 5 00:00:55 2016 From: alavarre at gmail.com (C. Andrews Lavarre) Date: Wed, 04 May 2016 20:00:55 -0400 Subject: Cannot connect to Dovecot IMAP or POP Message-ID: <1462406455.14742.34.camel@gmail.com> Hello all. Thank you for your service. Easy when you know how, but presently I do not. After literally months of research and experimentation we simply cannot log into our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email server with an ordinary email client, e.g., Evolution or Thunderbird. We can connect to the host server in a host of different ways (no pun intended)?http, https, ssh, vnc, telnet, openssl -sclient Similarly we can connect to postfix and dovecot in yet another number of ways?telnet, openssl -sclient?but cannot log in to the email server with a normal email client (either Evolution or Thunderbird) by either pop3 or imap. SSL certificates are in place, verified, and tested. Part of the problem is the many changes in all the involved operating systems and protocols (e.g., imaps and pop3s are deprecated, openSUSE has migrated to LEAP, etc.) so many of the docs from Google are no longer valid. Additionally, there simply are bugs: Leap 42.1 YAST does not work when it comes to setting up websites. Documented. But I digress. I'm sure it's something really simple, but it evades me. Research details below. Any help would be more than appreciated. Thanks in advance, Andy ======================= Configuration testing details ======================= System is: Linux openSUSE Leap 42.1 Dovecot --version 2.2.18, Postfix Version: 2.11.6-3.1 Apache2 Version: 2.4.16-9.1 Connections 1. Evolution or Thunderbird to pop3 or imap reports: The reported error was "Could not connect to mail.privustech.com: Connection refused". Both connect successfully to googlemail.com with the same protocol: Port 993 SSL on a dedicated port I have also tried Port 143 STARTTLS after connecting without success 2. openssl s_client -connect mail.privustech.com:xxx a. xxx=25, 110, 143 all return error:140770FC b. xxx=993, 995 return socket: Connection refused connect:errno=111 3.telnet to a. smtp works. b. pop3 andy at tm2t:~> telnet 70.186.159.22 110 ... +OK POP3 2007e.104 server ready <48fa.572a0769 at privustech.com> ... user andy -ERR Unknown AUTHORIZATION state command c. imap connects but does not allow login, and should not. http://marc.info/?l=imap&m=118775891829506&w=2 The most simple answer is "you cannot TELNET to a modern, correctly-configured, IMAP server and log in to it." andy at tm2t:~> telnet 70.186.159.22 143 ... * OK [...] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 10:26:28 -0400 (EDT) ... A NO Invalid login credentials Modules ? Apache2 works just fine. The server is up and answering. ping works just fine. We have http and https to all vhost sites (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and their www. subsites). ? Postfix reports no errors. We can log in on localhost, send a message to ourselves and see the message. ? Dovecot: a. Logging is enabled in 10-logging.conf to /var/log/dovecot.conf but no logging has occurred there. b. doveconf -n throws no errors. Checks and tests completed 1. /etc/hosts is just fine. 2. Firewall is open for telnet, postfix, dovecot. 3. Added andy to dovecot, postfix groups, in addition to mail, reset password to ANDYbbs14 at . 4. We tried enabling imaps, pop3s, but this command returns errors about these protocols being obsolete. https://tools.ietf.org/html/rfc2595 Use of these ports is discouraged in favor of the STARTTLS or STLS commands. 5. Reviewed doveconf -n: a. Note, there are no Dovecot users established other than user postfix group postfix service auth { unix_listener auth-userdb { group = postfix user = postfix } } i. postfix has its own set of users, including andy, which works just fine within postfix. We can send mail and read mail in the mailbox. b. Authentication is performed by PAM: passdb { driver = pam } i. Examined PAM: A. The files /etc/pam.d/xxx, where xxx = dovecot, pop, imap, are all the same lavarre:~ # cat /etc/pam.d/xxx #%PAM-1.0 auth include common-auth account include common-account password include common-password session include common-session B. They do not resemble at all the form presented in http://wiki2.dovecot.org/PasswordDatabase/PAM passdb { driver = pam args = %s } C. Add (B.) to see if that works: No change. Comment out the original (A.): No change. Restore it. c. SSL is required and apparently configured correctly (the less-than symbol '<'causes the succeeding file to be read into the variable): ssl = required ssl_cert = References: <1462406455.14742.34.camel@gmail.com> Message-ID: <572A9B76.2010704@pettijohn-web.com> Re-read the following: 1st http://wiki2.dovecot.org/PasswordDatabase 2nd http://wiki2.dovecot.org/Authentication/Mechanisms then edit /etc/dovecot/conf.d/10-auth.conf auth_mechanisms = plain login On 05/04/16 19:00, C. Andrews Lavarre wrote: > Hello all. Thank you for your service. > > Easy when you know how, but presently I do not. After literally months of research and experimentation we simply cannot log into our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email server with an ordinary email client, e.g., Evolution or Thunderbird. > > We can connect to the host server in a host of different ways (no pun intended)?http, https, ssh, vnc, telnet, openssl -sclient > > Similarly we can connect to postfix and dovecot in yet another number of ways?telnet, openssl -sclient?but cannot log in to the email server with a normal email client (either Evolution or Thunderbird) by either pop3 or imap. > > SSL certificates are in place, verified, and tested. > > Part of the problem is the many changes in all the involved operating systems and protocols (e.g., imaps and pop3s are deprecated, openSUSE has migrated to LEAP, etc.) so many of the docs from Google are no longer valid. Additionally, there simply are bugs: Leap 42.1 YAST does not work when it comes to setting up websites. Documented. But I digress. > > I'm sure it's something really simple, but it evades me. Research details below. Any help would be more than appreciated. > > Thanks in advance, Andy > > ======================= Configuration testing details ======================= > > System is: > Linux openSUSE Leap 42.1 > Dovecot --version 2.2.18, > Postfix Version: 2.11.6-3.1 > Apache2 Version: 2.4.16-9.1 > > Connections > 1. Evolution or Thunderbird to pop3 or imap reports: > The reported error was "Could not connect to mail.privustech.com: Connection refused". > > Both connect successfully to googlemail.com with the same protocol: > Port 993 SSL on a dedicated port > > I have also tried > Port 143 STARTTLS after connecting > without success > > 2. openssl s_client -connect mail.privustech.com:xxx > a. xxx=25, 110, 143 all return > error:140770FC > > b. xxx=993, 995 return > socket: Connection refused > connect:errno=111 > > 3.telnet to > a. smtp works. > > b. pop3 > andy at tm2t:~> telnet 70.186.159.22 110 > ... > +OK POP3 2007e.104 server ready <48fa.572a0769 at privustech.com> > ... > user andy > -ERR Unknown AUTHORIZATION state command > > c. imap connects but does not allow login, and should not. > http://marc.info/?l=imap&m=118775891829506&w=2 > The most simple answer is "you cannot TELNET to a modern, correctly-configured, > IMAP server and log in to it." > andy at tm2t:~> telnet 70.186.159.22 143 > ... > * OK [...] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 10:26:28 > -0400 (EDT) > ... A NO Invalid login credentials > > Modules > > ? Apache2 works just fine. The server is up and answering. ping works just fine. We have http and https to all vhost sites (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and their www. subsites). > > ? Postfix reports no errors. We can log in on localhost, send a message to ourselves and see the message. > > ? Dovecot: > a. Logging is enabled in 10-logging.conf to /var/log/dovecot.conf but no logging has occurred there. > > b. doveconf -n throws no errors. > > > Checks and tests completed > > 1. /etc/hosts is just fine. > > 2. Firewall is open for telnet, postfix, dovecot. > > 3. Added andy to dovecot, postfix groups, in addition to mail, reset password to ANDYbbs14 at . > > 4. We tried enabling imaps, pop3s, but this command returns errors about these protocols being obsolete. > https://tools.ietf.org/html/rfc2595 > Use of these ports is discouraged in favor of the STARTTLS or STLS > commands. > > 5. Reviewed doveconf -n: > a. Note, there are no Dovecot users established other than > user postfix > group postfix > service auth { > unix_listener auth-userdb { > group = postfix > user = postfix > } > } > > i. postfix has its own set of users, including andy, which works just fine within postfix. > We can send mail and read mail in the mailbox. > > b. Authentication is performed by PAM: > passdb { > driver = pam > } > > i. Examined PAM: > A. The files /etc/pam.d/xxx, where xxx = dovecot, pop, imap, are all the same > lavarre:~ # cat /etc/pam.d/xxx > #%PAM-1.0 > auth include common-auth > account include common-account > password include common-password > session include common-session > B. They do not resemble at all the form presented in > http://wiki2.dovecot.org/PasswordDatabase/PAM > passdb { > driver = pam > args = %s > } > C. Add (B.) to see if that works: No change. > Comment out the original (A.): No change. > Restore it. > > c. SSL is required and apparently configured correctly > (the less-than symbol '<'causes the succeeding file to be read into the variable): > ssl = required > ssl_cert = ssl_dh_parameters_length = 2048 > ssl_key = ssl_options = no_compression > ssl_prefer_server_ciphers = yes > userdb { > driver = passwd > } > > i. dovecot.pem, both cert and key, are installed in /etc/ssl as above and verified as a pair with > openssl x509. > And we point to them in /etc/dovecot/conf.d/10-ssl.conf as seen in the above. > > 6. Checked listening as it does not appear in doveconf -n: > lavarre:~ # doveconf protocols listen > protocols = imap pop3 lmtp > listen = *, :: > > a. conf.d/10-master.conf > ports for service xxx-login {inet_listener} are commented out. > In fact, the entire file is commented out. > > Uncomment the listeners, restart. But no change. So undo. From jtam.home at gmail.com Thu May 5 02:50:55 2016 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 4 May 2016 19:50:55 -0700 (PDT) Subject: Cannot connect to Dovecot IMAP or POP In-Reply-To: References: Message-ID: > 2. openssl s_client -connect mail.privustech.com:xxx > a. xxx=25, 110, 143 all return > error:140770FC You need to add the "-starttls protocol" parameter, otherwise "openssl s_client" thinks your using SSL, not STARTTLS i.e. openssl s_client -starttls pop3 -connect mail.privustech.com:110 openssl s_client -starttls imap -connect mail.privustech.com:143 But even if you do that, you get the same error because it can't start the TLS session: $ nc mail.privustech.com 143 * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS STARTTLS LOGINDISABLED AUTH=CRAM-MD5] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 22:26:36 -0400 (EDT) x STARTTLS x NO Server certificate not installed So there you go, dovecot can't get at your server certificate. You can start the search of your problem from there. The configuration verbose_ssl = yes will help to log the problem. > ssl_dh_parameters_length = 2048 Probably not related to your problem, but this will cause wildly variable dovecot initialization times, as temp key generation of this length can result in a lot of candidate primailty testing. Joseph Tam From dovecot-mailing-list at whyaskwhy.org Thu May 5 05:00:30 2016 From: dovecot-mailing-list at whyaskwhy.org (deoren) Date: Thu, 5 May 2016 00:00:30 -0500 Subject: Is there a way to override Sieve's "not sending notification for auto-submitted message" behavior? Message-ID: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> Goal: 1) Setup a Google Calendar entry for a biweekly task 2) Configure the email notification schedule 3) When the email notification from Google arrives have Sieve send a notification to an alias I have setup for my cell provider's email to text messaging gateway 4) Receive text message I know there are other products which likely handle this better, but I'm specifically attempting to replicate old behavior by getting text message reminders when a specific Google Calendar event occurs. The problem I'm having is that Sieve is attempting to help by NOT sending a notification for emails that it finds are automatically generated. I didn't found a lot of information when I searched for additional details, but I didn't find an earlier message thread on this list that led me to believe that the default behavior is likely chosen as some sort of safety net to prevent common issues from occurring. What I would like to do is override this behavior at some level (per rule, per user, system-wide, whatever) to allow for Sieve notifications when emails matching a specific pattern are detected regardless of whether they are auto-generated or not. I already found mention in the documentation[1] that the editheader extension refuses to remove the Auto-Submitted header, so setting up a per user or global rule to do just that wouldn't help. I also haven't come upon a way to simply modify the value for the Auto-Submitted header, so that doesn't look to work in this situation either. Does anyone know of a way to accomplish this? Thanks in advance for your help! [1] http://wiki2.dovecot.org/Pigeonhole/Sieve/Extensions/Editheader From gedalya at gedalya.net Thu May 5 15:42:10 2016 From: gedalya at gedalya.net (Gedalya) Date: Thu, 5 May 2016 11:42:10 -0400 Subject: Is there a way to override Sieve's "not sending notification for auto-submitted message" behavior? In-Reply-To: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> References: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> Message-ID: <572B69D2.1000203@gedalya.net> On 05/05/2016 01:00 AM, deoren wrote: > Goal: > > 1) Setup a Google Calendar entry for a biweekly task > 2) Configure the email notification schedule > 3) When the email notification from Google arrives have Sieve send a > notification to an alias I have setup for my cell provider's email to > text messaging gateway > 4) Receive text message > > I know there are other products which likely handle this better, but I'm > specifically attempting to replicate old behavior by getting text > message reminders when a specific Google Calendar event occurs. > > The problem I'm having is that Sieve is attempting to help by NOT > sending a notification for emails that it finds are automatically > generated. I didn't found a lot of information when I searched for > additional details, but I didn't find an earlier message thread on this > list that led me to believe that the default behavior is likely chosen > as some sort of safety net to prevent common issues from occurring. > > What I would like to do is override this behavior at some level (per > rule, per user, system-wide, whatever) to allow for Sieve notifications > when emails matching a specific pattern are detected regardless of > whether they are auto-generated or not. > > I already found mention in the documentation[1] that the editheader > extension refuses to remove the Auto-Submitted header, so setting up a > per user or global rule to do just that wouldn't help. I also haven't > come upon a way to simply modify the value for the Auto-Submitted > header, so that doesn't look to work in this situation either. > > Does anyone know of a way to accomplish this? Thanks in advance for your > help! > > [1] http://wiki2.dovecot.org/Pigeonhole/Sieve/Extensions/Editheader If you can't do it with dovecot / pigeonhole then consider doing something in the MTA like removing the Auto-Submitted header before delivery, or of course you can just send your notification out of there. Which MTA are you using? From aki.tuomi at dovecot.fi Thu May 5 16:25:25 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Thu, 5 May 2016 19:25:25 +0300 (EEST) Subject: Cannot connect to Dovecot IMAP or POP In-Reply-To: References: Message-ID: <1119302784.2028.1462465525953@appsuite-dev.open-xchange.com> > ssl_dh_parameters_length = 2048 > > Probably not related to your problem, but this will cause wildly variable > dovecot initialization times, as temp key generation of this length > can result in a lot of candidate primailty testing. > The key generation is done once and stored to disk. Then reused until eternity. --- Aki Tuomi From dovecot-mailing-list at whyaskwhy.org Thu May 5 17:02:43 2016 From: dovecot-mailing-list at whyaskwhy.org (deoren) Date: Thu, 5 May 2016 12:02:43 -0500 Subject: Is there a way to override Sieve's "not sending notification for auto-submitted message" behavior? In-Reply-To: <572B69D2.1000203@gedalya.net> References: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> <572B69D2.1000203@gedalya.net> Message-ID: <4c5f3488-3911-a1d9-5de7-884c8274385b@whyaskwhy.org> On 5/5/2016 10:42 AM, Gedalya wrote: > On 05/05/2016 01:00 AM, deoren wrote: >> Goal: >> >> 1) Setup a Google Calendar entry for a biweekly task >> 2) Configure the email notification schedule >> 3) When the email notification from Google arrives have Sieve send a >> notification to an alias I have setup for my cell provider's email to >> text messaging gateway >> 4) Receive text message >> >> ... >> > If you can't do it with dovecot / pigeonhole then consider doing something in the MTA like removing the Auto-Submitted header before delivery Thank you for taking the time to read my email and offer suggestions! I was starting to think the same thing. I've been thinking about using a local alias to pipe to a script to handle generating my own notifications for Google Calendar emails. I also thought about creating some sort of filter/milter to just strip out the header for those emails before letting the Sieve filter handle the rest, but I've not yet had a chance to research just how to go about that. > or of course you can just send your notification out of there. Like I mentioned above or is there a better way to go about it? > Which MTA are you using? > I'm using Postfix 2.11.x + Dovecot 2.2.x to handle our mail. Thanks again for your help! From gedalya at gedalya.net Thu May 5 17:33:42 2016 From: gedalya at gedalya.net (Gedalya) Date: Thu, 5 May 2016 13:33:42 -0400 Subject: Is there a way to override Sieve's "not sending notification for auto-submitted message" behavior? In-Reply-To: <4c5f3488-3911-a1d9-5de7-884c8274385b@whyaskwhy.org> References: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> <572B69D2.1000203@gedalya.net> <4c5f3488-3911-a1d9-5de7-884c8274385b@whyaskwhy.org> Message-ID: <572B83F6.5020502@gedalya.net> On 05/05/2016 01:02 PM, deoren wrote: > On 5/5/2016 10:42 AM, Gedalya wrote: >> On 05/05/2016 01:00 AM, deoren wrote: >>> Goal: >>> >>> 1) Setup a Google Calendar entry for a biweekly task >>> 2) Configure the email notification schedule >>> 3) When the email notification from Google arrives have Sieve send a >>> notification to an alias I have setup for my cell provider's email to >>> text messaging gateway >>> 4) Receive text message >>> >>> ... >>> >> If you can't do it with dovecot / pigeonhole then consider doing something in the MTA like removing the Auto-Submitted header before delivery > > Thank you for taking the time to read my email and offer suggestions! > > I was starting to think the same thing. I've been thinking about using a local alias to pipe to a script to handle generating my own notifications for Google Calendar emails. I also thought about creating some sort of filter/milter to just strip out the header for those emails before letting the Sieve filter handle the rest, but I've not yet had a chance to research just how to go about that. > >> or of course you can just send your notification out of there. > > Like I mentioned above or is there a better way to go about it? > >> Which MTA are you using? >> > > I'm using Postfix 2.11.x + Dovecot 2.2.x to handle our mail. > > Thanks again for your help! So yea if you're on postfix I don't know of better/other terms to think of this in. In exim, you could send out a notification and/or strip/add/modify headers without any external script or writing any "code" per se, just within exim's config file. Although writing a milter for postfix isn't all that complicated either. Postfix has [ http://www.postfix.org/header_checks.5.html ], you can use that to remove a header (IGNORE), you just might be able to set that up to test for the right conditions *when* to do this, and then proceed to remove the header. Gotta run now so I can't put more thought into it at the moment but do post if you figure it out :D From gedalya at gedalya.net Thu May 5 17:40:30 2016 From: gedalya at gedalya.net (Gedalya) Date: Thu, 5 May 2016 13:40:30 -0400 Subject: Is there a way to override Sieve's "not sending notification for auto-submitted message" behavior? In-Reply-To: <572B83F6.5020502@gedalya.net> References: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> <572B69D2.1000203@gedalya.net> <4c5f3488-3911-a1d9-5de7-884c8274385b@whyaskwhy.org> <572B83F6.5020502@gedalya.net> Message-ID: <572B858E.1030909@gedalya.net> On 05/05/2016 01:33 PM, Gedalya wrote: > you just might be able to set that up to test for the right conditions *when* to do this, and then proceed to remove the header Maybe using PCRE negative lookaheads /^Subject: (?!google-calendar-notification)/ DUNNO /^From: (?!google)/ DUNNO /^Auto-Submitted:/ IGNORE maybe something vaguely like this?? didn't test this anywhere outside of my message compose window From nospam.list at unclassified.de Thu May 5 19:10:14 2016 From: nospam.list at unclassified.de (Yves Goergen) Date: Thu, 5 May 2016 21:10:14 +0200 Subject: Long delays when deleting a message in inbox In-Reply-To: References: <5aa5ebb5-9adf-a5b2-3938-477f717c5cc7@unclassified.de> <5722F9AC.6030706@dovecot.fi> <293030d9-f723-f29b-1a1b-456d43dfe6bf@unclassified.de> Message-ID: Well, they're empty. So whatever is causing this delay, it's silent. Yves Goergen http://unclassified.software ________________________________________ Von: Steffen Kaiser Gesendet: Mo, 2016-05-02 08:39 +0200 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 1 May 2016, Yves Goergen wrote: > Which log would you suggest? There's nothing for these times in mail.log, > messages and syslog. Dovecot logs. > > Yves Goergen > http://unclassified.software > > ________________________________________ > Von: Aki Tuomi > Gesendet: Fr, 2016-04-29 08:05 +0200 > >> > On 28.04.2016 20:54, Yves Goergen wrote: >> Hi, >> >> Since recently, there's a long delay when deleting any message from my >> inbox folder. This affects deleting and moving messages, but only the >> inbox of a single mailbox. The delay is a few seconds. Other >> operations work instantly. >> >> The server is Ubuntu 14.04 running Dovecot version "1:2.2.9-1ubuntu2". >> The filesystem is local ext4. My MUA is Thunderbird on Windows, latest >> version, connecting with IMAP. >> >> What could be the cause for this extreme delay? >> >> Yves Goergen >> http://unclassified.software > > Have you looked at your logs? > > Aki > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVyb2CHz1H7kL/d9rAQJqAgf/f6g4VHhqxm14yGMobQyahQegdSqTTHso l2hfAZoxKfUmF6UliL1c1WZ7uWWOZas6ne9IFk6TJEV80iWCb0JNgE1f5/nUm99J 2PCwcPLp9iN/b1W/hZZ8RvUiAsaCD5Je6ghSZX6PQCUGUPhr0CHrRN/nSOhWA/nE sJ398k+CCXty09bmB+BkUXAerYocbWCweBK1rVVYZeBvGHdQ2C1GgapysE/15hGL MaxIH5bdegsNc5ooiGnF24c/Ro88ZuhklRYnYj+1b19hJKN8pOh+RHs69ls5P9R0 oGiE2Hya0NmyGh0kRmWEP4Uu8pvqCA7JEc2xtOxoQgTtY2UgKkjoWA== =OoL4 -----END PGP SIGNATURE----- From chris at wealdencottage.co.uk Thu May 5 19:40:51 2016 From: chris at wealdencottage.co.uk (Chris Smith) Date: Thu, 5 May 2016 20:40:51 +0100 Subject: Dovecot/Rainloop 2.0.13-1_129.el5 Message-ID: <046A5A34-DFDC-48CC-A249-F9FEDE2B8B0F@wealdencottage.co.uk> I configured Dovecot on our mail server under Centos 5.3 (I think) some while ago now (about 2 years) and, to the best of my knowledge,it had been working correctly all that time. However, one of the team wanted to use webmail while away from base and found that, although he could receive emails OK, he was unable to reply or send fresh emails. When he pressed Send he got a message ?Authentication failed" in a pop-up box. He assured me that sending mail had been working and I was certain that I had tested that aspect when I was setting Dovecot up, at least for local mail. I tried to send email from my Dovecot account and had the same problem. Our MTA is sendmail and that is not configured to authenticate on outgoing mail, The configuration for sendmail hasn?t changed at least since the last update in 16 March 2015 (sendmail 8.13.8-10.el5_11). Dovecot uses php-5.5.5 in this set up. The relevant entry in /var/log/maillog when I attempt to send an email from Dovecot is: May 5 16:34:29 firewall dovecot: auth: Debug: auth client connected (pid=13001) May 5 16:34:29 firewall dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=51861 resp=AGNocmlzADB1dFkwdUcwNDkh May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): lookup service=dovecot May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): #1/1 style=1 msg=Password: May 5 16:34:29 firewall dovecot: auth: Debug: client out: OK 1 user=chris May 5 16:34:29 firewall dovecot: auth: Debug: master in: REQUEST 2607546369 13001 1 0bcf2e3a108cd9cd18eaff4b7de9c428 May 5 16:34:29 firewall dovecot: auth: Debug: passwd(chris,127.0.0.1): lookup May 5 16:34:29 firewall dovecot: auth: Debug: master out: USER 2607546369 chris system_groups_user=chris uid=514 gid=100 home=/home/chris May 5 16:34:29 firewall dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=13002, secured May 5 16:34:29 firewall dovecot: imap(chris): Disconnected: Logged out bytes=11/334 May 5 16:34:29 firewall sendmail[13003]: u45FYTtd013003: localhost.localdomain [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA The webmail server and our mail server are the same system. The firewall is open for port 143. It is only send from Dovecot that fails authentication, all other outgoing mail is sent correctly. Does anyone have any idea where else I can look for clues as to why there is this behaviour? Suplementary question: what does the ?Sign me? check box on the webmail log in page do? Many thanks. Chris From edgar at pettijohn-web.com Thu May 5 20:03:52 2016 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Thu, 5 May 2016 15:03:52 -0500 Subject: Dovecot/Rainloop 2.0.13-1_129.el5 In-Reply-To: <046A5A34-DFDC-48CC-A249-F9FEDE2B8B0F@wealdencottage.co.uk> References: <046A5A34-DFDC-48CC-A249-F9FEDE2B8B0F@wealdencottage.co.uk> Message-ID: <572BA728.4070307@pettijohn-web.com> On 05/05/16 14:40, Chris Smith wrote: > I configured Dovecot on our mail server under Centos 5.3 (I think) some while ago now (about 2 years) and, to the best of my knowledge,it had been working correctly all that time. > > However, one of the team wanted to use webmail while away from base and found that, although he could receive emails OK, he was unable to reply or send fresh emails. When he pressed Send he got a message ?Authentication failed" in a pop-up box. > > He assured me that sending mail had been working and I was certain that I had tested that aspect when I was setting Dovecot up, at least for local mail. > > I tried to send email from my Dovecot account and had the same problem. > > Our MTA is sendmail and that is not configured to authenticate on outgoing mail, The configuration for sendmail hasn?t changed at least since the last update in 16 March 2015 (sendmail 8.13.8-10.el5_11). > > Dovecot uses php-5.5.5 in this set up. > > The relevant entry in /var/log/maillog when I attempt to send an email from Dovecot is: > > May 5 16:34:29 firewall dovecot: auth: Debug: auth client connected (pid=13001) > May 5 16:34:29 firewall dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=51861 resp=AGNocmlzADB1dFkwdUcwNDkh > May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): lookup service=dovecot > May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): #1/1 style=1 msg=Password: > May 5 16:34:29 firewall dovecot: auth: Debug: client out: OK 1 user=chris > May 5 16:34:29 firewall dovecot: auth: Debug: master in: REQUEST 2607546369 13001 1 0bcf2e3a108cd9cd18eaff4b7de9c428 > May 5 16:34:29 firewall dovecot: auth: Debug: passwd(chris,127.0.0.1): lookup > May 5 16:34:29 firewall dovecot: auth: Debug: master out: USER 2607546369 chris system_groups_user=chris uid=514 gid=100 home=/home/chris > May 5 16:34:29 firewall dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=13002, secured > May 5 16:34:29 firewall dovecot: imap(chris): Disconnected: Logged out bytes=11/334 > May 5 16:34:29 firewall sendmail[13003]: u45FYTtd013003: localhost.localdomain [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA Looks like a rainloop issue. Apparently it isn't speaking to sendmail correctly. > > > The webmail server and our mail server are the same system. The firewall is open for port 143. > > It is only send from Dovecot that fails authentication, all other outgoing mail is sent correctly. > > Does anyone have any idea where else I can look for clues as to why there is this behaviour? > > Suplementary question: what does the ?Sign me? check box on the webmail log in page do? > > Many thanks. > > Chris From alavarre at gmail.com Thu May 5 21:50:37 2016 From: alavarre at gmail.com (C. Andrews Lavarre) Date: Thu, 05 May 2016 17:50:37 -0400 Subject: Cannot connect to Dovecot IMAP or POP In-Reply-To: <572A9B76.2010704@pettijohn-web.com> References: <1462406455.14742.34.camel@gmail.com> <572A9B76.2010704@pettijohn-web.com> Message-ID: <1462485037.2356.30.camel@gmail.com> Hello all, thank you again for your help. Thanks to Edgar Pettijohn's inspiration, we changed /etc/dovecot/conf.d /10-auth.conf to include login (which did not work) and cram-md5 (whichdid work): auth_mechanisms = plain login cram-md5 and we no longer get Connection refused. Although it doesn't say so explicitly, my reading of http://wiki2.dovecot.org/Authentication/Mechanisms is that SSL/TLS puts a wrapper around plaintext passwords, so you don't need an encrypted password. However, obviously, you need a scheme to first decrypt the TLS envelope! So does cram-md5 do that? Seems to work. Thank you. So now, as Joseph Tam points out, (thank you for the exposure to nc?cool) we are back to "Server certificate not installed". But "the certificate" is installed AFAICT on mail.privustech.com and dovecot: So which server? The choices are ? The root server: 70.186.159.22 ? The virtual host mail server: mail.privustech.com ? The dovecot server: /etc/dovecot/dovecot.conf ? Something else. Presumably, as Joseph shows with his nc call, imap calls are to ServerName mail.privustech.com. So we need it to exist and we need cert files for that ServerName: ? We can connect, so the server exists and is responding. ? It is configured as a virtual host and has its own Apache2 configuration files mail.privustech.com.conf mail.privustech.com-ssl.conf These in turn specify SSL cert, key, and CA files with the CN mail.privustech.com This host is specified as a port 443 vhost, but changing to 143 had no effect. I can also connect with https, so the cert is valid. So I cannot imagine how better to "install" it to a valid host with a valid cert... ??? :-( I examined the other possible "servers" and they all seem correctly established as well. Details of today's angst appended below. Thanks again for the help and inspiration. Tomorrow is another day. Best regards, Andy ========================================== 1. The root server is 70.186.159.22 It is configured in /etc/apache2/default-server.conf This file specifies ServerName as 70.186.159.22 The root server under Apache2 does not have an SSL.conf file, however the root server also is installed as a virtual host in /etc/apache2/vhosts.d through /etc/apache2/vhosts.d/70.186.159.22.conf /etc/apache2/vhosts.d/70.186.159.22-ssl.conf The latter file specifies three SSL files: SSLCertificateFile /etc/apache2/ssl.crt/mail.privustech.com_start.crt SSLCertificateKeyFile /etc/apache2/ssl.key/mailprivustech.key SSLCertificateChainFile /etc/apache2/ssl.crt/mailprivustech_root_bundle.crt Of course, the Common Name (CN) in these files does not match the root ServerName. If dovecot connects from the root server rather than mail.privustech.com that would explain the matter. We'll check that out tomorrow. 2. We are not, however, trying to connect to the root server, rather to mail.privustech.com This virtual host is manifested in Apache2 through /etc/apache2/vhosts.d/mail.privustech.com.conf /etc/apache2/vhosts.d/mail.privustech.com-ssl.conf The ServerName does match the CN in this case. The port number in the vhost is 443 vice 143, but we changed that with no effect. So it does not make sense that an imap connection responds with "Server certificate not installed" How more to "install" the cert than to specify it in the vhost -ssl.conf file? The mail server vhost StartSSL certificate is /etc/apache2/ssl.crt/mail.privustech.com_start.crt and has been validated against its key. Its CN is mail.privustech.com. 3. The dovecot server SSL certificate is specified in the configuration file: /etc/dovecot/dovecot.conf It does not specify a key, however it includes all files in /etc/dovecot/conf.d This contains a number of files, including 10-auth.conf 10-ssl.conf The first includes auth-mechanisms plain login cram-md5 Adding cram-md5 today resolved the "Connection Refused" issue. Although it doesn't say so explicitly, my reading of http://wiki2.dovecot.org/Authentication/Mechanisms is that SSL/TLS puts a wrapper around plaintext passwords, so you don't need an encrypted database. However, obviously, you need a scheme to first decrypt the TLS envelope! So does cram-md5 do that? Seems to work. Thank you. Default settings are included but commented out. In particular, plaintext is by default disabled. So we uncomment and explicitly declare disable_plaintext_auth = no Restart: No change. Restore. /etc/dovecot/conf.d/10-ssl.conf contains explicit referral to the mail.privustech.com SSL files discussed above: ssl = required ssl_cert = Re-read the following: > > 1st > http://wiki2.dovecot.org/PasswordDatabase > > 2nd > http://wiki2.dovecot.org/Authentication/Mechanisms > > then edit /etc/dovecot/conf.d/10-auth.conf > auth_mechanisms = plain login > > On 05/04/16 19:00, C. Andrews Lavarre wrote: > > Hello all. Thank you for your service. > > > > Easy when you know how, but presently I do not. After literally > > months of research and experimentation we simply cannot log into > > our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email > > server with an ordinary email client, e.g., Evolution or > > Thunderbird. > > > > We can connect to the host server in a host of different ways (no > > pun intended)?http, https, ssh, vnc, telnet, openssl -sclient > > > > Similarly we can connect to postfix and dovecot in yet another > > number of ways?telnet, openssl -sclient?but cannot log in to the > > email server with a normal email client (either Evolution or > > Thunderbird) by either pop3 or imap. > > > > SSL certificates are in place, verified, and tested. > > > > Part of the problem is the many changes in all the involved > > operating systems and protocols (e.g., imaps and pop3s are > > deprecated, openSUSE has migrated to LEAP, etc.) so many of the > > docs from Google are no longer valid. Additionally, there simply > > are bugs: Leap 42.1 YAST does not work when it comes to setting up > > websites. Documented. But I digress. > > > > I'm sure it's something really simple, but it evades me. Research > > details below. Any help would be more than appreciated. > > > > Thanks in advance, Andy > > > > ======================= Configuration testing details > > ======================= > > > > System is: > > > > Linux openSUSE Leap 42.1 > > > > > > Dovecot --version 2.2.18, > > > > > > Postfix Version: 2.11.6-3.1 > > > > > > Apache2 Version: 2.4.16-9.1 > > > > Connections > > > > 1. Evolution or Thunderbird to pop3 or imap reports: > > > > > > The reported error was "Could not connect to > > mail.privustech.com: Connection refused". > > > > > > > > > > Both connect successfully to googlemail.com with the > > same protocol: > > > > > > > > Port 993 SSL on a dedicated port > > > > > > > > > > I have also tried > > > > > > > > > > Port 143 STARTTLS after connecting > > > > > > > > > > without success > > > > > > > > > > 2. openssl s_client -connect mail.privustech.com:xxx > > > > > > > > a. xxx=25, 110, 143 all return > > > > > > > > > > error:140770FC > > > > > > > > > > b. xxx=993, 995 return > > > > > > > > > > socket: Connection refused > > > > > > > > connect:errno=111 > > > > > > > > > > > > 3.telnet to > > > > > > a. smtp works. > > > > > > > > > > b. pop3 > > > > > > andy at tm2t:~> telnet 70.186.159.22 110 > > > > > > > > ... > > > > > > > > +OK POP3 2007e.104 server ready < > > 48fa.572a0769 at privustech.com> > > > > > > > > ... > > > > > > > > user andy > > > > > > > > -ERR Unknown AUTHORIZATION state command > > > > > > > > c. > > > > imap connects but does not allow login, and > > should not. > > > > > > > > > > > > http://marc.info/?l=imap&m=118775891829506&w=2 > > > > > > > > > > > > > > The most simple answer > > is "you cannot TELNET to a modern, correctly-configured, > > > > > > > > > > > > > > IMAP server and log in > > to it." > > > > > > andy at tm2t:~> telnet 70.186.159.22 143 > > > > > > > > ... > > > > > > > > * OK [...] privustech.com IMAP4rev1 2007e.404 > > at Wed, 4 May 2016 10:26:28 > > > > > > > > -0400 (EDT) > > > > > > > > ... A NO Invalid login credentials > > > > > > > > > > Modules > > > > > > ? Apache2 works just fine. The server is up and answering. ping > > works just fine. We have http and https to all vhost sites > > (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and > > their www. subsites). > > > > > > ? Postfix reports no errors. We can log in on localhost, send a > > message to ourselves and see the message. > > > > ? Dovecot: > > > > > > a. Logging is enabled in 10-logging.conf to > > /var/log/dovecot.conf but no logging has occurred there. > > > > > > > > > > b. doveconf -n throws no errors. > > > > > > > > > > Checks and tests completed > > > > > > 1. /etc/hosts is just fine. > > > > > > > > 2. Firewall is open for telnet, postfix, dovecot. > > > > > > 3. Added andy to dovecot, postfix groups, in addition to mail, > > reset password to ANDYbbs14 at . > > > > > > 4. We tried enabling imaps, pop3s, but this command returns > > errors about these protocols being obsolete. > > > > > > > > https://tools.ietf.org/html/rfc2595 > > > > > > > > Use of these ports is discouraged in favor of > > the STARTTLS or STLS > > > > commands. > > > > > > > > 5. Reviewed doveconf -n: > > > > > > > > a. Note, there are no Dovecot users established > > other than > > > > > > > > user postfix > > > > > > > > group postfix > > > > > > > > > > service auth { > > > > > > > > > > unix_listener auth-userdb { > > > > > > > > > > > > group = postfix > > > > > > > > > > > > user = postfix > > > > > > > > > > } > > > > > > > > > > } > > > > > > > > > > > > > > i. postfix has its own set of users, > > including andy, which works just fine within postfix. > > > > > > > > > > We can send mail and read mail in the > > mailbox. > > > > > > > > > > b. Authentication is performed by PAM: > > > > > > > > passdb { > > > > > > > > driver = pam > > > > > > > > } > > > > > > > > > > i. Examined PAM: > > > > > > > > > > A. The files /etc/pam.d/xxx, where xxx > > = dovecot, pop, imap, are all the same > > > > > > > > > > > > lavarre:~ # cat /etc/pam.d/xxx > > > > > > > > > > > > #%PAM-1.0 > > > > > > > > > > > > auth include common > > -auth > > > > > > > > > > > > account include common > > -account > > > > > > > > > > > > password include common > > -password > > > > > > > > > > > > session include common > > -session > > > > > > > > > > B. They do not resemble at all the form > > presented in > > > > > > > > > > > > http://wiki2.dovecot.org/PasswordDatabase/PAM > > > > > > > > > > > > > > passdb { > > > > > > > > > > > > > > driver = pam > > > > > > > > > > > > > > args = %s > > > > > > > > > > > > > > } > > > > > > > > > > C. Add (B.) to see if that works: No > > change. > > > > > > > > > > Comment out the original (A.): No > > change. > > > > > > > > > > Restore it. > > > > > > > > > > > > > > c. SSL is required and apparently configured correctly > > > > > > (the less-than symbol '<'causes the succeeding file to > > be read into the variable): > > > > > > > > ssl = required > > > > > > > > ssl_cert = > > > > > > > > ssl_dh_parameters_length = 2048 > > > > > > > > ssl_key = > > > > > > > > ssl_options = no_compression > > > > > > > > ssl_prefer_server_ciphers = yes > > > > > > > > userdb { > > > > > > > > driver = passwd > > > > > > > > } > > > > > > > > > > > > > > > > i. dovecot.pem, both cert and key, are > > installed in /etc/ssl as above and verified as a pair with > > > > > > > > > > openssl x509. > > > > > > > > > > > > And we point to them in > > /etc/dovecot/conf.d/10-ssl.conf as seen in the above. > > > > > > 6. Checked listening as it does not appear in doveconf -n: > > > > > > lavarre:~ # doveconf protocols listen > > > > > > protocols = imap pop3 lmtp > > > > > > listen = *, :: > > > > > > > > a. conf.d/10-master.conf > > > > > > > > ports for service xxx-login {inet_listener} are > > commented out. > > > > > > > > In fact, the entire file is commented out. > > > > > > > > > > Uncomment the listeners, restart. But no > > change. So undo. From marcec at gmx.de Thu May 5 13:21:49 2016 From: marcec at gmx.de (Marc Joliet) Date: Thu, 05 May 2016 15:21:49 +0200 Subject: pread() failures when using mdbox on btrfs Message-ID: <3209263.oXmjp69zg6@thetick> (Note: I am not subscribed to this ML, so please be sure to CC me in replies.) Hello, I see a strange bug when running dovecot under btrfs with a mailbox in mdbox format. At some point, which can take a few minutes or more than a day, dovecot starts logging pread() failures, like so: Apr 02 23:43:42 thetick dovecot[570]: imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/LinuxAudio/User/dbox- Mails/dovecot.index.log: Input/output error These would trigger when getmail runs (which in turn is configured to use dovecot-lda). It's only ever the *.index.log files, and it would usually happen to the same one many times in a row. To demonstrate, this is the full list of unique pread() failures on one host (my desktop PC): # journalctl -u dovecot -p 3 -o cat | sort -u | grep pread imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/BTRFS/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/Gentoo/User/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/LinuxAudio/User/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/Newsletters/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/SCons/User/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/storage/dovecot.map.index.log: Input/output error and on the current host (a home server): # journalctl -u dovecot -p 3 -o cat | sort -u | grep pread imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/mailboxes/Privat/dbox-Mails/dovecot.index.log: Input/output error imap(marcec): Error: pread() failed with file /home/marcec/.mdbox/storage/dovecot.map.index.log: Input/output error Rebooting the system would make it go away (without loss of mails), though remounting the file system should have worked, too. I tried a variety of options, but the only one that helped was to convert the mailbox from mdbox to maildir. I have not seen any pread() failures in the two weeks since. I originally wrote to the btrfs ML, see the thread at [0], in particular my messages [1] and [2], for additional details, including references to what little I could find on the web. However, nothing much came of that thread, so I am now posting here. My dovecot configuration: # doveconf -n # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 4.4.8-gentoo x86_64 Gentoo Base System release 2.2 auth_mechanisms = plain login mail_location = maildir:~/.maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = * driver = pam } plugin { sieve = file:~/.sieve;active=~/.dovecot.sieve } postmaster_address = root at diefledermaus protocols = imap sieve service imap-login { inet_listener imap { port = 10087 } inet_listener imaps { port = 10887 } } ssl = required ssl_cert = From wealden_smith at btinternet.com Thu May 5 22:20:54 2016 From: wealden_smith at btinternet.com (Chris Smith) Date: Thu, 5 May 2016 23:20:54 +0100 Subject: Dovecot/Rainloop 2.0.13-1_129.el5 In-Reply-To: <572BA728.4070307@pettijohn-web.com> References: <046A5A34-DFDC-48CC-A249-F9FEDE2B8B0F@wealdencottage.co.uk> <572BA728.4070307@pettijohn-web.com> Message-ID: > On 5 May 2016, at 21:03, Edgar Pettijohn wrote: > > > > On 05/05/16 14:40, Chris Smith wrote: >> I configured Dovecot on our mail server under Centos 5.3 (I think) some while ago now (about 2 years) and, to the best of my knowledge,it had been working correctly all that time. >> >> However, one of the team wanted to use webmail while away from base and found that, although he could receive emails OK, he was unable to reply or send fresh emails. When he pressed Send he got a message ?Authentication failed" in a pop-up box. >> >> He assured me that sending mail had been working and I was certain that I had tested that aspect when I was setting Dovecot up, at least for local mail. >> >> I tried to send email from my Dovecot account and had the same problem. >> >> Our MTA is sendmail and that is not configured to authenticate on outgoing mail, The configuration for sendmail hasn?t changed at least since the last update in 16 March 2015 (sendmail 8.13.8-10.el5_11). >> >> Dovecot uses php-5.5.5 in this set up. >> >> The relevant entry in /var/log/maillog when I attempt to send an email from Dovecot is: >> >> May 5 16:34:29 firewall dovecot: auth: Debug: auth client connected (pid=13001) >> May 5 16:34:29 firewall dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=51861 resp=AGNocmlzADB1dFkwdUcwNDkh >> May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): lookup service=dovecot >> May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): #1/1 style=1 msg=Password: >> May 5 16:34:29 firewall dovecot: auth: Debug: client out: OK 1 user=chris >> May 5 16:34:29 firewall dovecot: auth: Debug: master in: REQUEST 2607546369 13001 1 0bcf2e3a108cd9cd18eaff4b7de9c428 >> May 5 16:34:29 firewall dovecot: auth: Debug: passwd(chris,127.0.0.1): lookup >> May 5 16:34:29 firewall dovecot: auth: Debug: master out: USER 2607546369 chris system_groups_user=chris uid=514 gid=100 home=/home/chris >> May 5 16:34:29 firewall dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=13002, secured >> May 5 16:34:29 firewall dovecot: imap(chris): Disconnected: Logged out bytes=11/334 >> May 5 16:34:29 firewall sendmail[13003]: u45FYTtd013003: localhost.localdomain [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA > Looks like a rainloop issue. Apparently it isn't speaking to sendmail correctly. You are quite correct. I had misconfigured the domain to require authentication. All is working now. Many thanks > > >> >> >> The webmail server and our mail server are the same system. The firewall is open for port 143. >> >> It is only send from Dovecot that fails authentication, all other outgoing mail is sent correctly. >> >> Does anyone have any idea where else I can look for clues as to why there is this behaviour? >> >> Suplementary question: what does the ?Sign me? check box on the webmail log in page do? >> >> Many thanks. >> >> Chris From tss at iki.fi Thu May 5 22:51:34 2016 From: tss at iki.fi (Timo Sirainen) Date: Fri, 6 May 2016 01:51:34 +0300 Subject: pread() failures when using mdbox on btrfs In-Reply-To: <3209263.oXmjp69zg6@thetick> References: <3209263.oXmjp69zg6@thetick> Message-ID: <52EF3DAD-8747-4B3D-8219-ED9DFE20B9DF@iki.fi> On 05 May 2016, at 16:21, Marc Joliet wrote: > > (Note: I am not subscribed to this ML, so please be sure to CC me in replies.) > > Hello, > > I see a strange bug when running dovecot under btrfs with a mailbox in mdbox > format. At some point, which can take a few minutes or more than a day, > dovecot starts logging pread() failures, like so: > > Apr 02 23:43:42 thetick dovecot[570]: imap(marcec): Error: pread() failed with > file /home/marcec/.mdbox/mailboxes/LinuxAudio/User/dbox- > Mails/dovecot.index.log: Input/output error > > These would trigger when getmail runs (which in turn is configured to use > dovecot-lda). It's only ever the *.index.log files, and it would usually > happen to the same one many times in a row. To demonstrate, this is the full > list of unique pread() failures on one host (my desktop PC): *.index.log files are always appended to using O_APPEND flag. Maybe this is relevant. Also when a new .log file is created it's opened without the O_APPEND flag and the O_APPEND is added later. This was causing a bug recently in unionfs, which ignored the flag change and caused log file corruption. > Rebooting the system would make it go away (without loss of mails), though > remounting the file system should have worked, too. I tried a variety of > options, but the only one that helped was to convert the mailbox from mdbox to > maildir. I have not seen any pread() failures in the two weeks since. I can't really think of why Maildir format would get rid of the problem, since the dovecot.index.log files still exist and they're accessed the same way. Maybe with Maildir there are enough locks that it prevents some race condition that triggers this. You could try stress testing with imaptest to see if it's easy to reproduce: http://imapwiki.org/ImapTest From jtam.home at gmail.com Fri May 6 00:17:41 2016 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 5 May 2016 17:17:41 -0700 (PDT) Subject: Cannot connect to Dovecot IMAP or POP In-Reply-To: References: Message-ID: C. Andrews Lavarre writes: > Presumably, as Joseph shows with his nc call, imap calls are to ServerName mail.privustech.com. No, nc is just a dumb tool -- it knows nothing about IMAP or SSL. You didn't even get to the SSL handshake. Your problem has nothing to do with the interaction of client and server -- it's strictly a server problem. > ssl = required > ssl_cert = ssl_key = ssl_ca = From luckyfellow42 at gmail.com Fri May 6 10:14:05 2016 From: luckyfellow42 at gmail.com (Andreas Meyer) Date: Fri, 6 May 2016 12:14:05 +0200 Subject: Dovecot + libsodium Message-ID: Hi, Thank you very much for creating and maintaining dovecot! In my scenario, I want to use the password hash algorithms provided by libsodium: https://download.libsodium.org/doc/ So my difficulty is to have dovecot support libsodium's hash algorithms, particularly: crypto_pwhash_scryptsalsa208sha256_str On the sodium maillinglist I asked for help and received an adjusted dovecot code, which exactly does what I need. You find it here: https://github.com/jedisct1/core/tree/scrypt-argon2 Obviously I need to apply these changes everytime I upgrade to a new dovecot version now. So my question ist, what do I need to do so that you will include libsodium support in future versions of dovecot? Thank you very much for your attention, Andreas From tss at iki.fi Fri May 6 21:15:01 2016 From: tss at iki.fi (Timo Sirainen) Date: Sat, 7 May 2016 00:15:01 +0300 Subject: Dovecot + libsodium In-Reply-To: References: Message-ID: On 06 May 2016, at 13:14, Andreas Meyer wrote: > > Hi, > > Thank you very much for creating and maintaining dovecot! > > In my scenario, I want to use the password hash algorithms provided by > libsodium: https://download.libsodium.org/doc/ > > So my difficulty is to have dovecot support libsodium's hash algorithms, > particularly: crypto_pwhash_scryptsalsa208sha256_str > > On the sodium maillinglist I asked for help and received an adjusted > dovecot code, which exactly does what I need. You find it here: > https://github.com/jedisct1/core/tree/scrypt-argon2 > > Obviously I need to apply these changes everytime I upgrade to a new > dovecot version now. > > So my question ist, what do I need to do so that you will include libsodium > support in future versions of dovecot? You could also change it to be a plugin to avoid patching. This is a pretty old example, but it probably still works, at least with minor changes: http://dovecot.org/patches/password-scheme-lmpass.c Although it's still a good idea to recompile the plugin after a new version since sometimes the ABI changes. From dovecot-mailing-list at whyaskwhy.org Sat May 7 07:07:24 2016 From: dovecot-mailing-list at whyaskwhy.org (deoren) Date: Sat, 7 May 2016 02:07:24 -0500 Subject: Is there a way to override Sieve's "not sending notification for auto-submitted message" behavior? In-Reply-To: <572B858E.1030909@gedalya.net> References: <440b6540-08d5-4d11-49c5-7cb6ecfa4a43@whyaskwhy.org> <572B69D2.1000203@gedalya.net> <4c5f3488-3911-a1d9-5de7-884c8274385b@whyaskwhy.org> <572B83F6.5020502@gedalya.net> <572B858E.1030909@gedalya.net> Message-ID: <754d45a6-cab3-28a6-6b2d-eabaa3b96c96@whyaskwhy.org> On 5/5/2016 12:40 PM, Gedalya wrote: > On 05/05/2016 01:33 PM, Gedalya wrote: >> you just might be able to set that up to test for the right conditions *when* to do this, and then proceed to remove the header > > Maybe using PCRE negative lookaheads > > /^Subject: (?!google-calendar-notification)/ DUNNO > /^From: (?!google)/ DUNNO > /^Auto-Submitted:/ IGNORE > > maybe something vaguely like this?? didn't test this anywhere outside of my message compose window > Thanks. I tried it, but it appeared to strip the header from emails that the "/From: " line didn't match. I'm going to try going the route of a check_sender_access table entry that uses a 'FILTER transport:' action to send matched mail through a custom transport. That transport/service entry in master.cf will then apply a single header check to strip out the header. I've been fighting with the appropriate settings for the entry and haven't made much progress, so I will probably drop into the Postfix mailing list soon and ask for some help there pointing out my obvious mistake. Thanks for your feedback. From marcec at gmx.de Sat May 7 11:44:31 2016 From: marcec at gmx.de (Marc Joliet) Date: Sat, 07 May 2016 13:44:31 +0200 Subject: pread() failures when using mdbox on btrfs In-Reply-To: <52EF3DAD-8747-4B3D-8219-ED9DFE20B9DF@iki.fi> References: <3209263.oXmjp69zg6@thetick> <52EF3DAD-8747-4B3D-8219-ED9DFE20B9DF@iki.fi> Message-ID: <6621659.dbnSlKmncB@thetick> On Friday 06 May 2016 01:51:34 Timo Sirainen wrote: >On 05 May 2016, at 16:21, Marc Joliet wrote: >> (Note: I am not subscribed to this ML, so please be sure to CC me in >> replies.) >> >> Hello, >> >> I see a strange bug when running dovecot under btrfs with a mailbox in >> mdbox >> format. At some point, which can take a few minutes or more than a day, >> dovecot starts logging pread() failures, like so: >> >> Apr 02 23:43:42 thetick dovecot[570]: imap(marcec): Error: pread() failed >> with file /home/marcec/.mdbox/mailboxes/LinuxAudio/User/dbox- >> Mails/dovecot.index.log: Input/output error >> >> These would trigger when getmail runs (which in turn is configured to use >> dovecot-lda). It's only ever the *.index.log files, and it would usually >> happen to the same one many times in a row. To demonstrate, this is the >> full >> list of unique pread() failures on one host (my desktop PC): >*.index.log files are always appended to using O_APPEND flag. Maybe this is >relevant. > >Also when a new .log file is created it's opened without the O_APPEND flag >and the O_APPEND is added later. This was causing a bug recently in unionfs, >which ignored the flag change and caused log file corruption. I will mention that in the btrfs ML thread. Who knows, maybe it will lead to something. >> Rebooting the system would make it go away (without loss of mails), though >> remounting the file system should have worked, too. I tried a variety of >> options, but the only one that helped was to convert the mailbox from mdbox >> to maildir. I have not seen any pread() failures in the two weeks since. >I can't really think of why Maildir format would get rid of the problem, >since the dovecot.index.log files still exist and they're accessed the same >way. Maybe with Maildir there are enough locks that it prevents some race >condition that triggers this. > >You could try stress testing with imaptest to see if it's easy to reproduce: >http://imapwiki.org/ImapTest I will experiment with that when I have some more time. Thanks for the suggestions! Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From daniel.colchete at gmail.com Sat May 7 23:27:11 2016 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Sat, 7 May 2016 20:27:11 -0300 Subject: Panic: file doveadm-mail.c: line 417 (doveadm_mail_single_user): assertion failed: (cctx->username != NULL) Message-ID: Hello everyone! So, I'm trying to move away from a few older servers into new ones but I can't get dsync to work this time (the 2013 migration went perfectly). This is what I get: root at dovecot:/# dsync sync -u user at domain.org.br tcp:XXXX:12345 Panic: file doveadm-mail.c: line 417 (doveadm_mail_single_user): assertion failed: (cctx->username != NULL) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x85dee) [0x7f500d3cddee] -> /usr/lib/dovecot/libdovecot.so.0(+0x85e68) [0x7f500d3cde68] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f500d3725be] -> dsync(+0x28d50) [0x55dcb11bad50] -> dsync(+0x28ded) [0x55dcb11baded] -> dsync(doveadm_mail_try_run+0x219) [0x55dcb11bb779] -> dsync(main+0x441) [0x55dcb11aba91] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f500cfbeb45] -> dsync(+0x19c06) [0x55dcb11abc06] Aborted (core dumped) root at dovecot:/# doveconf -N is here: http://pastebin.ca/3594094 doveadm user '*" will list all the users correctly doveadm user user at domain.org.br will show all the user info correctly. Thank you very much! Best, Daniel Colchete From voytek at sbt.net.au Mon May 9 06:22:42 2016 From: voytek at sbt.net.au (voytek at sbt.net.au) Date: Mon, 9 May 2016 16:22:42 +1000 Subject: resorting emails by date order after move/copy to sub folder ? Message-ID: I have just used Squirrel web client to re organize some old emails to subfolders. that worked OK, EXCEPT, all these emails are now in order that I have moved them rather than original date, is that some 'simple' way to re-sort them back by original date ? thanks for any pointers, sorry for dumb question Voytek From mail at tomsommer.dk Mon May 9 07:01:30 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Mon, 09 May 2016 09:01:30 +0200 Subject: more info about deleting users In-Reply-To: <24963661.1711.1462386797729@appsuite-dev.open-xchange.com> References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> <24963661.1711.1462386797729@appsuite-dev.open-xchange.com> Message-ID: <976ded279ff32f605cc4906326cd8246@tomsommer.dk> On 2016-05-04 20:33, aki.tuomi at dovecot.fi wrote: >> On May 2, 2016 at 9:56 AM Tom Sommer wrote: >> On 2016-04-29 16:25, Matteo Gaito Test Account wrote: >> >> > i've search a lot in order to find a way to delete account by dovecot >> > but i >> > found only this way >> > >> > http://dovecot.org/list/dovecot/2011-November/079273.html >> > >> > It's possible today delete a maildir remotely with doveadm or with >> > dovecot api? >> >> Would love this as well > > > You could try using fs commands that are available in doveadm HTTP API. Looks doable. Any documentation? :) From aki.tuomi at dovecot.fi Mon May 9 07:11:09 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Mon, 9 May 2016 10:11:09 +0300 (EEST) Subject: more info about deleting users In-Reply-To: <976ded279ff32f605cc4906326cd8246@tomsommer.dk> References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> <24963661.1711.1462386797729@appsuite-dev.open-xchange.com> <976ded279ff32f605cc4906326cd8246@tomsommer.dk> Message-ID: <402943010.3976.1462777870416@appsuite-dev.open-xchange.com> > On May 9, 2016 at 10:01 AM Tom Sommer wrote: > > > On 2016-05-04 20:33, aki.tuomi at dovecot.fi wrote: > >> On May 2, 2016 at 9:56 AM Tom Sommer wrote: > >> On 2016-04-29 16:25, Matteo Gaito Test Account wrote: > >> > >> > i've search a lot in order to find a way to delete account by dovecot > >> > but i > >> > found only this way > >> > > >> > http://dovecot.org/list/dovecot/2011-November/079273.html > >> > > >> > It's possible today delete a maildir remotely with doveadm or with > >> > dovecot api? > >> > >> Would love this as well > > > > > > You could try using fs commands that are available in doveadm HTTP API. > > Looks doable. Any documentation? :) http://wiki2.dovecot.org/Design/DoveadmProtocol/HTTP --- Aki Tuomi From mail at tomsommer.dk Mon May 9 07:14:18 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Mon, 9 May 2016 09:14:18 +0200 Subject: more info about deleting users In-Reply-To: <402943010.3976.1462777870416@appsuite-dev.open-xchange.com> References: <260988049.76859.1461938117118.JavaMail.open-xchange@popper07.register.it> <1821159830.78070.1461939941971.JavaMail.open-xchange@popper07.register.it> <9a452d60e8fde753b24f5ed1d3da57d1@tomsommer.dk> <24963661.1711.1462386797729@appsuite-dev.open-xchange.com> <976ded279ff32f605cc4906326cd8246@tomsommer.dk> <402943010.3976.1462777870416@appsuite-dev.open-xchange.com> Message-ID: <68DBB4FC-F497-4B3C-AF80-94DF5EB39306@tomsommer.dk> > On 9. maj 2016, at 09.11, aki.tuomi at dovecot.fi wrote: > > >> On May 9, 2016 at 10:01 AM Tom Sommer wrote: >> >> >> On 2016-05-04 20:33, aki.tuomi at dovecot.fi wrote: >>>> On May 2, 2016 at 9:56 AM Tom Sommer wrote: >>>> On 2016-04-29 16:25, Matteo Gaito Test Account wrote: >>>> >>>>> i've search a lot in order to find a way to delete account by dovecot >>>>> but i >>>>> found only this way >>>>> >>>>> http://dovecot.org/list/dovecot/2011-November/079273.html >>>>> >>>>> It's possible today delete a maildir remotely with doveadm or with >>>>> dovecot api? >>>> >>>> Would love this as well >>> >>> >>> You could try using fs commands that are available in doveadm HTTP API. >> >> Looks doable. Any documentation? :) > > http://wiki2.dovecot.org/Design/DoveadmProtocol/HTTP > I meant the FS command :) From dovecot at netnerdz.se Mon May 9 15:50:51 2016 From: dovecot at netnerdz.se (Robban) Date: Mon, 09 May 2016 17:50:51 +0200 Subject: Workaround for rcpt suffix Message-ID: Hi all! I'm stuck again, let me start with explaining my setup. The PassDB and UserDB is LDAP (FreeIPA) and I'm using ldap mail attribute to search after the recipient so one user can have multiple mail aliases. So far so good. Next is to have an catchall at domain.tld, that is done via an static userdb backend (I know you are strongly against catchall adresses), and they are delivered to public folders (${mail_location}/domain.tld). I also use address extension so sort a mail to correct folder (+ sign). Now I need to fix so mail to -sievetest is handled by the user assigned to , is this somehow possible? //Robban From corneliuscharlie1 at gmail.com Mon May 9 21:20:48 2016 From: corneliuscharlie1 at gmail.com (Cornelius Charlie) Date: Mon, 9 May 2016 23:20:48 +0200 Subject: Mailbox doesn't exist response Message-ID: Hello, Using the latest dovecot version, selecting a non-existing mailbox with non printable characters encoded in utf7, the mailbox doesn't exist response contains the folder name encoded in utf8. Is that legal regarding the rfc ? If yes, how the client can be aware of the used encoding ? Thanks. From kremels at kreme.com Tue May 10 01:39:37 2016 From: kremels at kreme.com (@lbutlr) Date: Mon, 9 May 2016 19:39:37 -0600 Subject: resorting emails by date order after move/copy to sub folder ? In-Reply-To: References: Message-ID: <2FB86B88-6C18-49F0-883A-18C7D9A6B370@kreme.com> On Mon May 09 2016 00:22:42 voytek at sbt.net.au said: > > I have just used Squirrel web client to re organize some old emails to > subfolders. that worked OK, EXCEPT, all these emails are now in order that > I have moved them rather than original date, Your Squirreltail setup is broken then. > is that some 'simple' way to re-sort them back by original date? This might help: -- Tina... homecoming is spelled c *O* m From voytek at sbt.net.au Tue May 10 02:08:42 2016 From: voytek at sbt.net.au (voytek at sbt.net.au) Date: Tue, 10 May 2016 12:08:42 +1000 Subject: resorting emails by date order after move/copy to sub folder ? In-Reply-To: <2FB86B88-6C18-49F0-883A-18C7D9A6B370@kreme.com> References: <2FB86B88-6C18-49F0-883A-18C7D9A6B370@kreme.com> Message-ID: <7fc836f527c93be62d7ac5494fed769d.squirrel@sbt.net.au> On Tue, May 10, 2016 11:39 am, @lbutlr wrote: > Your Squirreltail setup is broken then. >> is that some 'simple' way to re-sort them back by original date? > This might help: > thanks ! I'll try V From skdovecot at smail.inf.fh-brs.de Tue May 10 06:06:51 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 10 May 2016 08:06:51 +0200 (CEST) Subject: Workaround for rcpt suffix In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 9 May 2016, Robban wrote: > I'm stuck again, let me start with explaining my setup. > The PassDB and UserDB is LDAP (FreeIPA) and I'm using ldap mail attribute to > search after the recipient so one user can have multiple mail aliases. So far > so good. > Next is to have an catchall at domain.tld, that is done via an static userdb > backend (I know you are strongly against catchall adresses), and they are > delivered to public folders (${mail_location}/domain.tld). > > I also use address extension so sort a mail to correct folder (+ sign). > Now I need to fix so mail to -sievetest is handled by the user > assigned to , is this somehow possible? your userdb query should return the data of "the user assigned to ", when Dovecot is to drop messages for -sievetest . I would add another LDAP attribute, store all "alternate recipient aliases" there and use a ldap filter query to test both attributes. You must return an "user" field in this case. http://wiki2.dovecot.org/UserDatabase/ExtraFields - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzF6e3z1H7kL/d9rAQLNOwf/a6DVdZa7vSYJkNWW57C4R3+0SMekiXWs SwyPcfyQeJUW+xCUpKjkUWMVfAI11YlSj1pYxS/Yh2emW8UbKfhHkccrEJf1Sewh CFwT2yOmgAEdEO5YkLFgILpTql7r8nEqx/csjsfdCLDzSJo11rmvRT058LnJ1MdG eTSN7DDNseIcExZE0276lqJGesZFIGNP/q3vwRh8ic+GzFLyQFpLAS/0B3zfHBG7 raMigR7fuggqXhu1sVsz0jRtUFhc5VN5EY30uSMGRotYON2jhQN0b8+hritp0TsY CReWQDX18knRmThKVeZCHFTZTeEf5CNlurujGSg4AJjj/5o5c1ZFMg== =OWh/ -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Tue May 10 06:13:45 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 10 May 2016 08:13:45 +0200 (CEST) Subject: Mailbox doesn't exist response In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 9 May 2016, Cornelius Charlie wrote: > Using the latest dovecot version, selecting a non-existing mailbox with non > printable characters encoded in utf7, the mailbox doesn't exist response > contains the folder name encoded in utf8. the comment is just an human readable string. > Is that legal regarding the rfc ? > > If yes, how the client can be aware of the used encoding ? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzF8GXz1H7kL/d9rAQLlAwf/RYA3chqcqvm/MkNI7PaEMCGn5loKCrcC T1AMMwhXfWzIH0+BNmr7gVVRQoWzxfGcfSvKUZbK20flaREt9S9SeokmAFc1qD7X LkQ6Wt87pS/24HZmUt/KhK9yhxxFfki3J77McRV2ookBmjx3x6oMYH5QdViOkNQg OBT7pGbbKieRpr6e9o3PFnXBx3hcGndGSFinybgE6KF19dI9chIOWW+B3XF2KYVf aB/wY4u2KI1nADHj/mfSTxqLGQgvwlEv95xofn+h5jjeTT+P01ML+rjbMaKDpgdd x5iMCIeKSZiSZpK71A1+4ySHXlG3qU6Cnl1yxJ8GGJAHks7YA7bFJQ== =UTgP -----END PGP SIGNATURE----- From abi at oleane.net Tue May 10 12:44:31 2016 From: abi at oleane.net (abi at oleane.net) Date: Tue, 10 May 2016 14:44:31 +0200 Subject: ACLs over imapc In-Reply-To: References: Message-ID: <5731D7AF.50907@oleane.net> hello i've struggled various times trying to setup shared mailboxes to no avail. the same setup with local file access works imapc seems to work with generic shared mailboxes nevertheless, i never managed to get a working setup with both user-defined acls and imapc has anyone been successful in doing so ? is this supposed to work ? i've been experimenting with dovecot 2.2.9 and 2.2.22 and never saw any imap acl commands issued by the builtin imap client. thanks abi at oleane.net From info at jonkri.org Tue May 10 19:02:46 2016 From: info at jonkri.org (Jon Kristensen) Date: Tue, 10 May 2016 21:02:46 +0200 Subject: Sieve extension for testing for mailbox Message-ID: <87h9e5wxbd.fsf@jonkri.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi! Which Pigeonhole Sieve extension should I use to process/test a message depending on whether the message is in a particular mailbox? (Is it the mboxmetadata extension by any chance? If so, I would appreciate if someone could tell me what the reason is for it not being enabled by default.) The reason I want to perform this test is that I have configured my MTA to deliver "outgoing" e-mails to my "Sent" IMAP folder, and these messages arrive unread. I would like to mark them as read using Sieve. Thanks in advance! - -- Jon Kristensen (jonkri) PGP fingerprint: 0377 F488 8908 0237 A2AE C484 387E C31D DA60 1350 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXMjBWAAoJEDh+wx3aYBNQfnIP/19D90MOwN1h/AXEuC2eSlXx MFHVuVgnXlc4mTXINkybBRxyjeaQlry7Lcbl2UR74rX5MLet6hEqu6l2tbBtbaKX 7VKDydOr55NsKUKvEDLg6DRhRRgHWYS1TxhENuRbhAriNZx4GbPnbAJAx8VL9wQ9 5W8PhTr8VMJUXiYZSCQ3LxXLHXrDAcIflh2vkJubiL5S0WQzbm1wyCplCAcUCTw4 Uk9StP0Zb9aMuwm+IsRDB9T0/2U90vCpz2b9UUJzg8yUwaTRIxSkoPNqaukSepsq tBNWKmQ8efKYq0AtISzQK5SSRQp7c4MudQUuNqU9oR85qMC5F/i/c//To7HBUuRd bkNr4gpvDXKKIYCAmcE8I1jU36M+E7Y5RE/gYkvA23/xC9cd7GpsaOAspWjVhm4l sp7DXnFkfGyM8/l3m5CVhfsFewFjrq8K9L5gVnCrEBRnfHqm/BeE7+rrQ4+LmPMl dZ45fwy8lFD2HIUE97p0E3n2xX4JvhMDhR/gilnSl/ssmVVpnbpVAgXrmqWJ9cr4 dfue7unhvrM5DTQ0hM9WoWXzCi/tSOHvYido1rjhtEYzCHIdTtJKAUVg69CZHLHu zDodyGn70kxxS5YOYMYuIahUFaZknbiuCV3MGh4p2sTGOY69kFANQlB00m0ZaiPZ D6cjf2Yz0Ny2RsAUgyPw =g2F3 -----END PGP SIGNATURE----- From stephan at rename-it.nl Wed May 11 00:37:53 2016 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 11 May 2016 02:37:53 +0200 Subject: Automatic packages for Dovecot v2.3 Message-ID: <57327EE1.5030105@rename-it.nl> Hi, The Xi repository for v2.3 is now public. The information is updated here: http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages Xi now modifies the version string in the Dovecot sources. This way `dovecot -n` output includes the Xi package version. This is currently only true for the v2.3 packages though. I am still testing a new v2.2 builder that follows both the master and latest release branch. Packages from the release branch are published in a separate repository, so that a more stable package stream is obtained; it is only updated when the developers start working on making a release. The intermittent beta or rc release tags on that release branch are recognized and reflected in the package versions. If a new release branch is detected, the the development packages for the master branch get an alpha version with a lower version digit that is one higher than the upcoming release. For example, when a release-2.2.25 branch is created, packages from the master branch will get version 2.2.26~alpha0-1 in anticipation of the next release in this series. Packages from the release branch will have version 2.2.25~beta0-1 until the first release tag is seen. Regards, Stephan. From skdovecot at smail.inf.fh-brs.de Wed May 11 06:04:06 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 11 May 2016 08:04:06 +0200 (CEST) Subject: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox) In-Reply-To: <87h9e5wxbd.fsf@jonkri.org> References: <87h9e5wxbd.fsf@jonkri.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 10 May 2016, Jon Kristensen wrote: > Which Pigeonhole Sieve extension should I use to process/test a message > depending on whether the message is in a particular mailbox? (Is it the > mboxmetadata extension by any chance? If so, I would appreciate if > someone could tell me what the reason is for it not being enabled by > default.) > The reason I want to perform this test is that I have configured my MTA > to deliver "outgoing" e-mails to my "Sent" IMAP folder, and these > messages arrive unread. I would like to mark them as read using Sieve. er, I guess, "message is in a particular mailbox" is meaning it to be spooled, or something like that, right? Do you use subaddressing (aka +addressing, +detail addressing)? Then look here: http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Plus_Addressed_mail_filtering If you want to resolve the question in the second paragraphe, search the mailing list archive about how to get the name of the mailbox, the implicit keep stores the message in. I'm pretty sure, that there was such question lately. If you actually want to resolve the question in the first paragraphe, to check whether or not the same message has been spooled in a folder already (aka duplicate check), check out the duplicate extension with ":handle" argument. But you probably need to know the target mailbox, too. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzLLVnz1H7kL/d9rAQJUdwgAi/mMAxMxAGleV1TkAhZH0p37bLH7NWcI OpkQ9VDQEXr8WFjXQesITpwJrnROZFL+IrmIpWY6s+tcjO045PU9N89gfPB+gjt+ g9fN+w6eKtOKvsLQdAsD8Ut+LP0qaLORaMgFHe+2ZpJItxp5iz7nSvXymusqB9ad lhIre61YfU7u97t/UUG2uvK7s4mdLhapX0wDP+Lp7KP3E19lWCK7IFvXZ6cWjNY/ JHZUytcKEghlF1LPnPp5372kUn/46NN8q9tYA/Qwsy4ErDImNBYmSnJUi05Q6uDR RYaBjZR1pq9mT2oOkBAds+z570YRIfO37HX7KVH+BvnHyimbItmPjg== =80sz -----END PGP SIGNATURE----- From thomas.hofmann at post.ch Wed May 11 08:22:05 2016 From: thomas.hofmann at post.ch (thomas.hofmann at post.ch) Date: Wed, 11 May 2016 08:22:05 +0000 Subject: Undeletable mailboxes Message-ID: <7175FBD8FD977B44AD1A48A3D7AB46A20BA7C5AB@HXMB14.pnet.ch> Hi I have to maintain a dovcot installation where users may access there mails through IMAPS and/or via self-developed portal. >From time to time users complain about mailboxes showing up they have deleted bevor. The only way to get these mailboxes away is to do a backup of the user and delete dovecot home of the user and do a selective restore. hostname:~ # doveadm mailbox list -u $u -s 'INBOX/01_OHO/*' hostname:~ # doveadm mailbox list -u $u 'INBOX/01_OHO/*' INBOX/01_OHO/02_Orchestermatin?e 10.01.16 hostname:~ # doveadm mailbox delete -u $u 'INBOX/01_OHO/02_Orchestermatin?e 10.01.16' doveadm(username at domain.com): Info: Mailbox deleted: INBOX/01_OHO/02_Orchestermatin?e 10.01.16 hostname:~ # doveadm mailbox list -u $u 'INBOX/01_OHO/*' INBOX/01_OHO/02_Orchestermatin?e 10.01.16 hostname:~ # doveadm -v purge -u $u hostname:~ # doveadm mailbox list -u $u 'INBOX/01_OHO/*' INBOX/01_OHO/02_Orchestermatin?e 10.01.16 hostname:~ # doveadm mailbox metadata list -u $u 'INBOX/01_OHO/02_Orchestermatin?e 10.01.16' doveadm(username at domain.com): Error: Failed to open mailbox: Mailbox doesn't exist: INBOX/01_OHO/02_Orchestermatin?e 10.01.16 hostname:~ # hostname:~ # dovecot --version 2.2.21 (5345f22) hostname:~ # Is there another (simpler) way to rid of these "deleted" mailboxes? Thanks for any help! Regards Thom From Ralf.Hildebrandt at charite.de Wed May 11 09:11:18 2016 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 11 May 2016 11:11:18 +0200 Subject: Automatic packages for Dovecot v2.3 In-Reply-To: <57327EE1.5030105@rename-it.nl> References: <57327EE1.5030105@rename-it.nl> Message-ID: <20160511091117.GD18043@charite.de> * Stephan Bosch : > Hi, > > The Xi repository for v2.3 is now public. The information is updated here: > > http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages Just made an upgrade from 2.2 to 2.3 and it's working ok (imapc, proxy setup) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From luckyfellow42 at gmail.com Wed May 11 11:50:31 2016 From: luckyfellow42 at gmail.com (Andreas Meyer) Date: Wed, 11 May 2016 13:50:31 +0200 Subject: Dovecot + libsodium In-Reply-To: References: Message-ID: 2016-05-06 23:15 GMT+02:00 Timo Sirainen : > On 06 May 2016, at 13:14, Andreas Meyer wrote: > > > > Hi, > > > > Thank you very much for creating and maintaining dovecot! > > > > In my scenario, I want to use the password hash algorithms provided by > > libsodium: https://download.libsodium.org/doc/ > > > > So my difficulty is to have dovecot support libsodium's hash algorithms, > > particularly: crypto_pwhash_scryptsalsa208sha256_str > > > > On the sodium maillinglist I asked for help and received an adjusted > > dovecot code, which exactly does what I need. You find it here: > > https://github.com/jedisct1/core/tree/scrypt-argon2 > > > > Obviously I need to apply these changes everytime I upgrade to a new > > dovecot version now. > > > > So my question ist, what do I need to do so that you will include > libsodium > > support in future versions of dovecot? > > You could also change it to be a plugin to avoid patching. This is a > pretty old example, but it probably still works, at least with minor > changes: > http://dovecot.org/patches/password-scheme-lmpass.c > > Although it's still a good idea to recompile the plugin after a new > version since sometimes the ABI changes. > > Hi Timo, thank you very much for your reply. Creating a plugin is an option. Though I don't possess the right abilities to do that right away. Nevertheless I want to re-ask my initial question: What is required to get libsodium support into the dovecot core? Or are there concerns about supporting it or is there simply no interest in doing so? As I understand, security is a relevant concern when developing Dovecot. The sodium crypto library focuses on: "... provide all of the core operations needed to build higher-level cryptographic tools." I am sure, utilizing this library by default can be of great benefit for Dovecot. It will help to easily support the latest password hashing algorithms, currently Scrypt and Argon2. And if used for additional cryptographic purposes, it also provides easy to use cryptographically secure pseudo random data, secret-key authenticated encryption and of course secure memory allocations, just to name three features. Thank you very much, Andreas From yacinechaouche at yahoo.com Wed May 11 13:28:04 2016 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Wed, 11 May 2016 13:28:04 +0000 (UTC) Subject: How do I share folders ? References: <5370327.466239.1462973284288.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <5370327.466239.1462973284288.JavaMail.yahoo@mail.yahoo.com> Hello dovecot, First poster here. What I want : give a.chaouche at algerian-radio.dz list and read permissions on a.chaouche at backup.algerian-radio.dz. Let's just discuss sharing the inbox then I can do the same for subfolders, in any. a.chaouche at algerian-radio.dz and a.chaouche at backup.algerian-radio.dz are two different users. Details about the domains aren't relevant here I guess. My dovecot configuration, what I believe to be the important parts :userdb { args = uid=vmail gid=vmail home=/var/vmail/%d/%n/ driver = static } namespace partage { location = maildir:%%h:INDEX=~/%%u prefix = shared.%%u. separator = . type = shared } mail_location = Maildir:~/ Both users can access their respective inboxes just fine, I only need to give a.chaouche at algerian-radio.dz access to a.chaouche at backup.algerian-radio.dz Full output of doveconf -n : # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.3 auth_mechanisms = plain login debug_log_path = /var/log/dovecot.debug disable_plaintext_auth = no first_valid_uid = 100 log_path = /var/log/dovecot.log mail_debug = yes mail_location = Maildir:~/ mail_plugins = " quota acl" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . type = private } namespace partage { hidden = no location = maildir:%%h:INDEX=~/%%u prefix = shared.%%u. separator = . type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile quota = maildir:User quota quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = root at algerian-radio.dz protocols = " imap lmtp sieve" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = no userdb { args = uid=vmail gid=vmail home=/var/vmail/%d/%n/ driver = static } protocol lmtp { mail_plugins = " quota acl sieve quota" } protocol lda { mail_plugins = " quota acl sieve" } protocol imap { mail_plugins = " quota acl imap_quota imap_acl" } I don't know if it's possible to share folders with the doveadm acl command ? root at messagerie[10.10.10.20] ~ # doveadm acl debug? -u a.chaouche at algerian-radio.dz shared/a.chaouche at backup.algerian-radio.dz doveadm(a.chaouche at algerian-radio.dz): Error: Can't open mailbox shared/a.chaouche at backup.algerian-radio.dz: Mailbox doesn't exist: shared/a.chaouche at backup.algerian-radio.dz root at messagerie[10.10.10.20] ~ # From skdovecot at smail.inf.fh-brs.de Wed May 11 14:09:32 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 11 May 2016 16:09:32 +0200 (CEST) Subject: How do I share folders ? In-Reply-To: <5370327.466239.1462973284288.JavaMail.yahoo@mail.yahoo.com> References: <5370327.466239.1462973284288.JavaMail.yahoo.ref@mail.yahoo.com> <5370327.466239.1462973284288.JavaMail.yahoo@mail.yahoo.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 11 May 2016, chaouche yacine wrote: > namespace inbox { > prefix = INBOX. > separator = . ^^^^^^^^ > type = private > } > namespace partage { > hidden = no > location = maildir:%%h:INDEX=~/%%u > prefix = shared.%%u. ^^^^^^^^^ > separator = . > type = shared > } looks fine for me, > I don't know if it's possible to share folders with the doveadm acl command ? > root at messagerie[10.10.10.20] ~ # doveadm acl debug? -u a.chaouche at algerian-radio.dz shared/a.chaouche at backup.algerian-radio.dz > doveadm(a.chaouche at algerian-radio.dz): Error: Can't open mailbox shared/a.chaouche at backup.algerian-radio.dz: Mailbox doesn't exist: shared/a.chaouche at backup.algerian-radio.dz you are using the dot as separator, that is: shared/a.chaouche at backup. should spell shared.a.chaouche at backup. However, because there are dots in the username, too, this probably won't work, because Dovecot probably picks up "a" as username. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzM9HHz1H7kL/d9rAQIGmwgAi7ga1a1Vv8HfvcAc6mIqxkBHdI5jL3MC YfMVwwuj7NJMjS8l/ouYxJCXdMn5k+sbRifwR/jyBcZtsvz3/ijkMx3RQyoq+LwZ r01Mxb4YR7ub5sNUIeIRtRcR/aNfDLNZb86PpDCHs2WKLU2atS5vVLmJWNiMaJNV MSG72OFKoLUJuRoSM26c2AH/f33WjuEinjjGMk481ezNBmZIuSWU2lJjmUXjfox9 BPzGRUkUKtlV9tkQjJUVjuwcfmvs+ChmTI3O2F37KWswP8zw0/EYXdyRpWCJso7F 8Ecrbu14cSHjisaerTQ4xko+ossPGoEE+Rh7XHuv8VDdNruur5AeVg== =DH0D -----END PGP SIGNATURE----- From daniel.colchete at gmail.com Wed May 11 19:38:54 2016 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Wed, 11 May 2016 16:38:54 -0300 Subject: Bug Report - Crash on imap with core dump - Alternate storage related Message-ID: I would like to report a bug we are seeing on an new deployment here: Linux: Debian Jessie, inside a Docker container Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1 doveconf -n: http://pastebin.com/2xA2xuRw Filesystem: CephFS - Jewel After migrating our users from our old server to our new one we can't access a few mailboxes. Backtrace follows: root at dovecot:/# gdb --args /usr/lib/dovecot/imap -u daniel.colchete at mav.com.br GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done. done. (gdb) run Starting program: /usr/lib/dovecot/imap -u daniel.colchete at mav.com.br * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as daniel.colchete at mav.com.br a select INBOX Program received signal SIGSEGV, Segmentation fault. strrchr () at ../sysdeps/x86_64/strrchr.S:32 32 ../sysdeps/x86_64/strrchr.S: No such file or directory. (gdb) where #0 strrchr () at ../sysdeps/x86_64/strrchr.S:32 #1 0x00007ffff7b0a261 in sdbox_file_move (file=file at entry=0x5555557e6530, alt_path=alt_path at entry=true) at sdbox-file.c:312 #2 0x00007ffff7b0cd11 in dbox_sync_file_move_if_needed (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x5555557e6530) at sdbox-sync.c:30 #3 sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120, seq=819, ctx=0x5555557e32e0) at sdbox-sync.c:58 #4 sdbox_sync_add (sync_rec=0x7fffffffe570, ctx=0x5555557e32e0) at sdbox-sync.c:96 #5 sdbox_sync_index (ctx=0x5555557e32e0) at sdbox-sync.c:131 #6 sdbox_sync_begin (mbox=mbox at entry=0x5555557ce500, flags=flags at entry=(unknown: 0), ctx_r=ctx_r at entry=0x7fffffffe680) at sdbox-sync.c:234 #7 0x00007ffff7b0d07d in sdbox_sync (mbox=mbox at entry=0x5555557ce500, flags=flags at entry=(unknown: 0)) at sdbox-sync.c:297 #8 0x00007ffff7b0d108 in sdbox_storage_sync_init (box=0x5555557ce500, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at sdbox-sync.c:323 #9 0x00007ffff7aea85b in mailbox_sync_init (box=0x5555557ce500, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at mail-storage.c:1664 #10 0x00007ffff7aea9a7 in mailbox_sync (box=, flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at mail-storage.c:1712 #11 0x0000555555568fa0 in select_open (readonly=false, mailbox=, ctx=0x5555557c9928) at cmd-select.c:303 #12 cmd_select_full (cmd=, readonly=) at cmd-select.c:426 #13 0x000055555556fdbd in command_exec (cmd=cmd at entry=0x5555557c9800) at imap-commands.c:180 #14 0x000055555556e182 in client_command_input (cmd=cmd at entry=0x5555557c9800) at imap-client.c:958 #15 0x000055555556e210 in client_command_input (cmd=0x5555557c9800) at imap-client.c:1018 #16 0x000055555556e5d5 in client_handle_next_command (remove_io_r=, client=0x5555557c8c00) at imap-client.c:1058 #17 client_handle_input (client=0x5555557c8c00) at imap-client.c:1070 #18 0x000055555556ead5 in client_input (client=0x5555557c8c00) at imap-client.c:1117 #19 0x00007ffff7837c3c in io_loop_call_io (io=0x5555557c96f0) at ioloop.c:564 #20 0x00007ffff78390a1 in io_loop_handler_run_internal (ioloop=ioloop at entry=0x5555557966e0) at ioloop-epoll.c:220 #21 0x00007ffff7837cc5 in io_loop_handler_run (ioloop=ioloop at entry=0x5555557966e0) at ioloop.c:612 #22 0x00007ffff7837e68 in io_loop_run (ioloop=0x5555557966e0) at ioloop.c:588 #23 0x00007ffff77ce0e3 in master_service_run (service=0x555555796580, callback=) at master-service.c:640 #24 0x0000555555561ab2 in main (argc=3, argv=0x555555796390) at main.c:454 (gdb) One important info is that I used to use Alternate Storage on the older servers, but the not on the new one. Code seems to be trying to move a message to an alternate storage that doesn't exist. doveadm dump dovecot.index will show that we still have the ALTERNATE_STORAGE flag set: root at dovecot:/srv/dovecot/mail/ mav.com.br/daniel.colchete/mailboxes/INBOX/dbox-Mails# doveadm dump dovecot.index|grep RECORD|head -- RECORDS: 932 RECORD: seq=1, uid=9393, flags=0x49 (Seen Answered) RECORD: seq=2, uid=9442, flags=0x40 RECORD: seq=3, uid=10253, flags=0x48 (Seen) RECORD: seq=4, uid=10268, flags=0x48 (Seen) RECORD: seq=5, uid=10283, flags=0x48 (Seen) RECORD: seq=6, uid=10309, flags=0x48 (Seen) RECORD: seq=7, uid=10311, flags=0x40 RECORD: seq=8, uid=10324, flags=0x49 (Seen Answered) RECORD: seq=9, uid=10330, flags=0x48 (Seen) Thank you very much. Best, Daniel Colchete From daniel.colchete at gmail.com Wed May 11 19:45:42 2016 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Wed, 11 May 2016 16:45:42 -0300 Subject: Bug Report - Crash on imap with core dump - Alternate storage related In-Reply-To: References: Message-ID: I can confirm that this bug also happens on the latest version available: root at dovecot:/# dovecot --version 2.2.devel (ebf2e93) root at dovecot:/# dpkg -l|grep dovecot ii dovecot-core 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - core files ii dovecot-dbg 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - debug symbols ii dovecot-imapd 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-lmtpd 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - LMTP server ii dovecot-lucene 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - Lucene support ii dovecot-mysql 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - Sieve filters support ii dovecot-solr 2:2.2.24-1~auto+23 amd64 secure POP3/IMAP server - Solr support On Wed, May 11, 2016 at 4:38 PM, Daniel van Ham Colchete < daniel.colchete at gmail.com> wrote: > I would like to report a bug we are seeing on an new deployment here: > > Linux: Debian Jessie, inside a Docker container > Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1 > doveconf -n: http://pastebin.com/2xA2xuRw > Filesystem: CephFS - Jewel > > After migrating our users from our old server to our new one we can't > access a few mailboxes. Backtrace follows: > > root at dovecot:/# gdb --args /usr/lib/dovecot/imap -u > daniel.colchete at mav.com.br > GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 > Copyright (C) 2014 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later < > http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > . > Find the GDB manual and other documentation resources online at: > . > For help, type "help". > Type "apropos word" to search for commands related to "word"... > Reading symbols from /usr/lib/dovecot/imap...Reading symbols from > /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done. > done. > (gdb) run > Starting program: /usr/lib/dovecot/imap -u daniel.colchete at mav.com.br > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN > CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as > daniel.colchete at mav.com.br > a select INBOX > > Program received signal SIGSEGV, Segmentation fault. > strrchr () at ../sysdeps/x86_64/strrchr.S:32 > 32 ../sysdeps/x86_64/strrchr.S: No such file or directory. > (gdb) where > #0 strrchr () at ../sysdeps/x86_64/strrchr.S:32 > #1 0x00007ffff7b0a261 in sdbox_file_move (file=file at entry=0x5555557e6530, > alt_path=alt_path at entry=true) at sdbox-file.c:312 > #2 0x00007ffff7b0cd11 in dbox_sync_file_move_if_needed > (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x5555557e6530) at > sdbox-sync.c:30 > #3 sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120, > seq=819, ctx=0x5555557e32e0) at sdbox-sync.c:58 > #4 sdbox_sync_add (sync_rec=0x7fffffffe570, ctx=0x5555557e32e0) at > sdbox-sync.c:96 > #5 sdbox_sync_index (ctx=0x5555557e32e0) at sdbox-sync.c:131 > #6 sdbox_sync_begin (mbox=mbox at entry=0x5555557ce500, flags=flags at entry=(unknown: > 0), ctx_r=ctx_r at entry=0x7fffffffe680) at sdbox-sync.c:234 > #7 0x00007ffff7b0d07d in sdbox_sync (mbox=mbox at entry=0x5555557ce500, > flags=flags at entry=(unknown: 0)) at sdbox-sync.c:297 > #8 0x00007ffff7b0d108 in sdbox_storage_sync_init (box=0x5555557ce500, > flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) > at sdbox-sync.c:323 > #9 0x00007ffff7aea85b in mailbox_sync_init (box=0x5555557ce500, > flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) > at mail-storage.c:1664 > #10 0x00007ffff7aea9a7 in mailbox_sync (box=, > flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at > mail-storage.c:1712 > #11 0x0000555555568fa0 in select_open (readonly=false, mailbox= out>, ctx=0x5555557c9928) at cmd-select.c:303 > #12 cmd_select_full (cmd=, readonly=) at > cmd-select.c:426 > #13 0x000055555556fdbd in command_exec (cmd=cmd at entry=0x5555557c9800) at > imap-commands.c:180 > #14 0x000055555556e182 in client_command_input (cmd=cmd at entry=0x5555557c9800) > at imap-client.c:958 > #15 0x000055555556e210 in client_command_input (cmd=0x5555557c9800) at > imap-client.c:1018 > #16 0x000055555556e5d5 in client_handle_next_command > (remove_io_r=, client=0x5555557c8c00) at > imap-client.c:1058 > #17 client_handle_input (client=0x5555557c8c00) at imap-client.c:1070 > #18 0x000055555556ead5 in client_input (client=0x5555557c8c00) at > imap-client.c:1117 > #19 0x00007ffff7837c3c in io_loop_call_io (io=0x5555557c96f0) at > ioloop.c:564 > #20 0x00007ffff78390a1 in io_loop_handler_run_internal (ioloop=ioloop at entry=0x5555557966e0) > at ioloop-epoll.c:220 > #21 0x00007ffff7837cc5 in io_loop_handler_run (ioloop=ioloop at entry=0x5555557966e0) > at ioloop.c:612 > #22 0x00007ffff7837e68 in io_loop_run (ioloop=0x5555557966e0) at > ioloop.c:588 > #23 0x00007ffff77ce0e3 in master_service_run (service=0x555555796580, > callback=) at master-service.c:640 > #24 0x0000555555561ab2 in main (argc=3, argv=0x555555796390) at main.c:454 > (gdb) > > One important info is that I used to use Alternate Storage on the older > servers, but the not on the new one. Code seems to be trying to move a > message to an alternate storage that doesn't exist. > > doveadm dump dovecot.index will show that we still have the > ALTERNATE_STORAGE flag set: > > root at dovecot:/srv/dovecot/mail/ > mav.com.br/daniel.colchete/mailboxes/INBOX/dbox-Mails# doveadm dump > dovecot.index|grep RECORD|head > -- RECORDS: 932 > RECORD: seq=1, uid=9393, flags=0x49 (Seen Answered) > RECORD: seq=2, uid=9442, flags=0x40 > RECORD: seq=3, uid=10253, flags=0x48 (Seen) > RECORD: seq=4, uid=10268, flags=0x48 (Seen) > RECORD: seq=5, uid=10283, flags=0x48 (Seen) > RECORD: seq=6, uid=10309, flags=0x48 (Seen) > RECORD: seq=7, uid=10311, flags=0x40 > RECORD: seq=8, uid=10324, flags=0x49 (Seen Answered) > RECORD: seq=9, uid=10330, flags=0x48 (Seen) > > Thank you very much. > > Best, > Daniel Colchete > From s.hoogeveen at nederhost.nl Wed May 11 20:49:35 2016 From: s.hoogeveen at nederhost.nl (NederHost/Sebastiaan Hoogeveen) Date: Wed, 11 May 2016 22:49:35 +0200 Subject: [patch] Fix for returning NULL values in SQL dict lookups Message-ID: Hi, I noticed a bug doing dict lookups on an SQLite database which had NULL values in its columns; a segmentation fault occurred, probably due to a null pointer dereference in str_tabescape. The problem is that sqlite3_column_text returns a null pointer for column values which are (SQL) NULL. It seems the other database drivers do something similar. The following patch makes the dict server check for null pointers and return a 'not found' reply in those cases (I changed the order around in the decision tree to avoid having to repeat return values): diff -Naur dovecot-2.2.24/src/dict/dict-commands.c dovecot-2.2.24-patched/src/dict/dict-commands.c --- dovecot-2.2.24/src/dict/dict-commands.c 2016-04-26 15:01:20.000000000 +0200 +++ dovecot-2.2.24-patched/src/dict/dict-commands.c 2016-05-11 22:04:06.000000000 +0200 @@ -83,14 +83,14 @@ { struct dict_connection_cmd *cmd = context; - if (result->ret > 0) { + if (result->ret > 0 && result->value) { cmd->reply = i_strdup_printf("%c%s\n", DICT_PROTOCOL_REPLY_OK, str_tabescape(result->value)); - } else if (result->ret == 0) { - cmd->reply = i_strdup_printf("%c\n", DICT_PROTOCOL_REPLY_NOTFOUND); - } else { + } else if (result->ret < 0) { i_error("%s", result->error); cmd->reply = i_strdup_printf("%c\n", DICT_PROTOCOL_REPLY_FAIL); + } else { + cmd->reply = i_strdup_printf("%c\n", DICT_PROTOCOL_REPLY_NOTFOUND); } dict_connection_cmds_flush(cmd->conn); } Kind regards, -- Sebastiaan Hoogeveen NederHost https://www.nederhost.nl/ KvK: 34099781 From aki.tuomi at dovecot.fi Thu May 12 06:03:48 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Thu, 12 May 2016 09:03:48 +0300 Subject: Bug Report - Crash on imap with core dump - Alternate storage related In-Reply-To: References: Message-ID: <57341CC4.5020507@dovecot.fi> On 11.05.2016 22:45, Daniel van Ham Colchete wrote: > I can confirm that this bug also happens on the latest version available: > > root at dovecot:/# dovecot --version > 2.2.devel (ebf2e93) > root at dovecot:/# dpkg -l|grep dovecot > ii dovecot-core 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - core files > ii dovecot-dbg 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - debug symbols > ii dovecot-imapd 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - IMAP daemon > ii dovecot-lmtpd 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - LMTP server > ii dovecot-lucene 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - Lucene support > ii dovecot-mysql 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - MySQL support > ii dovecot-pop3d 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - POP3 daemon > ii dovecot-sieve 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - Sieve filters support > ii dovecot-solr 2:2.2.24-1~auto+23 amd64 > secure POP3/IMAP server - Solr support > > > On Wed, May 11, 2016 at 4:38 PM, Daniel van Ham Colchete < > daniel.colchete at gmail.com> wrote: > >> I would like to report a bug we are seeing on an new deployment here: >> >> Linux: Debian Jessie, inside a Docker container >> Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1 >> doveconf -n: http://pastebin.com/2xA2xuRw >> Filesystem: CephFS - Jewel >> >> After migrating our users from our old server to our new one we can't >> access a few mailboxes. Backtrace follows: >> >> root at dovecot:/# gdb --args /usr/lib/dovecot/imap -u >> daniel.colchete at mav.com.br >> GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 >> Copyright (C) 2014 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later < >> http://gnu.org/licenses/gpl.html> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show copying" >> and "show warranty" for details. >> This GDB was configured as "x86_64-linux-gnu". >> Type "show configuration" for configuration details. >> For bug reporting instructions, please see: >> . >> Find the GDB manual and other documentation resources online at: >> . >> For help, type "help". >> Type "apropos word" to search for commands related to "word"... >> Reading symbols from /usr/lib/dovecot/imap...Reading symbols from >> /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done. >> done. >> (gdb) run >> Starting program: /usr/lib/dovecot/imap -u daniel.colchete at mav.com.br >> * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE >> IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT >> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS >> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN >> CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as >> daniel.colchete at mav.com.br >> a select INBOX >> >> Program received signal SIGSEGV, Segmentation fault. >> strrchr () at ../sysdeps/x86_64/strrchr.S:32 >> 32 ../sysdeps/x86_64/strrchr.S: No such file or directory. >> (gdb) where >> #0 strrchr () at ../sysdeps/x86_64/strrchr.S:32 >> #1 0x00007ffff7b0a261 in sdbox_file_move (file=file at entry=0x5555557e6530, >> alt_path=alt_path at entry=true) at sdbox-file.c:312 >> #2 0x00007ffff7b0cd11 in dbox_sync_file_move_if_needed >> (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x5555557e6530) at >> sdbox-sync.c:30 >> #3 sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120, >> seq=819, ctx=0x5555557e32e0) at sdbox-sync.c:58 >> #4 sdbox_sync_add (sync_rec=0x7fffffffe570, ctx=0x5555557e32e0) at >> sdbox-sync.c:96 >> #5 sdbox_sync_index (ctx=0x5555557e32e0) at sdbox-sync.c:131 >> #6 sdbox_sync_begin (mbox=mbox at entry=0x5555557ce500, flags=flags at entry=(unknown: >> 0), ctx_r=ctx_r at entry=0x7fffffffe680) at sdbox-sync.c:234 >> #7 0x00007ffff7b0d07d in sdbox_sync (mbox=mbox at entry=0x5555557ce500, >> flags=flags at entry=(unknown: 0)) at sdbox-sync.c:297 >> #8 0x00007ffff7b0d108 in sdbox_storage_sync_init (box=0x5555557ce500, >> flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) >> at sdbox-sync.c:323 >> #9 0x00007ffff7aea85b in mailbox_sync_init (box=0x5555557ce500, >> flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) >> at mail-storage.c:1664 >> #10 0x00007ffff7aea9a7 in mailbox_sync (box=, >> flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at >> mail-storage.c:1712 >> #11 0x0000555555568fa0 in select_open (readonly=false, mailbox=> out>, ctx=0x5555557c9928) at cmd-select.c:303 >> #12 cmd_select_full (cmd=, readonly=) at >> cmd-select.c:426 >> #13 0x000055555556fdbd in command_exec (cmd=cmd at entry=0x5555557c9800) at >> imap-commands.c:180 >> #14 0x000055555556e182 in client_command_input (cmd=cmd at entry=0x5555557c9800) >> at imap-client.c:958 >> #15 0x000055555556e210 in client_command_input (cmd=0x5555557c9800) at >> imap-client.c:1018 >> #16 0x000055555556e5d5 in client_handle_next_command >> (remove_io_r=, client=0x5555557c8c00) at >> imap-client.c:1058 >> #17 client_handle_input (client=0x5555557c8c00) at imap-client.c:1070 >> #18 0x000055555556ead5 in client_input (client=0x5555557c8c00) at >> imap-client.c:1117 >> #19 0x00007ffff7837c3c in io_loop_call_io (io=0x5555557c96f0) at >> ioloop.c:564 >> #20 0x00007ffff78390a1 in io_loop_handler_run_internal (ioloop=ioloop at entry=0x5555557966e0) >> at ioloop-epoll.c:220 >> #21 0x00007ffff7837cc5 in io_loop_handler_run (ioloop=ioloop at entry=0x5555557966e0) >> at ioloop.c:612 >> #22 0x00007ffff7837e68 in io_loop_run (ioloop=0x5555557966e0) at >> ioloop.c:588 >> #23 0x00007ffff77ce0e3 in master_service_run (service=0x555555796580, >> callback=) at master-service.c:640 >> #24 0x0000555555561ab2 in main (argc=3, argv=0x555555796390) at main.c:454 >> (gdb) >> >> One important info is that I used to use Alternate Storage on the older >> servers, but the not on the new one. Code seems to be trying to move a >> message to an alternate storage that doesn't exist. >> >> doveadm dump dovecot.index will show that we still have the >> ALTERNATE_STORAGE flag set: >> >> root at dovecot:/srv/dovecot/mail/ >> mav.com.br/daniel.colchete/mailboxes/INBOX/dbox-Mails# doveadm dump >> dovecot.index|grep RECORD|head >> -- RECORDS: 932 >> RECORD: seq=1, uid=9393, flags=0x49 (Seen Answered) >> RECORD: seq=2, uid=9442, flags=0x40 >> RECORD: seq=3, uid=10253, flags=0x48 (Seen) >> RECORD: seq=4, uid=10268, flags=0x48 (Seen) >> RECORD: seq=5, uid=10283, flags=0x48 (Seen) >> RECORD: seq=6, uid=10309, flags=0x48 (Seen) >> RECORD: seq=7, uid=10311, flags=0x40 >> RECORD: seq=8, uid=10324, flags=0x49 (Seen Answered) >> RECORD: seq=9, uid=10330, flags=0x48 (Seen) >> >> Thank you very much. >> >> Best, >> Daniel Colchete >> Hi! Thank you for your report, we will have a look. Can you provide output of doveconf -n please? --- Aki Tuomi Dovecot Oy From aki.tuomi at dovecot.fi Thu May 12 07:49:56 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Thu, 12 May 2016 10:49:56 +0300 Subject: Dovecot + libsodium In-Reply-To: References: Message-ID: <573435A4.10006@dovecot.fi> On 11.05.2016 14:50, Andreas Meyer wrote: > 2016-05-06 23:15 GMT+02:00 Timo Sirainen : > >> On 06 May 2016, at 13:14, Andreas Meyer wrote: >>> Hi, >>> >>> Thank you very much for creating and maintaining dovecot! >>> >>> In my scenario, I want to use the password hash algorithms provided by >>> libsodium: https://download.libsodium.org/doc/ >>> >>> So my difficulty is to have dovecot support libsodium's hash algorithms, >>> particularly: crypto_pwhash_scryptsalsa208sha256_str >>> >>> On the sodium maillinglist I asked for help and received an adjusted >>> dovecot code, which exactly does what I need. You find it here: >>> https://github.com/jedisct1/core/tree/scrypt-argon2 >>> >>> Obviously I need to apply these changes everytime I upgrade to a new >>> dovecot version now. >>> >>> So my question ist, what do I need to do so that you will include >> libsodium >>> support in future versions of dovecot? >> You could also change it to be a plugin to avoid patching. This is a >> pretty old example, but it probably still works, at least with minor >> changes: >> http://dovecot.org/patches/password-scheme-lmpass.c >> >> Although it's still a good idea to recompile the plugin after a new >> version since sometimes the ABI changes. >> >> > Hi Timo, > > > thank you very much for your reply. Creating a plugin is an option. Though > I don't possess the right abilities to do that right away. > > Nevertheless I want to re-ask my initial question: What is required to get > libsodium support into the dovecot core? > Or are there concerns about supporting it or is there simply no interest in > doing so? > > As I understand, security is a relevant concern when developing Dovecot. > The sodium crypto library focuses on: "... provide all of the core > operations needed to build higher-level cryptographic tools." > I am sure, utilizing this library by default can be of great benefit for > Dovecot. It will help to easily support the latest password hashing > algorithms, currently Scrypt and Argon2. > And if used for additional cryptographic purposes, it also provides easy to > use cryptographically secure pseudo random data, secret-key authenticated > encryption and of course secure memory allocations, just to name three > features. > > > Thank you very much, > > Andreas Hi! At the moment we do not want to add libsodium as permanent core dependency. If you are willing to make a plugin for sodium, that provides f.ex. Scrypt and Argon2 support, and make a pull request out of that, we are happy to include it as plugin. Due to how dovecot works adding new libraries to certain places is not really wanted, since the memory requirements on large installations will increase quite a lot. We already are able to handle most of the things sodium would provide with current code. Aki From daniel.colchete at gmail.com Thu May 12 09:53:47 2016 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Thu, 12 May 2016 06:53:47 -0300 Subject: Bug Report - Crash on imap with core dump - Alternate storage related In-Reply-To: <57341CC4.5020507@dovecot.fi> References: <57341CC4.5020507@dovecot.fi> Message-ID: Hi Aki! It was already there: http://pastebin.com/2xA2xuRw Best, Daniel Colchete On Thu, May 12, 2016 at 3:03 AM, Aki Tuomi wrote: > > > On 11.05.2016 22:45, Daniel van Ham Colchete wrote: > > I can confirm that this bug also happens on the latest version available: > > > > root at dovecot:/# dovecot --version > > 2.2.devel (ebf2e93) > > root at dovecot:/# dpkg -l|grep dovecot > > ii dovecot-core 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - core files > > ii dovecot-dbg 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - debug symbols > > ii dovecot-imapd 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - IMAP daemon > > ii dovecot-lmtpd 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - LMTP server > > ii dovecot-lucene 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - Lucene support > > ii dovecot-mysql 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - MySQL support > > ii dovecot-pop3d 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - POP3 daemon > > ii dovecot-sieve 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - Sieve filters support > > ii dovecot-solr 2:2.2.24-1~auto+23 amd64 > > secure POP3/IMAP server - Solr support > > > > > > On Wed, May 11, 2016 at 4:38 PM, Daniel van Ham Colchete < > > daniel.colchete at gmail.com> wrote: > > > >> I would like to report a bug we are seeing on an new deployment here: > >> > >> Linux: Debian Jessie, inside a Docker container > >> Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1 > >> doveconf -n: http://pastebin.com/2xA2xuRw > >> Filesystem: CephFS - Jewel > >> > >> After migrating our users from our old server to our new one we can't > >> access a few mailboxes. Backtrace follows: > >> > >> root at dovecot:/# gdb --args /usr/lib/dovecot/imap -u > >> daniel.colchete at mav.com.br > >> GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 > >> Copyright (C) 2014 Free Software Foundation, Inc. > >> License GPLv3+: GNU GPL version 3 or later < > >> http://gnu.org/licenses/gpl.html> > >> This is free software: you are free to change and redistribute it. > >> There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > >> and "show warranty" for details. > >> This GDB was configured as "x86_64-linux-gnu". > >> Type "show configuration" for configuration details. > >> For bug reporting instructions, please see: > >> . > >> Find the GDB manual and other documentation resources online at: > >> . > >> For help, type "help". > >> Type "apropos word" to search for commands related to "word"... > >> Reading symbols from /usr/lib/dovecot/imap...Reading symbols from > >> > /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done. > >> done. > >> (gdb) run > >> Starting program: /usr/lib/dovecot/imap -u daniel.colchete at mav.com.br > >> * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID > ENABLE > >> IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS > THREAD=ORDEREDSUBJECT > >> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS > >> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES > WITHIN > >> CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as > >> daniel.colchete at mav.com.br > >> a select INBOX > >> > >> Program received signal SIGSEGV, Segmentation fault. > >> strrchr () at ../sysdeps/x86_64/strrchr.S:32 > >> 32 ../sysdeps/x86_64/strrchr.S: No such file or directory. > >> (gdb) where > >> #0 strrchr () at ../sysdeps/x86_64/strrchr.S:32 > >> #1 0x00007ffff7b0a261 in sdbox_file_move (file=file at entry > =0x5555557e6530, > >> alt_path=alt_path at entry=true) at sdbox-file.c:312 > >> #2 0x00007ffff7b0cd11 in dbox_sync_file_move_if_needed > >> (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x5555557e6530) at > >> sdbox-sync.c:30 > >> #3 sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120, > >> seq=819, ctx=0x5555557e32e0) at sdbox-sync.c:58 > >> #4 sdbox_sync_add (sync_rec=0x7fffffffe570, ctx=0x5555557e32e0) at > >> sdbox-sync.c:96 > >> #5 sdbox_sync_index (ctx=0x5555557e32e0) at sdbox-sync.c:131 > >> #6 sdbox_sync_begin (mbox=mbox at entry=0x5555557ce500, flags=flags at entry > =(unknown: > >> 0), ctx_r=ctx_r at entry=0x7fffffffe680) at sdbox-sync.c:234 > >> #7 0x00007ffff7b0d07d in sdbox_sync (mbox=mbox at entry=0x5555557ce500, > >> flags=flags at entry=(unknown: 0)) at sdbox-sync.c:297 > >> #8 0x00007ffff7b0d108 in sdbox_storage_sync_init (box=0x5555557ce500, > >> flags=(MAILBOX_SYNC_FLAG_FULL_READ | > MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) > >> at sdbox-sync.c:323 > >> #9 0x00007ffff7aea85b in mailbox_sync_init (box=0x5555557ce500, > >> flags=(MAILBOX_SYNC_FLAG_FULL_READ | > MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) > >> at mail-storage.c:1664 > >> #10 0x00007ffff7aea9a7 in mailbox_sync (box=, > >> flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at > >> mail-storage.c:1712 > >> #11 0x0000555555568fa0 in select_open (readonly=false, > mailbox= >> out>, ctx=0x5555557c9928) at cmd-select.c:303 > >> #12 cmd_select_full (cmd=, readonly=) at > >> cmd-select.c:426 > >> #13 0x000055555556fdbd in command_exec (cmd=cmd at entry=0x5555557c9800) > at > >> imap-commands.c:180 > >> #14 0x000055555556e182 in client_command_input (cmd=cmd at entry > =0x5555557c9800) > >> at imap-client.c:958 > >> #15 0x000055555556e210 in client_command_input (cmd=0x5555557c9800) at > >> imap-client.c:1018 > >> #16 0x000055555556e5d5 in client_handle_next_command > >> (remove_io_r=, client=0x5555557c8c00) at > >> imap-client.c:1058 > >> #17 client_handle_input (client=0x5555557c8c00) at imap-client.c:1070 > >> #18 0x000055555556ead5 in client_input (client=0x5555557c8c00) at > >> imap-client.c:1117 > >> #19 0x00007ffff7837c3c in io_loop_call_io (io=0x5555557c96f0) at > >> ioloop.c:564 > >> #20 0x00007ffff78390a1 in io_loop_handler_run_internal > (ioloop=ioloop at entry=0x5555557966e0) > >> at ioloop-epoll.c:220 > >> #21 0x00007ffff7837cc5 in io_loop_handler_run (ioloop=ioloop at entry > =0x5555557966e0) > >> at ioloop.c:612 > >> #22 0x00007ffff7837e68 in io_loop_run (ioloop=0x5555557966e0) at > >> ioloop.c:588 > >> #23 0x00007ffff77ce0e3 in master_service_run (service=0x555555796580, > >> callback=) at master-service.c:640 > >> #24 0x0000555555561ab2 in main (argc=3, argv=0x555555796390) at > main.c:454 > >> (gdb) > >> > >> One important info is that I used to use Alternate Storage on the older > >> servers, but the not on the new one. Code seems to be trying to move a > >> message to an alternate storage that doesn't exist. > >> > >> doveadm dump dovecot.index will show that we still have the > >> ALTERNATE_STORAGE flag set: > >> > >> root at dovecot:/srv/dovecot/mail/ > >> mav.com.br/daniel.colchete/mailboxes/INBOX/dbox-Mails# doveadm dump > >> dovecot.index|grep RECORD|head > >> -- RECORDS: 932 > >> RECORD: seq=1, uid=9393, flags=0x49 (Seen Answered) > >> RECORD: seq=2, uid=9442, flags=0x40 > >> RECORD: seq=3, uid=10253, flags=0x48 (Seen) > >> RECORD: seq=4, uid=10268, flags=0x48 (Seen) > >> RECORD: seq=5, uid=10283, flags=0x48 (Seen) > >> RECORD: seq=6, uid=10309, flags=0x48 (Seen) > >> RECORD: seq=7, uid=10311, flags=0x40 > >> RECORD: seq=8, uid=10324, flags=0x49 (Seen Answered) > >> RECORD: seq=9, uid=10330, flags=0x48 (Seen) > >> > >> Thank you very much. > >> > >> Best, > >> Daniel Colchete > >> > Hi! > > Thank you for your report, we will have a look. Can you provide output > of doveconf -n please? > > --- > Aki Tuomi > Dovecot Oy > From aki.tuomi at dovecot.fi Thu May 12 10:30:50 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Thu, 12 May 2016 13:30:50 +0300 Subject: Bug Report - Crash on imap with core dump - Alternate storage related In-Reply-To: References: <57341CC4.5020507@dovecot.fi> Message-ID: <57345B5A.9060208@dovecot.fi> Ah, sorry, missed it =) Aki On 12.05.2016 12:53, Daniel van Ham Colchete wrote: > Hi Aki! > > It was already there: http://pastebin.com/2xA2xuRw > > Best, > Daniel Colchete > > On Thu, May 12, 2016 at 3:03 AM, Aki Tuomi wrote: > >> >> On 11.05.2016 22:45, Daniel van Ham Colchete wrote: >>> I can confirm that this bug also happens on the latest version available: >>> >>> root at dovecot:/# dovecot --version >>> 2.2.devel (ebf2e93) >>> root at dovecot:/# dpkg -l|grep dovecot >>> ii dovecot-core 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - core files >>> ii dovecot-dbg 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - debug symbols >>> ii dovecot-imapd 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - IMAP daemon >>> ii dovecot-lmtpd 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - LMTP server >>> ii dovecot-lucene 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - Lucene support >>> ii dovecot-mysql 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - MySQL support >>> ii dovecot-pop3d 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - POP3 daemon >>> ii dovecot-sieve 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - Sieve filters support >>> ii dovecot-solr 2:2.2.24-1~auto+23 amd64 >>> secure POP3/IMAP server - Solr support >>> >>> >>> On Wed, May 11, 2016 at 4:38 PM, Daniel van Ham Colchete < >>> daniel.colchete at gmail.com> wrote: >>> >>>> I would like to report a bug we are seeing on an new deployment here: >>>> >>>> Linux: Debian Jessie, inside a Docker container >>>> Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1 >>>> doveconf -n: http://pastebin.com/2xA2xuRw >>>> Filesystem: CephFS - Jewel >>>> >>>> After migrating our users from our old server to our new one we can't >>>> access a few mailboxes. Backtrace follows: >>>> >>>> root at dovecot:/# gdb --args /usr/lib/dovecot/imap -u >>>> daniel.colchete at mav.com.br >>>> GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 >>>> Copyright (C) 2014 Free Software Foundation, Inc. >>>> License GPLv3+: GNU GPL version 3 or later < >>>> http://gnu.org/licenses/gpl.html> >>>> This is free software: you are free to change and redistribute it. >>>> There is NO WARRANTY, to the extent permitted by law. Type "show >> copying" >>>> and "show warranty" for details. >>>> This GDB was configured as "x86_64-linux-gnu". >>>> Type "show configuration" for configuration details. >>>> For bug reporting instructions, please see: >>>> . >>>> Find the GDB manual and other documentation resources online at: >>>> . >>>> For help, type "help". >>>> Type "apropos word" to search for commands related to "word"... >>>> Reading symbols from /usr/lib/dovecot/imap...Reading symbols from >>>> >> /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done. >>>> done. >>>> (gdb) run >>>> Starting program: /usr/lib/dovecot/imap -u daniel.colchete at mav.com.br >>>> * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID >> ENABLE >>>> IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS >> THREAD=ORDEREDSUBJECT >>>> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS >>>> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES >> WITHIN >>>> CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as >>>> daniel.colchete at mav.com.br >>>> a select INBOX >>>> >>>> Program received signal SIGSEGV, Segmentation fault. >>>> strrchr () at ../sysdeps/x86_64/strrchr.S:32 >>>> 32 ../sysdeps/x86_64/strrchr.S: No such file or directory. >>>> (gdb) where >>>> #0 strrchr () at ../sysdeps/x86_64/strrchr.S:32 >>>> #1 0x00007ffff7b0a261 in sdbox_file_move (file=file at entry >> =0x5555557e6530, >>>> alt_path=alt_path at entry=true) at sdbox-file.c:312 >>>> #2 0x00007ffff7b0cd11 in dbox_sync_file_move_if_needed >>>> (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x5555557e6530) at >>>> sdbox-sync.c:30 >>>> #3 sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120, >>>> seq=819, ctx=0x5555557e32e0) at sdbox-sync.c:58 >>>> #4 sdbox_sync_add (sync_rec=0x7fffffffe570, ctx=0x5555557e32e0) at >>>> sdbox-sync.c:96 >>>> #5 sdbox_sync_index (ctx=0x5555557e32e0) at sdbox-sync.c:131 >>>> #6 sdbox_sync_begin (mbox=mbox at entry=0x5555557ce500, flags=flags at entry >> =(unknown: >>>> 0), ctx_r=ctx_r at entry=0x7fffffffe680) at sdbox-sync.c:234 >>>> #7 0x00007ffff7b0d07d in sdbox_sync (mbox=mbox at entry=0x5555557ce500, >>>> flags=flags at entry=(unknown: 0)) at sdbox-sync.c:297 >>>> #8 0x00007ffff7b0d108 in sdbox_storage_sync_init (box=0x5555557ce500, >>>> flags=(MAILBOX_SYNC_FLAG_FULL_READ | >> MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) >>>> at sdbox-sync.c:323 >>>> #9 0x00007ffff7aea85b in mailbox_sync_init (box=0x5555557ce500, >>>> flags=(MAILBOX_SYNC_FLAG_FULL_READ | >> MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) >>>> at mail-storage.c:1664 >>>> #10 0x00007ffff7aea9a7 in mailbox_sync (box=, >>>> flags=, flags at entry=MAILBOX_SYNC_FLAG_FULL_READ) at >>>> mail-storage.c:1712 >>>> #11 0x0000555555568fa0 in select_open (readonly=false, >> mailbox=>>> out>, ctx=0x5555557c9928) at cmd-select.c:303 >>>> #12 cmd_select_full (cmd=, readonly=) at >>>> cmd-select.c:426 >>>> #13 0x000055555556fdbd in command_exec (cmd=cmd at entry=0x5555557c9800) >> at >>>> imap-commands.c:180 >>>> #14 0x000055555556e182 in client_command_input (cmd=cmd at entry >> =0x5555557c9800) >>>> at imap-client.c:958 >>>> #15 0x000055555556e210 in client_command_input (cmd=0x5555557c9800) at >>>> imap-client.c:1018 >>>> #16 0x000055555556e5d5 in client_handle_next_command >>>> (remove_io_r=, client=0x5555557c8c00) at >>>> imap-client.c:1058 >>>> #17 client_handle_input (client=0x5555557c8c00) at imap-client.c:1070 >>>> #18 0x000055555556ead5 in client_input (client=0x5555557c8c00) at >>>> imap-client.c:1117 >>>> #19 0x00007ffff7837c3c in io_loop_call_io (io=0x5555557c96f0) at >>>> ioloop.c:564 >>>> #20 0x00007ffff78390a1 in io_loop_handler_run_internal >> (ioloop=ioloop at entry=0x5555557966e0) >>>> at ioloop-epoll.c:220 >>>> #21 0x00007ffff7837cc5 in io_loop_handler_run (ioloop=ioloop at entry >> =0x5555557966e0) >>>> at ioloop.c:612 >>>> #22 0x00007ffff7837e68 in io_loop_run (ioloop=0x5555557966e0) at >>>> ioloop.c:588 >>>> #23 0x00007ffff77ce0e3 in master_service_run (service=0x555555796580, >>>> callback=) at master-service.c:640 >>>> #24 0x0000555555561ab2 in main (argc=3, argv=0x555555796390) at >> main.c:454 >>>> (gdb) >>>> >>>> One important info is that I used to use Alternate Storage on the older >>>> servers, but the not on the new one. Code seems to be trying to move a >>>> message to an alternate storage that doesn't exist. >>>> >>>> doveadm dump dovecot.index will show that we still have the >>>> ALTERNATE_STORAGE flag set: >>>> >>>> root at dovecot:/srv/dovecot/mail/ >>>> mav.com.br/daniel.colchete/mailboxes/INBOX/dbox-Mails# doveadm dump >>>> dovecot.index|grep RECORD|head >>>> -- RECORDS: 932 >>>> RECORD: seq=1, uid=9393, flags=0x49 (Seen Answered) >>>> RECORD: seq=2, uid=9442, flags=0x40 >>>> RECORD: seq=3, uid=10253, flags=0x48 (Seen) >>>> RECORD: seq=4, uid=10268, flags=0x48 (Seen) >>>> RECORD: seq=5, uid=10283, flags=0x48 (Seen) >>>> RECORD: seq=6, uid=10309, flags=0x48 (Seen) >>>> RECORD: seq=7, uid=10311, flags=0x40 >>>> RECORD: seq=8, uid=10324, flags=0x49 (Seen Answered) >>>> RECORD: seq=9, uid=10330, flags=0x48 (Seen) >>>> >>>> Thank you very much. >>>> >>>> Best, >>>> Daniel Colchete >>>> >> Hi! >> >> Thank you for your report, we will have a look. Can you provide output >> of doveconf -n please? >> >> --- >> Aki Tuomi >> Dovecot Oy >> From pali.rohar at gmail.com Thu May 12 11:24:36 2016 From: pali.rohar at gmail.com (Pali =?utf-8?B?Um9ow6Fy?=) Date: Thu, 12 May 2016 13:24:36 +0200 Subject: [PATCH 0/7] Fixes for lib-mail message-address In-Reply-To: <5ECC7DDB-9497-458A-8891-569386939F9D@iki.fi> References: <1459789983-10827-1-git-send-email-pali.rohar@gmail.com> <1659571648.16346.720408d5-801e-44fb-ac8f-1881f8bd92d1.open-xchange@appsuite.open-xchange.com> <201604041935.23571@pali> <201604061629.43612@pali> <20160411104402.GR8413@pali> <5ECC7DDB-9497-458A-8891-569386939F9D@iki.fi> Message-ID: <20160512112436.GB29844@pali> On Thursday 21 April 2016 14:24:09 Timo Sirainen wrote: > On 11 Apr 2016, at 13:44, Pali Roh?r wrote: > > > > On Thursday 07 April 2016 16:11:12 Timo Sirainen wrote: > >> On 06 Apr 2016, at 17:29, Pali Roh?r wrote: > >>> > >>> On Monday 04 April 2016 19:35:22 Pali Roh?r wrote: > >>>> On Monday 04 April 2016 19:31:06 aki.tuomi at dovecot.fi wrote: > >>>>> Can you please send them directly to me, please? > >>>>> --- > >>>>> Aki Tuomi > >>>> > >>>> Sent. > >>> > >>> Received? Are patches OK now? > >> > >> It would be nice if each commit was accompanied with the corresponding unit test change in test-message-address.c. Now that the unit test changes are done in a separate commit I'm not really sure which test change is testing which commit or if some tests are missing. At least the "Quote and escape strings if needed" patch needs more tests for different kinds of escapes/atext/non-atext chars and =? in strings. > >> > > > > Ok. If you really need that I can rework my patches. But I'm thinking > > what to do with tests... I have written lot of examples and tests > > scenarios in perl for that new perl module. But dovecot test framework > > is not easy to use as perl Test::More/Test::Spec and such complicated > > tests which I have are really hard to write directory for dovecot... > > Although it would be nice to have a lot of unit tests for everything, I think it would be enough to have just some to show what the change actually fixes. Maybe even just in the git commit message. I remember that by looking at the patches I didn't completely understand what all of the changes did. Hi! Here are examples/test cases which patches fixes. Patches 3 and 4 should be self-explained... It is enough? Or do you need more detailed description about problems in parser/formatter? PATCH 1: struct: { { name = NULL, mailbox = "group", domain = NULL}, { name = NULL, mailbox = NULL, domain = NULL } } should be formatted as string: group:; before patch it was: group; PATCH 2: struct: { { name = NULL, mailbox = "", domain = NULL}, { name = NULL, mailbox = NULL, domain = NULL } } should be formatted as string: "":; before patch it was: :; PATCH 5: input string: test will be parsed as struct: { name = "test", mailbox = NULL, domain = NULL } before patch it was struct: { name = NULL, mailbox = "test", domain = NULL } PATCH 6: struct: { name = "test\"test", mailbox = "user", domain = "host" } should be formatted as string: "test\"test" before patch it was: test"test -- Pali Roh?r pali.rohar at gmail.com From yacinechaouche at yahoo.com Thu May 12 12:35:08 2016 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Thu, 12 May 2016 12:35:08 +0000 (UTC) Subject: How do I share folders ? In-Reply-To: References: <5370327.466239.1462973284288.JavaMail.yahoo.ref@mail.yahoo.com> <5370327.466239.1462973284288.JavaMail.yahoo@mail.yahoo.com> Message-ID: <111613066.1074799.1463056508598.JavaMail.yahoo@mail.yahoo.com> Thanks Steffen. I went the quick'n'dirty way described at : http://wiki2.dovecot.org/SharedMailboxes/Symlinks and I must say it works fine as far as I have tested. Here's what I did : $ cd /var/vmail/algerian-radio.dz/a.chaouche/ $ ln -s /var/vmail/backup.algerian-radio.dz/a.chaouche/ .shared.a_chaouche at backup_algerian-radio_dz <---- change "." to "_" If I didn't change the dots they would appear as subfolder separators to dovecot (Maildir++ specs ? ) If you do this then a.chaouche at algerian-radio.dz can do whatever he wants with a.chaouche at backup.algerian-radio.dz mailbox. We don't want this. We want him to only read the mail, not delete it. For this, he needs two permissions : reading of course and listing, because he needs to list the emails in order to choose which one he wants to read. If he doesn't have the listing permission then the MUA would get an error when it tries to list all the messages in the inbox. We do this by adding a dovecot-acl file in the mailbox we want to share : root at messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # cat dovecot-acl user=a.chaouche at algerian-radio.dz lr root at messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # This says that user a.chaouche at algerian-radio.dz has the "l" right (listing) and the "r" right (reading). You can make sure that it's setup as expected with the acl tool : root at messagerie[10.10.10.20] ~ # doveadm acl get -u a.chaouche at algerian-radio.dz INBOX.shared.a_chaouche at backup_algerian-radio_dz ID Global Rights user=a.chaouche at algerian-radio.dz lookup read root at messagerie[10.10.10.20] ~ # 1. No need for SQL tables 2. No need for dictionary mappings 3. No need for doveadm acl commands Yassine. ________________________________ From: Steffen Kaiser To: chaouche yacine Cc: "dovecot at dovecot.org" Sent: Wednesday, May 11, 2016 3:09 PM Subject: Re: How do I share folders ? -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 11 May 2016, chaouche yacine wrote: > namespace inbox { > prefix = INBOX. > separator = . ^^^^^^^^ > type = private > } > namespace partage { > hidden = no > location = maildir:%%h:INDEX=~/%%u > prefix = shared.%%u. ^^^^^^^^^ > separator = . > type = shared > } looks fine for me, > I don't know if it's possible to share folders with the doveadm acl command ? > root at messagerie[10.10.10.20] ~ # doveadm acl debug -u a.chaouche at algerian-radio.dz shared/a.chaouche at backup.algerian-radio.dz > doveadm(a.chaouche at algerian-radio.dz): Error: Can't open mailbox shared/a.chaouche at backup.algerian-radio.dz: Mailbox doesn't exist: shared/a.chaouche at backup.algerian-radio.dz you are using the dot as separator, that is: shared/a.chaouche at backup. should spell shared.a.chaouche at backup. However, because there are dots in the username, too, this probably won't work, because Dovecot probably picks up "a" as username. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzM9HHz1H7kL/d9rAQIGmwgAi7ga1a1Vv8HfvcAc6mIqxkBHdI5jL3MC YfMVwwuj7NJMjS8l/ouYxJCXdMn5k+sbRifwR/jyBcZtsvz3/ijkMx3RQyoq+LwZ r01Mxb4YR7ub5sNUIeIRtRcR/aNfDLNZb86PpDCHs2WKLU2atS5vVLmJWNiMaJNV MSG72OFKoLUJuRoSM26c2AH/f33WjuEinjjGMk481ezNBmZIuSWU2lJjmUXjfox9 BPzGRUkUKtlV9tkQjJUVjuwcfmvs+ChmTI3O2F37KWswP8zw0/EYXdyRpWCJso7F 8Ecrbu14cSHjisaerTQ4xko+ossPGoEE+Rh7XHuv8VDdNruur5AeVg== =DH0D -----END PGP SIGNATURE----- From yacinechaouche at yahoo.com Thu May 12 12:46:52 2016 From: yacinechaouche at yahoo.com (chaouche yacine) Date: Thu, 12 May 2016 12:46:52 +0000 (UTC) Subject: How do I share folders ? In-Reply-To: <111613066.1074799.1463056508598.JavaMail.yahoo@mail.yahoo.com> References: <5370327.466239.1462973284288.JavaMail.yahoo.ref@mail.yahoo.com> <5370327.466239.1462973284288.JavaMail.yahoo@mail.yahoo.com> <111613066.1074799.1463056508598.JavaMail.yahoo@mail.yahoo.com> Message-ID: <721347695.1139370.1463057212871.JavaMail.yahoo@mail.yahoo.com> There's also no need to create the shared namespace, so this part below is also unnecessary namespace partage { location = maildir:%%h:INDEX=~/%%u prefix = shared.%%u. separator = . type = shared } I've commented it out, restarted dovecot, sharing still works. Yassine. ----- Original Message ----- From: chaouche yacine To: "dovecot at dovecot.org" Sent: Thursday, May 12, 2016 1:35 PM Subject: Re: How do I share folders ? Thanks Steffen. I went the quick'n'dirty way described at : http://wiki2.dovecot.org/SharedMailboxes/Symlinks and I must say it works fine as far as I have tested. Here's what I did : $ cd /var/vmail/algerian-radio.dz/a.chaouche/ $ ln -s /var/vmail/backup.algerian-radio.dz/a.chaouche/ .shared.a_chaouche at backup_algerian-radio_dz <---- change "." to "_" If I didn't change the dots they would appear as subfolder separators to dovecot (Maildir++ specs ? ) If you do this then a.chaouche at algerian-radio.dz can do whatever he wants with a.chaouche at backup.algerian-radio.dz mailbox. We don't want this. We want him to only read the mail, not delete it. For this, he needs two permissions : reading of course and listing, because he needs to list the emails in order to choose which one he wants to read. If he doesn't have the listing permission then the MUA would get an error when it tries to list all the messages in the inbox. We do this by adding a dovecot-acl file in the mailbox we want to share : root at messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # cat dovecot-acl user=a.chaouche at algerian-radio.dz lr root at messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # This says that user a.chaouche at algerian-radio.dz has the "l" right (listing) and the "r" right (reading). You can make sure that it's setup as expected with the acl tool : root at messagerie[10.10.10.20] ~ # doveadm acl get -u a.chaouche at algerian-radio.dz INBOX.shared.a_chaouche at backup_algerian-radio_dz ID Global Rights user=a.chaouche at algerian-radio.dz lookup read root at messagerie[10.10.10.20] ~ # 1. No need for SQL tables 2. No need for dictionary mappings 3. No need for doveadm acl commands Yassine. ________________________________ From: Steffen Kaiser To: chaouche yacine Cc: "dovecot at dovecot.org" Sent: Wednesday, May 11, 2016 3:09 PM Subject: Re: How do I share folders ? -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 11 May 2016, chaouche yacine wrote: > namespace inbox { > prefix = INBOX. > separator = . ^^^^^^^^ > type = private > } > namespace partage { > hidden = no > location = maildir:%%h:INDEX=~/%%u > prefix = shared.%%u. ^^^^^^^^^ > separator = . > type = shared > } looks fine for me, > I don't know if it's possible to share folders with the doveadm acl command ? > root at messagerie[10.10.10.20] ~ # doveadm acl debug -u a.chaouche at algerian-radio.dz shared/a.chaouche at backup.algerian-radio.dz > doveadm(a.chaouche at algerian-radio.dz): Error: Can't open mailbox shared/a.chaouche at backup.algerian-radio.dz: Mailbox doesn't exist: shared/a.chaouche at backup.algerian-radio.dz you are using the dot as separator, that is: shared/a.chaouche at backup. should spell shared.a.chaouche at backup. However, because there are dots in the username, too, this probably won't work, because Dovecot probably picks up "a" as username. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzM9HHz1H7kL/d9rAQIGmwgAi7ga1a1Vv8HfvcAc6mIqxkBHdI5jL3MC YfMVwwuj7NJMjS8l/ouYxJCXdMn5k+sbRifwR/jyBcZtsvz3/ijkMx3RQyoq+LwZ r01Mxb4YR7ub5sNUIeIRtRcR/aNfDLNZb86PpDCHs2WKLU2atS5vVLmJWNiMaJNV MSG72OFKoLUJuRoSM26c2AH/f33WjuEinjjGMk481ezNBmZIuSWU2lJjmUXjfox9 BPzGRUkUKtlV9tkQjJUVjuwcfmvs+ChmTI3O2F37KWswP8zw0/EYXdyRpWCJso7F 8Ecrbu14cSHjisaerTQ4xko+ossPGoEE+Rh7XHuv8VDdNruur5AeVg== =DH0D -----END PGP SIGNATURE----- From info at jonkri.org Wed May 11 18:34:20 2016 From: info at jonkri.org (Jon Kristensen) Date: Wed, 11 May 2016 20:34:20 +0200 Subject: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox) In-Reply-To: (Steffen Kaiser's message of "Wed, 11 May 2016 08:04:06 +0200 (CEST)") References: <87h9e5wxbd.fsf@jonkri.org> Message-ID: <8737pov3yr.fsf@jonkri.org> Steffen Kaiser writes: > On Tue, 10 May 2016, Jon Kristensen wrote: > >> Which Pigeonhole Sieve extension should I use to process/test a >> message >> depending on whether the message is in a particular mailbox? >> (Is it the >> mboxmetadata extension by any chance? If so, I would appreciate >> if >> someone could tell me what the reason is for it not being >> enabled by >> default.) > >> The reason I want to perform this test is that I have >> configured my MTA >> to deliver "outgoing" e-mails to my "Sent" IMAP folder, and >> these >> messages arrive unread. I would like to mark them as read using >> Sieve. > > er, I guess, "message is in a particular mailbox" is meaning it > to be > spooled, or something like that, right? > > Do you use subaddressing (aka +addressing, +detail addressing)? > Then look > here: > http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Plus_Addressed_mail_filtering > > If you want to resolve the question in the second paragraphe, > search the > mailing list archive about how to get the name of the mailbox, > the > implicit keep stores the message in. I'm pretty sure, that there > was such > question lately. > > If you actually want to resolve the question in the first > paragraphe, to > check whether or not the same message has been spooled in a > folder already > (aka duplicate check), check out the duplicate extension with > ":handle" > argument. But you probably need to know the target mailbox, too. I apologize for not expressing my problem in clear enough terms. My e-mail server will put sent e-mail to the ~/.Maildir/.Sent (which is also a Maildir directory); naturally, the e-mail is first written into ~/.Maildir/.Sent/tmp, and then it's moved to ~/.Maildir/.Sent/new. After this has happened, I would like Dovecot to run a Sieve program that will discover that the e-mail is in the "Sent" folder, and mark it as seen. To explain what I'm imagining with code: if folder :is "Sent" { setflag "\\Seen"; } So I don't use subaddressing. I always know the target mailbox, but it's always "Sent" (this is configured in the MTA). I guess knowing the name of the mailbox that implicit keep stores the message in would solve my problem. I'm assuming you are referring to this: https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt And in particular this: "vnd.dovecot.default-mailbox" The default mailbox that is used by the (implicit) keep action. Normally, this will always yield "INBOX", unless this is changed in the configuration. It says that this normally will yield "INBOX". Do you, given my explanation above, think that it would yield "Sent" in my case? If not, how would I be able to use this? Thanks a million! -- Jon Kristensen (jonkri) PGP fingerprint: 0377 F488 8908 0237 A2AE C484 387E C31D DA60 1350 From info at jonkri.org Wed May 11 19:15:01 2016 From: info at jonkri.org (Jon Kristensen) Date: Wed, 11 May 2016 21:15:01 +0200 Subject: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox) In-Reply-To: <8737pov3yr.fsf@jonkri.org> (Jon Kristensen's message of "Wed, 11 May 2016 20:34:20 +0200") References: <87h9e5wxbd.fsf@jonkri.org> <8737pov3yr.fsf@jonkri.org> Message-ID: <878tzgtnii.fsf@jonkri.org> Jon Kristensen writes: > I'm assuming you are referring to this: > > https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt Is it only me, or this not documented at ? If it's not documented, should a bug be filed somewhere? -- Jon Kristensen (jonkri) PGP fingerprint: 0377 F488 8908 0237 A2AE C484 387E C31D DA60 1350 From prinaldo at robinson.it Thu May 12 16:32:31 2016 From: prinaldo at robinson.it (Paolo Rinaldo) Date: Thu, 12 May 2016 16:32:31 +0000 Subject: Dovecot 2.1.7 randomly lost emails Message-ID: <87BF8752232B33459840F73C1BE9E5CD30F7534A@ROBMAIL2.robinson.local> Hi Dovecot 2.1.7 Postfix 2.9.6 Maildir backend, emails filtered by a Postfix frontend (Sqlgrey, Amavis). Lmtp delivering emails using a simple sieve script. Ubuntu 12.04 VMware virtual server. When email are CCed to other recipients on the same domain (the same backend), sometimes they are lost. We hosts approx. 1500 mailboxes, the problem occurs one or two times a day. Recently I changed the frontend, the previous one was delivering one email for each recipient, we had a couple of error per year, we thought it was due to client-side error. The frequency increased with the new frontend, no more ?splitting? messages for more than one recipient. The following sample is from a client using a POP3 connection, leaving emails on server. the Postfix mail.log May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: from=, size=11822371, nrcpt=2 (queue active) May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: to=, relay=mail4.robinson.it[private/dovecot-lmtp], delay=0.64, delays=0.45/0/0/0.19, dsn=2.0.0, status=sent (250 2.0.0 0vNzHyptMlezHgAAk+lh+g Saved) May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: to=user-two at domain.com>, relay=mail4.robinson.it[private/dovecot-lmtp], delay=0.74, delays=0.45/0/0/0.29, dsn=2.0.0, status=sent (250 2.0.0 user-two at domain.com> 0vNzHyptMlezHgAAk+lh+g Saved) May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: removed the dovecot-info.log May 11 01:26:55 lmtp(7859, user-one at domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: sieve: msgid=<003A2D931FFF0E48A2314A1ADC87E23E8E4BA1DE at t49-ms-exmb02.INTRANET.SAKSROOT.SAKSINC.com>: stored mail into mailbox 'INBOX' May 11 01:26:55 lmtp(7859, user-two at domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: sieve: msgid=<003A2D931FFF0E48A2314A1ADC87E23E8E4BA1DE at t49-ms-exmb02.INTRANET.SAKSROOT.SAKSINC.com>: stored mail into mailbox 'INBOX' I cannot find the email 0vNzHyptMlezHgAAk+lh+g for user-two at domain.com, nor I cannot suppose problems on the client, nothing changed on the back-end. Please let me know if you need some more info or config dump. Any help will be appreciated Paolo Rinaldo Dovecot config: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.11.0-15-generic x86_64 Ubuntu 12.04.4 LTS auth_mechanisms = plain login debug_log_path = /var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no info_log_path = /var/log/dovecot/dovecot-info.log log_path = /var/log/dovecot/dovecot.log mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = INBOX.Trash autocreate2 = INBOX.Spam autocreate3 = INBOX.Sent autocreate4 = INBOX.Drafts autosubscribe = INBOX.Trash autosubscribe2 = INBOX.Spam autosubscribe3 = INBOX.Sent autosubscribe4 = INBOX.Drafts quota = maildir:User quota quota_exceeded_message = Impossibile consegnare il messaggio: spazio insufficiente sulla casella di destinazione quota_grace = 10%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /var/lib/dovecot/sieve/spam.sieve } protocols = imap pop3 lmtp sieve service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = dovecot } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl_cert = References: <87BF8752232B33459840F73C1BE9E5CD30F7534A@ROBMAIL2.robinson.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 12 May 2016, Paolo Rinaldo wrote: > the Postfix mail.log > May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: from=, size=11822371, nrcpt=2 (queue active) > May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: to=, relay=mail4.robinson.it[private/dovecot-lmtp], delay=0.64, delays=0.45/0/0/0.19, dsn=2.0.0, status=sent (250 2.0.0 0vNzHyptMlezHgAAk+lh+g Saved) > May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: to=user-two at domain.com>, relay=mail4.robinson.it[private/dovecot-lmtp], delay=0.74, delays=0.45/0/0/0.29, dsn=2.0.0, status=sent (250 2.0.0 user-two at domain.com> 0vNzHyptMlezHgAAk+lh+g Saved) > May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: removed > > the dovecot-info.log > May 11 01:26:55 lmtp(7859, user-one at domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: sieve: msgid=<003A2D931FFF0E48A2314A1ADC87E23E8E4BA1DE at t49-ms-exmb02.INTRANET.SAKSROOT.SAKSINC.com>: stored mail into mailbox 'INBOX' > May 11 01:26:55 lmtp(7859, user-two at domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: sieve: msgid=<003A2D931FFF0E48A2314A1ADC87E23E8E4BA1DE at t49-ms-exmb02.INTRANET.SAKSROOT.SAKSINC.com>: stored mail into mailbox 'INBOX' > > I cannot find the email 0vNzHyptMlezHgAAk+lh+g for user-two at domain.com, nor I cannot suppose problems on the client, nothing changed on the back-end. try the Maillog plugin first: http://wiki2.dovecot.org/Plugins/MailLog - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzVumHz1H7kL/d9rAQI3RQf/VhrIbiQd8tlrtWkPA5Awr2ZBN8QcclsP OoMGM6GAgs3ElSmGJx8VYv4nQMycjcmcvRbA0czGGoHvlO8RyL13MH3UKvLOv2wQ zotVdm4VFMM4z0jxZzoO9CWdeC0QZ0NrqQknaBxG/SoOPPlfIlmTUfG+o8O/2WNM DxU8GdxLL+vbLqsQ5Gmg1wsm2FPGlZ89CoFYR1STu5AmR1lB1SgmF+qyZ/g2ARov bU9kK7fmthQdTWamRXcw2RgASkXks11GMrb9WbEF8awdI0dCfFgNdIMmsP3xrDv6 Evi+HnQImnowS0s19B8TJkv3KMJlKPHZalTZBeI10Qw1axgxPFA3PQ== =3m11 -----END PGP SIGNATURE----- From p.heinlein at heinlein-support.de Fri May 13 10:30:00 2016 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Fri, 13 May 2016 12:30:00 +0200 Subject: autocreate: We need "auto=init"! Message-ID: <5735ACA8.8080303@heinlein-support.de> There had been several discussions about the common problem, that there's need for a autocreate-function that creates mailboxes ONLY for new users that had never been logged in before. The reason: Existing users already HAVE "special folders" with localized names. Creating "standard names" at every login will produce confusion, because then they have multiple Sent-Folders and aren't be able to delete them, because they'll always be recreated. I would love to have: 1) auto = init Create the folders ONLY if the whole Storage-Folder (e.g.: ~/Maildir or e.g. `/mdbox) is create (=creation of the INBOX itself). Don't create any folder if the main INBOX already exists. Since there has to be a hook to autoreate a non-existing INBOX it should be able to use this action to also create the folders with auto=init at the same time. 2) auto by type Create that "special folder" only if no other special folder exists. If we have "Sent" and "Sent Items" we could set "mailbox Sent { auto=create-by-type }" so this folder will only be created if no other "\Sent" folder (like "Sent Items") already exists. Dovecot could make sure, that there is AT LEAST ONE or JUST ONE of the special folder \Draft, \Trahs, \Sent. Personally: I would prefere (1) -- and that's what several users already requestet here at the list in the last two years. Function (1) is the only way to change the behavior on a complex, already running system with existing users. That's the way how to introduce new folders for new users without breaking everything for existing users. Any chance for us the get this? Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From tomi.vainio at iki.fi Fri May 13 12:05:47 2016 From: tomi.vainio at iki.fi (Tomi Vainio) Date: Fri, 13 May 2016 15:05:47 +0300 Subject: 2.2.24 ldap-client.c compile failing on Solaris 10 Message-ID: <47e1bdd9-b0ca-73fd-9f5f-1f5db120f6d1@iki.fi> Hi, I'm trying to compile 2.2.24 on Solaris 10 with SolarisStudio 12.4 compiler. Looks like BerVarray is totallly missing from Solaris ldap. BR, Tomppa Making all in lib-ldap source='ldap-client.c' object='ldap-client.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -std=c11 -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-master -I../../src/lib-ssl-iostream -I/usr/local/include -I/usr/sfw/include -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 -I/usr/local/include -c -o ldap-client.lo ldap-client.c libtool: compile: cc -std=c11 -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-master -I../../src/lib-ssl-iostream -I/usr/local/include -I/usr/sfw/include -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 -I/usr/local/include -c ldap-client.c -KPIC -DPIC -o .libs/ldap-client.o "ldap-private.h", line 63: syntax error before or at: BerVarray "ldap-client.c", line 46: void function cannot return value "ldap-client.c", line 54: void function cannot return value cc: acomp failed for ldap-client.c *** Error code 1 make: Fatal error: Command failed for target `ldap-client.lo' Current working directory /v/tomppa/src/dovecot-2.2.24/src/lib-ldap *** Error code 1 CC='cc -std=c11' CXX='CC -std=c++11' CFLAGS=-xO4 CFLAGS="${CFLAGS} -`fpversion -foption`" CPPFLAGS="-I/usr/local/include" CPPFLAGS="${CPPFLAGS} -I/usr/sfw/include" LDFLAGS="-R/usr/local/lib:/usr/sfw/lib:/usr/lib/sasl:/usr/lib/mps" LDFLAGS="${LDFLAGS} -L/usr/local/lib -L/usr/sfw/lib" LDFLAGS="${LDFLAGS} -L/usr/lib/sasl -L/usr/lib/mps" export CC CXX CFLAGS CPPFLAGS LDFLAGS ./configure \ --with-gssapi \ --with-ldap \ --with-mysql \ --with-pgsql \ --with-sqlite \ --with-solr \ --with-libwrap \ --with-ssl=openssl \ --with-ssldir=/usr/local/openssl From aki.tuomi at dovecot.fi Fri May 13 13:56:14 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Fri, 13 May 2016 16:56:14 +0300 (EEST) Subject: 2.2.24 ldap-client.c compile failing on Solaris 10 In-Reply-To: <47e1bdd9-b0ca-73fd-9f5f-1f5db120f6d1@iki.fi> References: <47e1bdd9-b0ca-73fd-9f5f-1f5db120f6d1@iki.fi> Message-ID: <200085692.389.1463147775677@appsuite-dev.open-xchange.com> > On May 13, 2016 at 3:05 PM Tomi Vainio wrote: > > > Hi, > > I'm trying to compile 2.2.24 on Solaris 10 with SolarisStudio 12.4 > compiler. Looks like BerVarray is totallly missing from Solaris ldap. > > BR, > Tomppa > > Making all in lib-ldap > source='ldap-client.c' object='ldap-client.lo' libtool=yes \ > DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ > /bin/bash ../../libtool --tag=CC --mode=compile cc -std=c11 > -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test > -I../../src/lib-settings -I../../src/lib-master > -I../../src/lib-ssl-iostream -I/usr/local/include -I/usr/sfw/include > -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 -I/usr/local/include -c > -o ldap-client.lo ldap-client.c > libtool: compile: cc -std=c11 -DHAVE_CONFIG_H -I. -I../.. > -I../../src/lib -I../../src/lib-test -I../../src/lib-settings > -I../../src/lib-master -I../../src/lib-ssl-iostream -I/usr/local/include > -I/usr/sfw/include -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 > -I/usr/local/include -c ldap-client.c -KPIC -DPIC -o .libs/ldap-client.o > "ldap-private.h", line 63: syntax error before or at: BerVarray > "ldap-client.c", line 46: void function cannot return value > "ldap-client.c", line 54: void function cannot return value > cc: acomp failed for ldap-client.c > *** Error code 1 > make: Fatal error: Command failed for target `ldap-client.lo' > Current working directory /v/tomppa/src/dovecot-2.2.24/src/lib-ldap > *** Error code 1 > > CC='cc -std=c11' > CXX='CC -std=c++11' > CFLAGS=-xO4 > CFLAGS="${CFLAGS} -`fpversion -foption`" > CPPFLAGS="-I/usr/local/include" > CPPFLAGS="${CPPFLAGS} -I/usr/sfw/include" > LDFLAGS="-R/usr/local/lib:/usr/sfw/lib:/usr/lib/sasl:/usr/lib/mps" > LDFLAGS="${LDFLAGS} -L/usr/local/lib -L/usr/sfw/lib" > LDFLAGS="${LDFLAGS} -L/usr/lib/sasl -L/usr/lib/mps" > export CC CXX CFLAGS CPPFLAGS LDFLAGS > > ./configure \ > --with-gssapi \ > --with-ldap \ > --with-mysql \ > --with-pgsql \ > --with-sqlite \ > --with-solr \ > --with-libwrap \ > --with-ssl=openssl \ > --with-ssldir=/usr/local/openssl Hi! Currently we only support openldap. However, what is the ldap library version? --- Aki Tuomi From tomi.vainio at iki.fi Fri May 13 16:32:46 2016 From: tomi.vainio at iki.fi (Tomi Vainio) Date: Fri, 13 May 2016 19:32:46 +0300 Subject: 2.2.24 ldap-client.c compile failing on Solaris 10 In-Reply-To: <200085692.389.1463147775677@appsuite-dev.open-xchange.com> References: <47e1bdd9-b0ca-73fd-9f5f-1f5db120f6d1@iki.fi> <200085692.389.1463147775677@appsuite-dev.open-xchange.com> Message-ID: <77a48db6-95a5-c1c9-f19b-f457f11500c6@iki.fi> On 13.5.2016 16.56, aki.tuomi at dovecot.fi wrote: > >> On May 13, 2016 at 3:05 PM Tomi Vainio wrote: >> >> >> Hi, >> >> I'm trying to compile 2.2.24 on Solaris 10 with SolarisStudio 12.4 >> compiler. Looks like BerVarray is totallly missing from Solaris ldap. >> >> BR, >> Tomppa >> >> Making all in lib-ldap >> source='ldap-client.c' object='ldap-client.lo' libtool=yes \ >> DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ >> /bin/bash ../../libtool --tag=CC --mode=compile cc -std=c11 >> -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test >> -I../../src/lib-settings -I../../src/lib-master >> -I../../src/lib-ssl-iostream -I/usr/local/include -I/usr/sfw/include >> -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 -I/usr/local/include -c >> -o ldap-client.lo ldap-client.c >> libtool: compile: cc -std=c11 -DHAVE_CONFIG_H -I. -I../.. >> -I../../src/lib -I../../src/lib-test -I../../src/lib-settings >> -I../../src/lib-master -I../../src/lib-ssl-iostream -I/usr/local/include >> -I/usr/sfw/include -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 >> -I/usr/local/include -c ldap-client.c -KPIC -DPIC -o .libs/ldap-client.o >> "ldap-private.h", line 63: syntax error before or at: BerVarray >> "ldap-client.c", line 46: void function cannot return value >> "ldap-client.c", line 54: void function cannot return value >> cc: acomp failed for ldap-client.c >> *** Error code 1 >> make: Fatal error: Command failed for target `ldap-client.lo' >> Current working directory /v/tomppa/src/dovecot-2.2.24/src/lib-ldap >> *** Error code 1 >> >> CC='cc -std=c11' >> CXX='CC -std=c++11' >> CFLAGS=-xO4 >> CFLAGS="${CFLAGS} -`fpversion -foption`" >> CPPFLAGS="-I/usr/local/include" >> CPPFLAGS="${CPPFLAGS} -I/usr/sfw/include" >> LDFLAGS="-R/usr/local/lib:/usr/sfw/lib:/usr/lib/sasl:/usr/lib/mps" >> LDFLAGS="${LDFLAGS} -L/usr/local/lib -L/usr/sfw/lib" >> LDFLAGS="${LDFLAGS} -L/usr/lib/sasl -L/usr/lib/mps" >> export CC CXX CFLAGS CPPFLAGS LDFLAGS >> >> ./configure \ >> --with-gssapi \ >> --with-ldap \ >> --with-mysql \ >> --with-pgsql \ >> --with-sqlite \ >> --with-solr \ >> --with-libwrap \ >> --with-ssl=openssl \ >> --with-ssldir=/usr/local/openssl > > Hi! > > Currently we only support openldap. However, what is the ldap library version? > > --- > Aki Tuomi > Ok. This is Solaris 10 native ldap which used to work or at least could compile it --with-ldap flag. If this is not supported anymore then I just drop this configure flag. Tomppa From aki.tuomi at dovecot.fi Fri May 13 16:37:11 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Fri, 13 May 2016 19:37:11 +0300 (EEST) Subject: 2.2.24 ldap-client.c compile failing on Solaris 10 In-Reply-To: <77a48db6-95a5-c1c9-f19b-f457f11500c6@iki.fi> References: <47e1bdd9-b0ca-73fd-9f5f-1f5db120f6d1@iki.fi> <200085692.389.1463147775677@appsuite-dev.open-xchange.com> <77a48db6-95a5-c1c9-f19b-f457f11500c6@iki.fi> Message-ID: <2044957428.58.1463157431550@appsuite-dev.open-xchange.com> > On May 13, 2016 at 7:32 PM Tomi Vainio wrote: > > > On 13.5.2016 16.56, aki.tuomi at dovecot.fi wrote: > > > >> On May 13, 2016 at 3:05 PM Tomi Vainio wrote: > >> > >> > >> Hi, > >> > >> I'm trying to compile 2.2.24 on Solaris 10 with SolarisStudio 12.4 > >> compiler. Looks like BerVarray is totallly missing from Solaris ldap. > >> > >> BR, > >> Tomppa > >> > >> Making all in lib-ldap > >> source='ldap-client.c' object='ldap-client.lo' libtool=yes \ > >> DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ > >> /bin/bash ../../libtool --tag=CC --mode=compile cc -std=c11 > >> -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test > >> -I../../src/lib-settings -I../../src/lib-master > >> -I../../src/lib-ssl-iostream -I/usr/local/include -I/usr/sfw/include > >> -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 -I/usr/local/include -c > >> -o ldap-client.lo ldap-client.c > >> libtool: compile: cc -std=c11 -DHAVE_CONFIG_H -I. -I../.. > >> -I../../src/lib -I../../src/lib-test -I../../src/lib-settings > >> -I../../src/lib-master -I../../src/lib-ssl-iostream -I/usr/local/include > >> -I/usr/sfw/include -xO4 -xtarget=ultra4 -xcache=64/32/4:8192/128/2 > >> -I/usr/local/include -c ldap-client.c -KPIC -DPIC -o .libs/ldap-client.o > >> "ldap-private.h", line 63: syntax error before or at: BerVarray > >> "ldap-client.c", line 46: void function cannot return value > >> "ldap-client.c", line 54: void function cannot return value > >> cc: acomp failed for ldap-client.c > >> *** Error code 1 > >> make: Fatal error: Command failed for target `ldap-client.lo' > >> Current working directory /v/tomppa/src/dovecot-2.2.24/src/lib-ldap > >> *** Error code 1 > >> > >> CC='cc -std=c11' > >> CXX='CC -std=c++11' > >> CFLAGS=-xO4 > >> CFLAGS="${CFLAGS} -`fpversion -foption`" > >> CPPFLAGS="-I/usr/local/include" > >> CPPFLAGS="${CPPFLAGS} -I/usr/sfw/include" > >> LDFLAGS="-R/usr/local/lib:/usr/sfw/lib:/usr/lib/sasl:/usr/lib/mps" > >> LDFLAGS="${LDFLAGS} -L/usr/local/lib -L/usr/sfw/lib" > >> LDFLAGS="${LDFLAGS} -L/usr/lib/sasl -L/usr/lib/mps" > >> export CC CXX CFLAGS CPPFLAGS LDFLAGS > >> > >> ./configure \ > >> --with-gssapi \ > >> --with-ldap \ > >> --with-mysql \ > >> --with-pgsql \ > >> --with-sqlite \ > >> --with-solr \ > >> --with-libwrap \ > >> --with-ssl=openssl \ > >> --with-ssldir=/usr/local/openssl > > > > Hi! > > > > Currently we only support openldap. However, what is the ldap library version? > > > > --- > > Aki Tuomi > > > Ok. This is Solaris 10 native ldap which used to work or at least could > compile it --with-ldap flag. If this is not supported anymore then I > just drop this configure flag. > > Tomppa I'll see if we could provide compability for Solaris too. It will take a while. --- Aki Tuomi From stephan at rename-it.nl Sun May 15 10:02:44 2016 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 15 May 2016 12:02:44 +0200 Subject: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox) In-Reply-To: <8737pov3yr.fsf@jonkri.org> References: <87h9e5wxbd.fsf@jonkri.org> <8737pov3yr.fsf@jonkri.org> Message-ID: <57384944.8070005@rename-it.nl> Op 5/11/2016 om 8:34 PM schreef Jon Kristensen: > Steffen Kaiser writes: > >> On Tue, 10 May 2016, Jon Kristensen wrote: >> >>> Which Pigeonhole Sieve extension should I use to process/test a >>> message depending on whether the message is in a particular mailbox? >>> (Is it the mboxmetadata extension by any chance? If so, I would >>> appreciate if someone could tell me what the reason is for it not >>> being enabled by default.) >> >>> The reason I want to perform this test is that I have configured my >>> MTA to deliver "outgoing" e-mails to my "Sent" IMAP folder, and >>> these messages arrive unread. I would like to mark them as read >>> using Sieve. >> >> er, I guess, "message is in a particular mailbox" is meaning it to be >> spooled, or something like that, right? >> >> Do you use subaddressing (aka +addressing, +detail addressing)? Then >> look here: >> http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Plus_Addressed_mail_filtering >> >> If you want to resolve the question in the second paragraphe, search >> the mailing list archive about how to get the name of the mailbox, >> the implicit keep stores the message in. I'm pretty sure, that there >> was such question lately. >> >> If you actually want to resolve the question in the first paragraphe, >> to check whether or not the same message has been spooled in a folder >> already (aka duplicate check), check out the duplicate extension with >> ":handle" argument. But you probably need to know the target mailbox, >> too. > > I apologize for not expressing my problem in clear enough terms. > > My e-mail server will put sent e-mail to the ~/.Maildir/.Sent (which > is also a Maildir directory); naturally, the e-mail is first written > into ~/.Maildir/.Sent/tmp, and then it's moved to ~/.Maildir/.Sent/new. > > After this has happened, I would like Dovecot to run a Sieve program > that will discover that the e-mail is in the "Sent" folder, and mark > it as seen. > > To explain what I'm imagining with code: > > if folder :is "Sent" { > setflag "\\Seen"; > } > > So I don't use subaddressing. I always know the target mailbox, but > it's always "Sent" (this is configured in the MTA). > > I guess knowing the name of the mailbox that implicit keep stores the > message in would solve my problem. > > I'm assuming you are referring to this: > > https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt > > > And in particular this: > > "vnd.dovecot.default-mailbox" > The default mailbox that is used by the (implicit) keep > action. > Normally, this will always yield "INBOX", unless this is > changed in the configuration. > > It says that this normally will yield "INBOX". Do you, given my > explanation above, think that it would yield "Sent" in my case? If > not, how would I be able to use this? Your explanation suggests the MTA is directly delivering messages to the Sent folder. In that case Sieve is never involved. Sieve is run only at delivery when said delivery is performed using Dovecot's LDA or LMTP agents. If you're using dovecot-lda, the target (default) mailbox can be specified using the -m option. LMTP is less flexible and can only use what's after the `recipient_delimiter' in the local part of the address as target mailbox when `lmtp_save_to_detail_mailbox=yes'. Sieve will be able to see the configured default mailbox in the environment extension you mentioned above. You can define an administrator script for marking messages as seen using the sieve_before setting. This script can also prevent the normal user script from being executed by canceling the implicit keep. Regards, Stephan. From p.heinlein at heinlein-support.de Sun May 15 11:14:16 2016 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sun, 15 May 2016 13:14:16 +0200 Subject: FTS and autoindex in a cluster? Message-ID: <57385A08.1020404@heinlein-support.de> In a dovecot replication cluster where both sides have fts_autoindex=yes we have a indexer-worker(user at example.com): Indexed 1 messages in INBOX on BOTH sides. a) Is that okay because the duplicated message will be skipped or doesn't harm? b) Or should "autoindex" only be activated on one side? Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Sun May 15 11:48:03 2016 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sun, 15 May 2016 13:48:03 +0200 Subject: indexer-worker out of memory with vsz_limit=750 MB?! Message-ID: <573861F3.3030203@heinlein-support.de> We're running 2.2.24 with fts_lucene and I wonder why indexer-working can be out-of-memory even with vsz_limit=750 MB ?! Peer May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal: pool_system_realloc(134217728): Out of memory May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7) [0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851) [0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405) [0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2) [0x7f37027b36e2] -> /usr/lib/dovecot/libdovecot.so.0(+0xa5813) [0x7f37027b3813] -> /usr/lib/dovecot/libdovecot.so.0(+0xa59e3) [0x7f37027b39e3] -> /usr/lib/dovecot/libdovecot.so.0(buffer_write+0x37) [0x7f37027b3cf5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10b7a6) [0x7f3702b617a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10c0c3) [0x7f3702b620c3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x34d) [0x7f3702b625e5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10fb49) [0x7f3702b65b49] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x38) [0x7f3702b66262] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x5ba) [0x7f3702b669b2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0xbf) [0x7f3702b59eb7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_refresh+0x1d) [0x7f3702b79e97] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x11cc33) [0x7f3702b72c33] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x127) [0x7f3702b72dba] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit+0x1f) [0x7f3702b72c91] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_index_set_header+0x68) [0x7f37014ebfd0] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_index_set_last_uid+0x3b) [0x7f37014ec00d] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x4093) [0x7f3700686093] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_get_last_uid+0x81) [0x7f37014eb3e5] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11b66) [0x7f37014f3b66] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11c40) [0x7f37014f3c40] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xea0d) [0x7f3701936a0d] -> /usr/lib/dovecot/modules/lib01_acl_plugin.so(+0x11f19) [0x7f3701b50f19] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x75) [0x7f3702aa2db4] May 15 12:14:51 dobby4 dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for user at example.com May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal: master: service(indexer-worker): child 179030 returned error 83 (Out of memory (service indexer-worker { vsz_limit=750 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From patrickdk at patrickdk.com Sun May 15 13:36:42 2016 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 15 May 2016 09:36:42 -0400 Subject: SSL on doveadm port Message-ID: <20160515093642.Horde.DC1h_Nftfliu2BkuxesDh98@mail.patrickdk.com> I have been attempting to setup ssl on the doveadm port. So far this works fine, as long as the global certificate is used. I wanted to use a different certificate, one that machines the machine name, instead of the generic mail certificate, but I have not been able to figure out how to do this. Attempts to add the certificate in the services section with the ssl = yes option, cause syntax error, and dovecot doesn't start. Creating a protocol doveadm section with it, is ignored, and setting up a local section with it in it, is ignored also. From aki.tuomi at dovecot.fi Sun May 15 14:02:29 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Sun, 15 May 2016 17:02:29 +0300 (EEST) Subject: SSL on doveadm port In-Reply-To: <20160515093642.Horde.DC1h_Nftfliu2BkuxesDh98@mail.patrickdk.com> References: <20160515093642.Horde.DC1h_Nftfliu2BkuxesDh98@mail.patrickdk.com> Message-ID: <910547469.150.1463320950439@appsuite-dev.open-xchange.com> > On May 15, 2016 at 4:36 PM Patrick Domack wrote: > > > > I have been attempting to setup ssl on the doveadm port. So far this > works fine, as long as the global certificate is used. > > I wanted to use a different certificate, one that machines the machine > name, instead of the generic mail certificate, but I have not been > able to figure out how to do this. Attempts to add the certificate in > the services section with the ssl = yes option, cause syntax error, > and dovecot doesn't start. Creating a protocol doveadm section with > it, is ignored, and setting up a local section with it in it, is > ignored also. This is because it is not supported as of yet. Aki From aki.tuomi at dovecot.fi Sun May 15 14:28:11 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Sun, 15 May 2016 17:28:11 +0300 (EEST) Subject: indexer-worker out of memory with vsz_limit=750 MB?! In-Reply-To: <573861F3.3030203@heinlein-support.de> References: <573861F3.3030203@heinlein-support.de> Message-ID: <578383843.157.1463322491701@appsuite-dev.open-xchange.com> > On May 15, 2016 at 2:48 PM Peer Heinlein wrote: > > > > We're running 2.2.24 with fts_lucene and I wonder why indexer-working > can be out-of-memory even with > > vsz_limit=750 MB > > ?! > > Peer > > > May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal: > pool_system_realloc(134217728): Out of memory > May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Error: > Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7) > [0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851) > [0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) > [0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405) > [0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2) Any chance for you to get a corefile and run gdb command 'bt full' on it? Aki From moiseev at mezonplus.ru Mon May 16 06:27:13 2016 From: moiseev at mezonplus.ru (Alexander Moisseev) Date: Mon, 16 May 2016 09:27:13 +0300 Subject: SIS: dovecot creates duplicated attachments (sometimes) Message-ID: <8067e1ed-6668-d6a7-950d-9e1d57dcafa1@mezonplus.ru> SIS deduplication was broken in 2.2.16 and has been fixed with https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c841584ea1 but still sometimes (just several times in a month or so) dovecot creates duplicated attachments. As you can see in directory listings below all attachments was created at (about) the same time, but one of them has different inode number. It easy to discover using https://github.com/moisseev/doveadm-tools/blob/master/bin/dsisck # dsisck -n mail_uid=vmail mail_attachment_dir=/vmail/attachments ==> Checking SIS... # ln -f ./30/b3/30b367c584a123eee59478adf3e4f4c9e1226545-c56eae04a67c3157287f01003d96bafd ./30/b3/30b367c584a123eee59478adf3e4f4c9e1226545-7ad0a411d17c31572b7a01003d96bafd # mkdir -p -m 700 ./64/8f/hashes && ln ./64/8f/648f5cfa27af6d20c8570fdcaeab997663e15105-55430d2cd4432c571cb600003d96bafd ./64/8f/hashes/648f5cfa27af6d20c8570fdcaeab997663e15105 # ln -f ./6f/3f/6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-b1935817ed753557546700003d96bafd ./6f/3f/6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-72da1f32e38e3557947b00003d96bafd # ln -f ./dd/d9/ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-6bf7c122fefb3157ad3500003d96bafd ./dd/d9/ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-90fd1415e7e53157112800003d96bafd -------------------------------------------------------------- Unexpected objects found: 0 Attachments processed: 43500 Different attachments with similar hashes skipped: 0 Attachments deduplicated: 3 File system blocks freed up: 1800 Attachment deduplication attempts failed: 0 Hash files created: 1 Hash files creation attempts failed: 0 Hash files re-linked: 0 Hash files re-link attempts failed: 0 Orphaned hash files: 0 Invalid hash files (not a regular file): 0 -------------------------------------------------------------- # cd /vmail/attachments # ls -li ./30/b3/ ./30/b3/hashes/ ./30/b3/: total 1996 7464083 -rw------- 1 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-7ad0a411d17c31572b7a01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bb6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bd6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bf6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-c36eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-c56eae04a67c3157287f01003d96bafd 7464084 drwx------ 2 vmail vmail 512 May 10 09:16 hashes ./30/b3/hashes/: total 332 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545 # ls -li ./6f/3f/ ./6f/3f/hashes/ ./6f/3f/: total 644 15088414 -rw------- 1 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-72da1f32e38e3557947b00003d96bafd 15088422 -rw------- 2 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-b1935817ed753557546700003d96bafd 15088448 drwx------ 2 vmail vmail 512 May 13 11:23 hashes ./6f/3f/hashes/: total 320 15088422 -rw------- 2 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe # ls -li ./dd/d9/ ./dd/d9/hashes/ ./dd/d9/: total 748 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-68f7c122fefb3157ad3500003d96bafd 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-6bf7c122fefb3157ad3500003d96bafd 80547 -rw------- 1 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-90fd1415e7e53157112800003d96bafd 80549 drwx------ 2 vmail vmail 512 May 10 18:19 hashes ./dd/d9/hashes/: total 248 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b # doveconf -n # 2.2.24 (a82c823): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.14 (099a97c) # OS: FreeBSD 10.3-RELEASE i386 auth_default_realm = example.com auth_mechanisms = digest-md5 cram-md5 plain apop doveadm_password = # hidden, use -P to show it first_valid_gid = 1000 first_valid_uid = 1000 lda_mailbox_autosubscribe = yes listen = * mail_attachment_dir = /vmail/attachments mail_gid = vmail mail_home = /vmail/%d/%n mail_location = mdbox:~/mdbox mail_plugins = quota zlib acl mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = mdbox:%%h/mdbox:INDEXPVT=~/mdbox/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Archives { special_use = \Archive } mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=plain username_format=%n /usr/local/etc/dovecot/dovecot.auth/%d.passwd driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/vmail/shared-mailboxes.db antispam_backend = mailtrain antispam_mail_notspam = report_ham antispam_mail_sendmail = /usr/local/libexec/dovecot/dovecot-lda antispam_mail_sendmail_args = -d;spam at example.com;-m antispam_mail_spam = report_spam antispam_spam = Junk antispam_trash = Trash;train_ham;train_prob;train_spam quota = dict:User quota::file:%h/dovecot-quota quota_rule = *:storage=2G quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_status_nouser = DUNNO quota_status_success = DUNNO sieve_after = /usr/local/etc/dovecot/sieve/sieve.after sieve_before = /usr/local/etc/dovecot/sieve/sieve.before sieve_vacation_min_period = 0 zlib_save = gz zlib_save_level = 3 } postmaster_address = postmaster at example.com protocols = imap lmtp sieve pop3 quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { user = vmail } } service config { unix_listener config { mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 0 } process_limit = 200 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service pop3-login { inet_listener pop3s { port = 0 } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { user = postfix } } ssl_cert = Hello! At the start dovecot complains all occurrences |mail_plugins settings |in all configs. For example 20-lmtp.conf contents: |protocol lmtp { mail_plugins = $mail_plugins quota } # doveconf | grep '\(mail_plugin_dir\|mail_plugins\)' doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/20-lmtp.conf line 2: Unknown setting: mail_plugins || ||| If I insert directly those strings in dovecot.conf and comment the !include conf.d/*.conf , I have no errors: # doveconf | grep '\(mail_plugin_dir\|mail_plugins\)' mail_plugin_dir = /usr/lib64/dovecot mail_plugins = mail_plugins = " quota" My settings: |# dovecot --version 2.2.24 (a82c823) # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql sqlite Passdb: checkpassword ldap passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql| |# ls -l /usr/lib64/dovecot/ | grep quota -rwxr-xr-x 1 root root 990 may 12 18:52 lib10_quota_plugin.la -rwxr-xr-x 1 root root 78696 may 12 18:52 lib10_quota_plugin.so -rwxr-xr-x 1 root root 1061 may 12 18:52 lib11_imap_quota_plugin.la -rwxr-xr-x 1 root root 11584 may 12 18:52 lib11_imap_quota_plugin.so -rwxr-xr-x 1 root root 1026 may 12 18:52 lib20_quota_clone_plugin.la -rwxr-xr-x 1 root root 8992 may 12 18:52 lib20_quota_clone_plugin.so| Thank you in advance. From dion at dion.org.ua Mon May 16 08:41:54 2016 From: dion at dion.org.ua (Dmitry Nezhevenko) Date: Mon, 16 May 2016 11:41:54 +0300 Subject: fts-lucene crashes on dovecot-2.2.24 Message-ID: <20160516084154.GA24247@debpad.local> Hi, I've tried to upgrade to dovecot 2.2.24. It seems that lucene reindex issue should be fixed. But instead of this dovecot indexer-worker crashes very often for me. I'm attaching backtrace. Please let me know if you need backtrace from binary without optimizations. -- WBR, Dmitry -------------- next part -------------- #0 rescan_clear_unseen_mailbox (rescan_ctx=rescan_ctx at entry=0x0, vname=0x55c7212fd120 "blogs", hdr=hdr at entry=0x7ffddf5e3120) at lucene-wrapper.cc:830 box = 0x7ffddf5e30c0 metadata = {guid = void, virtual_size = 140728350945568, physical_size = 94313743381008, first_save_date = 94313186852864, cache_fields = 0x16008bb1a6be7800, precache_fields = (MAIL_FETCH_STREAM_BODY | MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_NUL_STATE | MAIL_FETCH_IMAP_ENVELOPE | MAIL_FETCH_FROM_ENVELOPE | MAIL_FETCH_STORAGE_ID | MAIL_FETCH_UIDL_BACKEND | MAIL_FETCH_MAILBOX_NAME | MAIL_FETCH_GUID | MAIL_FETCH_BODY_SNIPPET | unknown: 536872960), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 3747492128)} #1 0x00007f83c892bc9e in rescan_clear_unseen_mailboxes (index=index at entry=0x55c7212bf210, rescan_ctx=rescan_ctx at entry=0x0) at lucene-wrapper.cc:863 iter = 0x55c7212fbf80 info = hdr = {last_indexed_uid = 0, settings_checksum = 2205687194, unused = 0} ns = 0x55c721268870 vname = #2 0x00007f83c892cbf4 in lucene_settings_check (index=0x55c7212bf210) at lucene-wrapper.cc:429 set_checksum = ret = #3 lucene_index_build_init (index=0x55c7212bf210) at lucene-wrapper.cc:448 lock_path = 0x55c72124a5c0 "/home/dion/mail/lucene-indexes/write.lock" st = {st_dev = 472446402651, st_ino = 0, st_nlink = 0, st_mode = 119, st_uid = 124, st_gid = 3747492592, __pad0 = 32765, st_rdev = 140204016954144, st_size = 6, st_blksize = 12528, st_blocks = 94313743630096, st_atim = {tv_sec = 94313743584528, tv_nsec = 94313743567592}, st_mtim = {tv_sec = 1585420663957387264, tv_nsec = 94313727365448}, st_ctim = {tv_sec = 5, tv_nsec = 94313743126928}, __glibc_reserved = {94313743380704, 140728350946096, 94313743584528}} exists = #4 0x00007f83c892af6e in fts_backend_lucene_update_set_build_key (_ctx=0x55c7212d0890, key=0x7ffddf5e3330) at fts-backend-lucene.c:366 ctx = 0x55c7212d0890 backend = 0x55c7212bf0e0 __FUNCTION__ = "fts_backend_lucene_update_set_build_key" #5 0x00007f83c9e17204 in fts_backend_update_set_build_key (ctx=0x55c7212d0890, key=key at entry=0x7ffddf5e3330) at fts-api.c:175 __FUNCTION__ = "fts_backend_update_set_build_key" #6 0x00007f83c9e1857d in fts_build_mail_header (block=0x7ffddf5e3310, block=0x7ffddf5e3310, ctx=0x7ffddf5e3360) at fts-build-mail.c:174 hdr = 0x55c7212ecae8 key = {uid = 157102, type = FTS_BACKEND_BUILD_KEY_HDR, part = 0x55c72124a560, hdr_name = 0x55c7212f0de0 "Return-path", body_content_type = 0x0, body_content_disposition = 0x0} ret = #7 fts_build_mail_real (mail=0x55c7212ca260, update_ctx=0x55c7212d0890) at fts-build-mail.c:548 block = {part = 0x55c72124a560, hdr = 0x55c7212ecae8, data = 0x6e0000005b , size = 0} ret = input = 0x55c7212ec500 raw_block = {part = 0x55c72124a560, hdr = 0x55c7212f0d10, data = 0x0, size = 0} skip_body = false ctx = {mail = 0x55c7212ca260, update_ctx = 0x55c7212d0890, content_type = 0x0, content_disposition = 0x0, body_parser = 0x0, word_buf = 0x0, pending_input = 0x0, cur_user_lang = 0x0} prev_part = parser = 0x55c7212ec700 decoder = 0x55c7212ecad0 parts = 0x55c700000007 body_part = false body_added = false binary_body = error = 0x265ae #8 fts_build_mail (update_ctx=0x55c7212d0890, mail=mail at entry=0x55c7212ca260) at fts-build-mail.c:594 _data_stack_cur_id = 6 #9 0x00007f83c9e1e93e in fts_mail_index (_mail=0x55c7212ca260) at fts-storage.c:501 ft = 0x55c7212d0a80 flist = 0x55c721281568 #10 fts_mail_precache (_mail=0x55c7212ca260) at fts-storage.c:520 _data_stack_cur_id = 5 mail = 0x55c7212ca260 fmail = ft = #11 0x00007f83cad49ab9 in mail_precache (mail=0x55c7212ca260) at mail.c:401 _data_stack_cur_id = 4 p = 0x55c7212ca260 #12 0x000055c720177508 in index_mailbox_precache (conn=0x55c721258550, box=0x55c7212c1630) at master-connection.c:93 box_vname = 0x55c7212c1930 "trash" percentage_str = "\340R%!" percentage_sent = 0 ret = 0 storage = username = 0x55c72127d278 "dion" status = {messages = 5, recent = 0, unseen = 0, uidvalidity = 1459164554, uidnext = 157107, first_unseen_seq = 0, first_recent_uid = 157102, last_cached_seq = 0, highest_modseq = 0, highest_pvt_modseq = 0, keywords = 0x0, permanent_flags = 0, permanent_keywords = 0, allow_new_keywords = 0, nonpermanent_modseqs = 0, no_modseq_tracking = 0, have_guids = 1, have_save_guids = 1, have_only_guid128 = 0} ctx = 0x55c7212cfdb0 seq = percentage = trans = 0x55c7212d1170 search_args = 0x0 mail = 0x55c7212ca260 metadata = {guid = '\000' , virtual_size = 0, physical_size = 0, first_save_date = 0, cache_fields = 0x0, precache_fields = (MAIL_FETCH_STREAM_HEADER | MAIL_FETCH_STREAM_BODY | MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_VIRTUAL_SIZE | MAIL_FETCH_UIDL_BACKEND | MAIL_FETCH_GUID | MAIL_FETCH_POP3_ORDER), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 0)} counter = 0 max = 5 #13 index_mailbox (what=, max_recent_msgs=, mailbox=, user=, conn=0x55c721258550) at master-connection.c:186 box = 0x55c7212c1630 status = {messages = 3747493248, recent = 32765, unseen = 107, uidvalidity = 32765, uidnext = 8192, first_unseen_seq = 0, first_recent_uid = 8240, last_cached_seq = 0, highest_modseq = 128, highest_pvt_modseq = 755914244609, keywords = 0x10, permanent_flags = MAIL_FLAGGED, permanent_keywords = 0, allow_new_keywords = 0, nonpermanent_modseqs = 0, no_modseq_tracking = 0, have_guids = 0, have_save_guids = 0, have_only_guid128 = 0} errstr = ret = 0 ns = path = 0x55c721249f60 "/home/dion/mail/mailboxes/trash/dbox-Mails" error = MAIL_ERROR_NOQUOTA sync_flags = #14 master_connection_input_line (line=, conn=0x55c721258550) at master-connection.c:228 str = max_recent_msgs = 0 ret = args = input = {module = 0x55c720177bcf "mail", service = 0x55c720177bc0 "indexer-worker", username = 0x55c721247060 "dion", session_id = 0x0, session_id_prefix = 0x55c72124706b "JxcJB7KEOVcVWQAA0vq6Vw", local_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0, debug = 0} service_user = 0x55c72125a9f0 user = 0x55c72127d180 error = 0x7f83caabf8db "H\205\300x t.H\001\305H)\303u\334H\203\304\b1\300[]A\\A]\303f\017\037\204" #15 master_connection_input (conn=0x55c721258550) at master-connection.c:265 _data_stack_cur_id = 3 line = #16 0x00007f83caaa454c in io_loop_call_io (io=0x55c721258590) at ioloop.c:564 ioloop = 0x55c72124f700 t_id = 2 __FUNCTION__ = "io_loop_call_io" #17 0x00007f83caaa59e1 in io_loop_handler_run_internal (ioloop=ioloop at entry=0x55c72124f700) at ioloop-epoll.c:220 ctx = 0x55c7212552e0 io = tv = {tv_sec = 2147483, tv_usec = 0} events_count = msecs = ret = 1 i = 0 j = call = __FUNCTION__ = "io_loop_handler_run_internal" #18 0x00007f83caaa45d5 in io_loop_handler_run (ioloop=ioloop at entry=0x55c72124f700) at ioloop.c:612 No locals. #19 0x00007f83caaa4778 in io_loop_run (ioloop=0x55c72124f700) at ioloop.c:588 __FUNCTION__ = "io_loop_run" #20 0x00007f83caa3a653 in master_service_run (service=0x55c72124f5a0, callback=) at master-service.c:640 No locals. #21 0x000055c720176f37 in main (argc=1, argv=0x55c72124f390) at indexer-worker.c:78 storage_service_flags = (MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT | MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP | MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP | MAIL_STORAGE_SERVICE_FLAG_NO_IDLE_TIMEOUT) c = From bmdivakar at gmail.com Mon May 16 08:54:56 2016 From: bmdivakar at gmail.com (Divakar BM) Date: Mon, 16 May 2016 08:54:56 +0000 Subject: Dovecot configuration: Home dir not found Message-ID: Hi, I've my mail solution configured on Server-A with webmail, Dovecot and Postfix. I'm trying to deploy only Dovecot and postfix on Server-B to reduce IMAP/POP3/SMTP load on server-A. Please note server-B does not mailboxes. It should be mapped to mailboxes on server-A. While the LDAP query from Server-B to Server-A is going successfully, the imap service on Server-B is throwing the below error. imap(user at example.com): Debug: Home dir not found: /mail/vmail/**/ I guess imap is looking for *home dir* in server-B. How do I tel the imap that home dir is on server-A and not server-B. Please let me know if there exists a configuration parameter for the above.. Thanks Divakar From adrian.minta at gmail.com Mon May 16 09:14:31 2016 From: adrian.minta at gmail.com (Adrian M) Date: Mon, 16 May 2016 12:14:31 +0300 Subject: Dovecot configuration: Home dir not found In-Reply-To: References: Message-ID: mount the /mail/vmail/**/ on server-B via nfs. On Mon, May 16, 2016 at 11:54 AM, Divakar BM wrote: > Hi, > I've my mail solution configured on Server-A with webmail, Dovecot and > Postfix. I'm trying to deploy only Dovecot and postfix on Server-B to > reduce IMAP/POP3/SMTP load on server-A. Please note server-B does not > mailboxes. It should be mapped to mailboxes on server-A. While the LDAP > query from Server-B to Server-A is going successfully, the imap service on > Server-B is throwing the below error. > > imap(user at example.com): Debug: Home dir not found: /mail/vmail/**/ > > I guess imap is looking for *home dir* in server-B. How do I tel the imap > that home dir is on server-A and not server-B. > > Please let me know if there exists a configuration parameter for the > above.. > > Thanks > Divakar > From bmdivakar at gmail.com Mon May 16 10:18:20 2016 From: bmdivakar at gmail.com (Divakar BM) Date: Mon, 16 May 2016 10:18:20 +0000 Subject: Dovecot configuration: Home dir not found In-Reply-To: References: Message-ID: Thanks for suggestion. Starting NFS would consume more cpu on Server-A as I need to start mountd, nfsd (couple more) and also sync from nfs client would further degrade system performance. Is there way of telling dovecot on system-B that *Maildir* is located on server-A and map it to home dir attribute... On Mon, May 16, 2016 at 2:44 PM Adrian M wrote: > mount the /mail/vmail/**/ on server-B via nfs. > > On Mon, May 16, 2016 at 11:54 AM, Divakar BM wrote: > >> Hi, >> I've my mail solution configured on Server-A with webmail, Dovecot and >> Postfix. I'm trying to deploy only Dovecot and postfix on Server-B to >> reduce IMAP/POP3/SMTP load on server-A. Please note server-B does not >> mailboxes. It should be mapped to mailboxes on server-A. While the LDAP >> query from Server-B to Server-A is going successfully, the imap service on >> Server-B is throwing the below error. >> >> imap(user at example.com): Debug: Home dir not found: /mail/vmail/**/ >> >> I guess imap is looking for *home dir* in server-B. How do I tel the imap > > >> that home dir is on server-A and not server-B. >> >> Please let me know if there exists a configuration parameter for the >> above.. >> >> Thanks >> Divakar >> > From adrian.minta at gmail.com Mon May 16 11:28:37 2016 From: adrian.minta at gmail.com (Adrian M) Date: Mon, 16 May 2016 14:28:37 +0300 Subject: Dovecot configuration: Home dir not found In-Reply-To: References: Message-ID: Starting NFS will consume more cpu on system-A, but you will free a lot more CPU cycles by serving half of the IMAP clients directly on system-B. Another option is to use an imap proxy on system-B. On Mon, May 16, 2016 at 1:18 PM, Divakar BM wrote: > Thanks for suggestion. Starting NFS would consume more cpu on Server-A as > I need to start mountd, nfsd (couple more) and also sync from nfs client > would further degrade system performance. Is there way of telling dovecot > on system-B that *Maildir* is located on server-A and map it to home dir > attribute... > > > On Mon, May 16, 2016 at 2:44 PM Adrian M wrote: > >> mount the /mail/vmail/**/ on server-B via nfs. >> >> On Mon, May 16, 2016 at 11:54 AM, Divakar BM wrote: >> >>> Hi, >>> I've my mail solution configured on Server-A with webmail, Dovecot and >>> Postfix. I'm trying to deploy only Dovecot and postfix on Server-B to >>> reduce IMAP/POP3/SMTP load on server-A. Please note server-B does not >>> mailboxes. It should be mapped to mailboxes on server-A. While the LDAP >>> query from Server-B to Server-A is going successfully, the imap service >>> on >>> Server-B is throwing the below error. >>> >>> imap(user at example.com): Debug: Home dir not found: /mail/vmail/**/ >>> >>> I guess imap is looking for *home dir* in server-B. How do I tel the imap >> >> >>> that home dir is on server-A and not server-B. >>> >>> Please let me know if there exists a configuration parameter for the >>> above.. >>> >>> Thanks >>> Divakar >>> >> From info at jonkri.org Mon May 16 19:39:31 2016 From: info at jonkri.org (Jon Kristensen) Date: Mon, 16 May 2016 21:39:31 +0200 Subject: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox) In-Reply-To: <57384944.8070005@rename-it.nl> (Stephan Bosch's message of "Sun, 15 May 2016 12:02:44 +0200") References: <87h9e5wxbd.fsf@jonkri.org> <8737pov3yr.fsf@jonkri.org> <57384944.8070005@rename-it.nl> Message-ID: <8760ud23nw.fsf@jonkri.org> Stephan Bosch writes: > Op 5/11/2016 om 8:34 PM schreef Jon Kristensen: >> Steffen Kaiser writes: >> >>> On Tue, 10 May 2016, Jon Kristensen wrote: >>> >>>> Which Pigeonhole Sieve extension should I use to process/test >>>> a >>>> message depending on whether the message is in a particular >>>> mailbox? >>>> (Is it the mboxmetadata extension by any chance? If so, I >>>> would >>>> appreciate if someone could tell me what the reason is for it >>>> not >>>> being enabled by default.) >>> >>>> The reason I want to perform this test is that I have >>>> configured my >>>> MTA to deliver "outgoing" e-mails to my "Sent" IMAP folder, >>>> and >>>> these messages arrive unread. I would like to mark them as >>>> read >>>> using Sieve. >>> >>> er, I guess, "message is in a particular mailbox" is meaning >>> it to be >>> spooled, or something like that, right? >>> >>> Do you use subaddressing (aka +addressing, +detail >>> addressing)? Then >>> look here: >>> http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Plus_Addressed_mail_filtering >>> >>> If you want to resolve the question in the second paragraphe, >>> search >>> the mailing list archive about how to get the name of the >>> mailbox, >>> the implicit keep stores the message in. I'm pretty sure, that >>> there >>> was such question lately. >>> >>> If you actually want to resolve the question in the first >>> paragraphe, >>> to check whether or not the same message has been spooled in a >>> folder >>> already (aka duplicate check), check out the duplicate >>> extension with >>> ":handle" argument. But you probably need to know the target >>> mailbox, >>> too. >> >> I apologize for not expressing my problem in clear enough >> terms. >> >> My e-mail server will put sent e-mail to the ~/.Maildir/.Sent >> (which >> is also a Maildir directory); naturally, the e-mail is first >> written >> into ~/.Maildir/.Sent/tmp, and then it's moved to >> ~/.Maildir/.Sent/new. >> >> After this has happened, I would like Dovecot to run a Sieve >> program >> that will discover that the e-mail is in the "Sent" folder, and >> mark >> it as seen. >> >> To explain what I'm imagining with code: >> >> if folder :is "Sent" { >> setflag "\\Seen"; >> } >> >> So I don't use subaddressing. I always know the target mailbox, >> but >> it's always "Sent" (this is configured in the MTA). >> >> I guess knowing the name of the mailbox that implicit keep >> stores the >> message in would solve my problem. >> >> I'm assuming you are referring to this: >> >> https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt >> >> >> And in particular this: >> >> "vnd.dovecot.default-mailbox" >> The default mailbox that is used by the (implicit) keep >> action. >> Normally, this will always yield "INBOX", unless this >> is >> changed in the configuration. >> >> It says that this normally will yield "INBOX". Do you, given my >> explanation above, think that it would yield "Sent" in my case? >> If >> not, how would I be able to use this? > > Your explanation suggests the MTA is directly delivering > messages to the > Sent folder. In that case Sieve is never involved. Sieve is run > only at > delivery when said delivery is performed using Dovecot's LDA or > LMTP agents. > > If you're using dovecot-lda, the target (default) mailbox can be > specified using the -m option. LMTP is less flexible and can > only use > what's after the `recipient_delimiter' in the local part of the > address > as target mailbox when `lmtp_save_to_detail_mailbox=yes'. > > Sieve will be able to see the configured default mailbox in the > environment extension you mentioned above. You can define an > administrator script for marking messages as seen using the > sieve_before > setting. This script can also prevent the normal user script > from being > executed by canceling the implicit keep. > > Regards, > > Stephan. Thanks for your clarifications! Just to confirm: Since my MTA is delivering my messages to ~/Mailbox/.Sent directly, neither Sieve or Dovecot's LDA or LMTP facilities can/will be involved, and I can't really solve my original problem (having messages in the Sent folder being marked automatically as read, on the server) using any of these utilities? Or can perhaps dovecot-lda/lmtp be used to process these "Sent" messages? Any suggestions on elegant approaches that would solve that particular problem would be very welcome. -- Jon Kristensen (jonkri) PGP fingerprint: 0377 F488 8908 0237 A2AE C484 387E C31D DA60 1350 From skdovecot at smail.inf.fh-brs.de Tue May 17 06:13:48 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 17 May 2016 08:13:48 +0200 (CEST) Subject: Dovecot configuration: Home dir not found In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 16 May 2016, Divakar BM wrote: > Thanks for suggestion. Starting NFS would consume more cpu on Server-A as I > need to start mountd, nfsd (couple more) and also sync from nfs client > would further degrade system performance. Is there way of telling dovecot > on system-B that *Maildir* is located on server-A and map it to home dir > attribute... How do you access the messages located on server A from the Dovecot on server B? > On Mon, May 16, 2016 at 2:44 PM Adrian M wrote: > >> mount the /mail/vmail/**/ on server-B via nfs. >> >> On Mon, May 16, 2016 at 11:54 AM, Divakar BM wrote: >> >>> Hi, >>> I've my mail solution configured on Server-A with webmail, Dovecot and >>> Postfix. I'm trying to deploy only Dovecot and postfix on Server-B to >>> reduce IMAP/POP3/SMTP load on server-A. Please note server-B does not >>> mailboxes. It should be mapped to mailboxes on server-A. While the LDAP >>> query from Server-B to Server-A is going successfully, the imap service on >>> Server-B is throwing the below error. >>> >>> imap(user at example.com): Debug: Home dir not found: /mail/vmail/**/ >>> >>> I guess imap is looking for *home dir* in server-B. How do I tel the imap >> >> >>> that home dir is on server-A and not server-B. >>> >>> Please let me know if there exists a configuration parameter for the >>> above.. >>> >>> Thanks >>> Divakar >>> >> > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVzq2nHz1H7kL/d9rAQIrKAf/YL89nuXVzc+AL6rhrnYIl+EIFFowf7x0 xUTj4CDK2c04FnSxLDovpFu2cmeBmGtHBG3RTsL0yB0BrjEEBfF3SAKIkuwXE9wc 6bC7I0MoWgwBvp6poFlYmqyJv8t7WRNi5d0vxLwND27q9Qc1Ml4EJDOL/iDUFFvC V7zcXDAMCK/FmGf5+OO0Dqp7T/MKWTQnBaDh/w3OrsAJaBPXpKqOPd1L2duRfRGt 4UWx/C6JQVJ6Wcn7g1JurfsEhcLx14nZNSI1QSuxM7Y29ChOu40Yq3/nOaSwh2+W SvNBRbLrdh4YhtWQT5ZsUtnYsFnUJZfbqotDkqWn6OOf70pOmPKcpw== =0dCo -----END PGP SIGNATURE----- From graf at slackware.su Tue May 17 07:16:55 2016 From: graf at slackware.su (Graf) Date: Tue, 17 May 2016 10:16:55 +0300 Subject: Unknown setting: mail_plugins In-Reply-To: <573984C2.7000109@slackware.su> References: <573984C2.7000109@slackware.su> Message-ID: <573AC567.70000@slackware.su> The cause was in lack of character "}" in file /etc/dovecot/conf.d/15-mailboxes.conf. Thanks to all. 16.05.2016 11:28, Graf ?????: > Hello! > > At the start dovecot complains all occurrences |mail_plugins settings > |in all configs. > For example 20-lmtp.conf contents: > > |protocol lmtp { > mail_plugins = $mail_plugins quota > } > > > # doveconf | grep '\(mail_plugin_dir\|mail_plugins\)' > doveconf: Fatal: Error in configuration file > /etc/dovecot/conf.d/20-lmtp.conf line 2: Unknown setting: mail_plugins > || > ||| > If I insert directly those strings in dovecot.conf and comment the > !include conf.d/*.conf , I have no errors: > > # doveconf | grep '\(mail_plugin_dir\|mail_plugins\)' > mail_plugin_dir = /usr/lib64/dovecot > mail_plugins = > mail_plugins = " quota" > > My settings: > > |# dovecot --version > 2.2.24 (a82c823) > > # dovecot --build-options > Build options: ioloop=epoll notify=inotify ipv6 openssl > io_block_size=8192 > Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail > SQL drivers: mysql sqlite > Passdb: checkpassword ldap passwd passwd-file shadow sql > Userdb: checkpassword ldap nss passwd prefetch passwd-file sql| > > |# ls -l /usr/lib64/dovecot/ | grep quota > -rwxr-xr-x 1 root root 990 may 12 18:52 lib10_quota_plugin.la > -rwxr-xr-x 1 root root 78696 may 12 18:52 lib10_quota_plugin.so > -rwxr-xr-x 1 root root 1061 may 12 18:52 lib11_imap_quota_plugin.la > -rwxr-xr-x 1 root root 11584 may 12 18:52 lib11_imap_quota_plugin.so > -rwxr-xr-x 1 root root 1026 may 12 18:52 lib20_quota_clone_plugin.la > -rwxr-xr-x 1 root root 8992 may 12 18:52 lib20_quota_clone_plugin.so| > > Thank you in advance. From andrew at dcs.bbk.ac.uk Tue May 17 12:57:38 2016 From: andrew at dcs.bbk.ac.uk (Andrew Watkins) Date: Tue, 17 May 2016 13:57:38 +0100 Subject: Errors on Cross-Protocol Locking nbmand (ZFS Solaris) Message-ID: Hello, I get the following errors when I move large e-mails (i.e. about 15MB) into mail files. May 17 13:26:48 mailer3 dovecot: [ID 583609 mail.error] imap(andrew): Error: fdatasync() failed with mbox file /home/andrew/Mail/nbmandON/mbox: Permission denied May 17 13:26:48 mailer3 dovecot: [ID 583609 mail.error] imap(andrew): Error: ftruncate() failed with mbox file /home/andrew/Mail/nbmandON/mbox: Permission denied May 17 13:26:48 mailer3 dovecot: [ID 583609 mail.error] imap(andrew): Error: close() failed with mbox file /home/andrew/Mail/nbmandON/mbox: Permission denied Background I am using Solaris 11 ZFS & SMB for my users home files with Solaris SMB protocol mandatory locking (nbmand=on) If I turn nbmand=off then I do not get any errors, so it is to do with SMB locking. I know this may not be a dovecot error, but any pointers why it is doing this so that I can report it to Oracle. I tried creating a small C program to mimic the same problem but no luck, since I know it would make it easier to get Oracle to fix it. # /usr/local/dovecot/sbin/dovecot --version 2.2.24 (a82c823) Thanks for now, Andrew -- Andrew Watkins * Birkbeck, University of London * Computer Science * * http://notallmicrosoft.blogspot.com * * UKOUG Systems SIG Chair * * UKOUG Tech committee * * tel: 020 7631 6720 * From mail at tomsommer.dk Tue May 17 14:14:31 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Tue, 17 May 2016 16:14:31 +0200 Subject: New feature: HTTP API In-Reply-To: <2143737481.6249.1459863315740@appsuite-dev.open-xchange.com> References: <289895264.13993.22ea9490-41a0-4846-81ec-d4706f3bbfc6.open-xchange@appsuite.open-xchange.com> <1988331466.16402.720408d5-801e-44fb-ac8f-1881f8bd92d1.open-xchange@appsuite.open-xchange.com> <853408930.4781.1459838699525@appsuite-dev.open-xchange.com> <868b534e29558c25035e536a604e7544@tomsommer.dk> <173870327.5192.1459848290040@appsuite-dev.open-xchange.com> <2143737481.6249.1459863315740@appsuite-dev.open-xchange.com> Message-ID: On 2016-04-05 15:35, aki.tuomi at dovecot.fi wrote: >> On April 5, 2016 at 12:24 PM aki.tuomi at dovecot.fi wrote: >> >> >> >> > On April 5, 2016 at 9:57 AM Tom Sommer wrote: >> > >> > >> > On 2016-04-05 08:44, aki.tuomi at dovecot.fi wrote: >> > >> On April 5, 2016 at 9:27 AM Tom Sommer wrote: >> > >> >> > >> >> > >> On 2016-04-04 20:22, aki.tuomi at dovecot.fi wrote: >> > >> >> > >> > Since the API is still a little beta, it will not return anything in >> > >> > case >> > >> > doveadm does not like the arguments (that is, calls i_fatal). Also, >> > >> > success is >> > >> > often indicated with 200 OK along with [["doveadmResponse",[],"c01"]]. >> > >> >> > >> I don't see the "doveadm proxy kick" command in the list? >> > > >> > > Not all commands are included, unfortunately. Also, the list only >> > > shows commands that are available with doveadm as per configuration. >> > >> > Hrmm, is it possible to add a non-http inet_listener to doveadm? And >> > feed it commands directly? >> > >> > I really need a remote proxy kick for my director. >> >> Unfortunately it's not possible with 2.2.23. It has been fixed >> internally and will eventually pop out to public repo and into some >> release. >> >> Aki > > Fixed hopefully in > https://github.com/dovecot/core/commit/76537b1991e7815c7a867a997f7fa2b3c17412d4 Works like a charm. Thank you so much. From tom at talpey.com Tue May 17 16:01:58 2016 From: tom at talpey.com (Tom Talpey) Date: Tue, 17 May 2016 12:01:58 -0400 Subject: Errors on Cross-Protocol Locking nbmand (ZFS Solaris) In-Reply-To: References: Message-ID: <0d8279a4-b2ef-e7ad-793d-d2cdc3e4090e@talpey.com> On 5/17/2016 8:57 AM, Andrew Watkins wrote: > ... > Background I am using Solaris 11 ZFS & SMB for my users home files with > Solaris SMB protocol mandatory locking (nbmand=on) > > If I turn nbmand=off then I do not get any errors, so it is to do with > SMB locking. Posix doesn't have mandatory locking, so it doesn't surprise me that you get issues from Dovecot. Can you just leave nbmand=off? If not, have you tried Dovecot's other locking methods, such as dotlock? From andrew at dcs.bbk.ac.uk Tue May 17 16:40:52 2016 From: andrew at dcs.bbk.ac.uk (Andrew Watkins) Date: Tue, 17 May 2016 17:40:52 +0100 Subject: Errors on Cross-Protocol Locking nbmand (ZFS Solaris) In-Reply-To: <0d8279a4-b2ef-e7ad-793d-d2cdc3e4090e@talpey.com> References: <0d8279a4-b2ef-e7ad-793d-d2cdc3e4090e@talpey.com> Message-ID: <40295294-a8aa-37ef-cd8d-033cb429445d@dcs.bbk.ac.uk> On 05/17/16 17:01, Tom Talpey wrote: > On 5/17/2016 8:57 AM, Andrew Watkins wrote: >> ... >> Background I am using Solaris 11 ZFS & SMB for my users home files with >> Solaris SMB protocol mandatory locking (nbmand=on) >> >> If I turn nbmand=off then I do not get any errors, so it is to do with >> SMB locking. > > Posix doesn't have mandatory locking, so it doesn't surprise me that > you get issues from Dovecot. Can you just leave nbmand=off? If not, > have you tried Dovecot's other locking methods, such as dotlock? For got to mention that the user files are on NFS file server and are being severed to NFS clients (including the Dovecot server) and SMB clients, so really I should have "Cross-Protocol Locking" on. I may end up switching it off nbmand=off since it is very unlikely that any file will be accessed from both a NFS client and a SMB client at the same time, but I thought I would look into it to see if the problem could be fixed in Solaris. I will also look at the dotlock options. Thanks, Andrew -- Andrew Watkins * Birkbeck, University of London * Computer Science * * http://notallmicrosoft.blogspot.com * * UKOUG Systems SIG Chair * * UKOUG Tech committee * * tel: 020 7631 6720 * From kremels at kreme.com Tue May 17 18:26:34 2016 From: kremels at kreme.com (@lbutlr) Date: Tue, 17 May 2016 12:26:34 -0600 Subject: Reloading dovecot and postfix after updates Message-ID: <446A3A6D-4C65-469D-B008-F42B05DE8E61@kreme.com> Recently someone used webmin to update dovecot, which went fine. However, after the update no one could login to the mail server because of the mismatch in the versions between what was installed and what postfix was expecting. Other that bapping the person on the head and saying ?don?t do that? is there anyway to force a reload of postfix after dovecot reloads? Like adding a postfix reload to the dovecot rc.d script or someway of linking them so that if either version changes they both get reloaded? (I don?t even use webmin myself, so maybe it?s even possible through it?) I mean, it?s unlikely to happen again at this point, but still? -- "...and Digby considered how much he liked salt..." From aki.tuomi at dovecot.fi Tue May 17 18:56:23 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Tue, 17 May 2016 21:56:23 +0300 (EEST) Subject: Reloading dovecot and postfix after updates In-Reply-To: <446A3A6D-4C65-469D-B008-F42B05DE8E61@kreme.com> References: <446A3A6D-4C65-469D-B008-F42B05DE8E61@kreme.com> Message-ID: <1252712957.1082.1463511384116@appsuite-dev.open-xchange.com> > On May 17, 2016 at 9:26 PM "@lbutlr" wrote: > > > Recently someone used webmin to update dovecot, which went fine. However, after the update no one could login to the mail server because of the mismatch in the versions between what was installed and what postfix was expecting. > > Other that bapping the person on the head and saying ?don?t do that? is there anyway to force a reload of postfix after dovecot reloads? Like adding a postfix reload to the dovecot rc.d script or someway of linking them so that if either version changes they both get reloaded? > > (I don?t even use webmin myself, so maybe it?s even possible through it?) > > I mean, it?s unlikely to happen again at this point, but still? > > -- > "...and Digby considered how much he liked salt..." This sounds very local requirement. You should be able to do the required changes in postfix rc file. --- Aki Tuomi From tss at iki.fi Tue May 17 23:01:27 2016 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 May 2016 02:01:27 +0300 Subject: indexer-worker out of memory with vsz_limit=750 MB?! In-Reply-To: <578383843.157.1463322491701@appsuite-dev.open-xchange.com> References: <573861F3.3030203@heinlein-support.de> <578383843.157.1463322491701@appsuite-dev.open-xchange.com> Message-ID: <703DC4E7-E512-45DF-97FC-F90275CB0099@iki.fi> > On 15 May 2016, at 17:28, aki.tuomi at dovecot.fi wrote: > >> >> On May 15, 2016 at 2:48 PM Peer Heinlein wrote: >> >> >> >> We're running 2.2.24 with fts_lucene and I wonder why indexer-working >> can be out-of-memory even with >> >> vsz_limit=750 MB >> >> ?! >> >> Peer >> >> >> May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Fatal: >> pool_system_realloc(134217728): Out of memory >> May 15 12:14:51 dobby4 dovecot: indexer-worker(user at example.com): Error: >> Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7) >> [0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851) >> [0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) >> [0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405) >> [0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2) > > Any chance for you to get a corefile and run gdb command 'bt full' on it? Setting CORE_OUTOFMEM=1 environment will produce core dump on out of memory (a huge one of course). From tom at foscore.com Wed May 18 08:40:37 2016 From: tom at foscore.com (Tom) Date: Wed, 18 May 2016 04:40:37 -0400 Subject: doveconf filter broken? Message-ID: <3d8fa4da-6500-42b4-ace7-227046423bbf@foscore.com> I'm running dovecot 2.0.9 According to the man page, the command: # doveconf -f service=imap is supposed to show just the settings for imap. Is this correct? When I issue the command it shows everything. Why? From aki.tuomi at dovecot.fi Wed May 18 09:05:39 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 18 May 2016 12:05:39 +0300 Subject: doveconf filter broken? In-Reply-To: <3d8fa4da-6500-42b4-ace7-227046423bbf@foscore.com> References: <3d8fa4da-6500-42b4-ace7-227046423bbf@foscore.com> Message-ID: <573C3063.9070707@dovecot.fi> On 18.05.2016 11:40, Tom wrote: > I'm running dovecot 2.0.9 > According to the man page, the command: > # doveconf -f service=imap > is supposed to show just the settings for imap. Is this correct? > > When I issue the command it shows everything. Why? Hi! it seems to be broken, we are going to fix it eventually. Aki From mail at tomsommer.dk Wed May 18 09:44:54 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Wed, 18 May 2016 11:44:54 +0200 Subject: mailbox.auto ignores dovecot-uidlist.lock Message-ID: I'm trying to lock down a maildir from modifications using dovecot-uidlist.lock, but when a user with mailbox.auto = create logs in, then the folder is created regardless of dovecot-uidlist.lock existing or not. Is there no way to prevent dovecot from touching anything in a maildir folder? -- Tom From aki.tuomi at dovecot.fi Wed May 18 09:52:23 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Wed, 18 May 2016 12:52:23 +0300 Subject: mailbox.auto ignores dovecot-uidlist.lock In-Reply-To: References: Message-ID: <573C3B57.3030909@dovecot.fi> On 18.05.2016 12:44, Tom Sommer wrote: > I'm trying to lock down a maildir from modifications using > dovecot-uidlist.lock, but when a user with mailbox.auto = create logs > in, then the folder is created regardless of dovecot-uidlist.lock > existing or not. > > Is there no way to prevent dovecot from touching anything in a maildir > folder? > Could you provide more information, like: what you are trying to achieve what did you do what you configuration looks like what version you are using Aki From neural at dirtybass.fm Wed May 18 11:33:02 2016 From: neural at dirtybass.fm (NeuralNET) Date: Wed, 18 May 2016 12:33:02 +0100 Subject: intermittant passdb error Message-ID: <573C52EE.7010201@dirtybass.fm> I'm having great difficulty debugging this problem.. I'm having intermittant problems connecting to dovecot. This started a couple of days ago following an apt-get update on the server. I'm running Ubuntu 14.04, dovecot 2.2.9. Here's my dovecot -n --- # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.19.0-59-generic x86_64 Ubuntu 14.04.4 LTS auth_mechanisms = plain login mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve_dir = ~/sieve } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } user = $default_internal_user } ssl = required ssl_cert = , method=PLAIN, rip=10.0.0.3, lip=10.0.0.99, TLS, session=<********> May 18 12:08:27 box dovecot: imap-login: Disconnected (auth failed, 3 attempts in 20 secs): user=, method=PLAIN, rip=10.0.0.3, lip=10.0.0.99, TLS, session=<********> May 18 12:08:50 box dovecot: imap-login: Disconnected (auth failed, 3 attempts in 12 secs): user=, method=PLAIN, rip=10.0.0.3, lip=10.0.0.99, TLS, session=<********> --- do I conclude that method=plain is failing, whereas method=login succeeds ? I tried setting auth_debug_password=yes and get the following; --- May 18 12:26:42 box dovecot: auth: Debug: client in: CONT#0111#011A********== (previous base64 data may contain sensitive data) May 18 12:26:42 box dovecot: auth-worker(12214): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth May 18 12:26:42 box dovecot: auth-worker(12214): Debug: pam(expert,10.0.0.3): lookup service=dovecot May 18 12:26:42 box dovecot: auth-worker(12214): Debug: pam(expert,10.0.0.3): #1/1 style=1 msg=Password: May 18 12:26:42 box dovecot: auth-worker(12211): pam(username,10.0.0.3): pam_authenticate() failed: Authentication failure (password mismatch?) (given password: ******) May 18 12:26:44 box dovecot: auth: Debug: client passdb out: FAIL#0112#011user=username --- Any assistance debugging this further is greatly recieved! Thanks. From mail at tomsommer.dk Wed May 18 13:24:32 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Wed, 18 May 2016 15:24:32 +0200 Subject: mailbox.auto ignores dovecot-uidlist.lock In-Reply-To: <573C3B57.3030909@dovecot.fi> References: <573C3B57.3030909@dovecot.fi> Message-ID: <897a96155e9582824b7eb853731d0b73@tomsommer.dk> On 2016-05-18 11:52, Aki Tuomi wrote: > On 18.05.2016 12:44, Tom Sommer wrote: >> I'm trying to lock down a maildir from modifications using >> dovecot-uidlist.lock, but when a user with mailbox.auto = create logs >> in, then the folder is created regardless of dovecot-uidlist.lock >> existing or not. >> >> Is there no way to prevent dovecot from touching anything in a maildir >> folder? >> > Could you provide more information, like: > > what you are trying to achieve > what did you do > what you configuration looks like > what version you are using I'm trying to move all files and folder from /foo.com/bar/ to /foo.com/bar/Maildir without moving .dovecot.svbin and .dovecot.sieve etc. (homedir files) So I wanted to lock the folder, kick the user and the move the files/folders, but the user logs in just after I kick him and so the auto-folders are created and my migration script fails, locking the folder with dovecot-uidlist.lock and maildirlock does not work in this case. From sylvain.allemand at math.u-bordeaux.fr Wed May 18 15:32:31 2016 From: sylvain.allemand at math.u-bordeaux.fr (Sylvain Allemand) Date: Wed, 18 May 2016 17:32:31 +0200 Subject: synchronous replication based on priority Message-ID: <573C8B0F.3080902@math.u-bordeaux.fr> Hi, i see in this page (http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.html) : "Optionally the replication plugin can also support synchronous replication of new mail deliveries. In this way it connects to replication-notify UNIX socket, tells it to replicate the user with sync (=highest) priority and waits until it is done or replication_sync_timeout occurs. The IMAP/LMTP client won't see an "OK" reply until the mail is replicated (or the replication has failed). The synchronous replication probably adds a noticeable delay, so it might not be acceptable for IMAP, but might be for LMTP." is this fonctionnality effective for LMTP ? If so, where is the config ? The wiki says that replication is asynchronous. Thanks for help, Best regards -- Sylvain Allemand Cellule Informatique Institut de Math?matiques de Bordeaux - UMR5251 05 40 00 68 86 www.math.u-bordeaux.fr/imb/cellule From neural at dirtybass.fm Wed May 18 16:04:30 2016 From: neural at dirtybass.fm (NeuralNET) Date: Wed, 18 May 2016 17:04:30 +0100 Subject: intermittant passdb error In-Reply-To: <573C52EE.7010201@dirtybass.fm> References: <573C52EE.7010201@dirtybass.fm> Message-ID: <573C928E.3000503@dirtybass.fm> Problem worked-around.. After working on this for a couple of days, I finally found this post; http://askubuntu.com/questions/760026/problem-with-pam-authorization-in-dovecot in /etc/dovecot/conf.d/10-master.conf uncomment this line; #default_vsz_limit = 256M and increase the vsz_limit. e.g. default_vsz_limit = 512M Seems like a bug/change in the recent build of PAM. Hopefully this helps someone else doing a bunch of fruitless seaching / debugging :) From ifrpilot541 at gmail.com Wed May 18 22:07:06 2016 From: ifrpilot541 at gmail.com (Mike Ponto) Date: Wed, 18 May 2016 17:07:06 -0500 Subject: passwd mismatch Message-ID: <573CE78A.2010803@gmail.com> I have been trying to follow the "instructions" at "http://blog.iandreev.com/?p=1975#dovecot" and they seem to be pretty good. everything works until I get to the part were I test the dovecot installation and it keeps failing. I get an AUTH fail error at the "telnet localhost 110" login commands I am trying to setup 3 domains: domain1.us domain2.com domain3.com I have 4 users entered in the /etc/dovecot/users file I followed the instructions for creating an encrypted password but don't think it is correct and I don't have the knowledge to say why. Just doesn't feel right. I admit I am over my head and the learning curve just got exponential - any help at all would be really appreciated. System: QEMU/KVM vitrual install Centos 7.01 basic install 4096 G Ram 4 proc 120 G HD unit is behind an IPCop firewall with port forwarding. internal IP 192.168.1.155 external IP on request have username in /etc/dovecot/users as such: mynamehere at mydomainhere.us: Turned on debuggin and log file shows: May 18 16:45:02 auth: Debug: auth client connected (pid=5639) May 18 16:45:16 auth: Debug: client in: AUTH 1 PLAIN service=pop3 secured session=2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 lport=110 rport=33742 $ May 18 16:45:16 auth: Debug: passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): lookup: user=mynamehere at mydomainhere.us file=/etc/dovecot/users May 18 16:45:16 auth: Info: passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): Password mismatch May 18 16:45:18 auth: Debug: client passdb out: FAIL 1 user=mynameherel at mydomainhere.us May 18 16:45:20 pop3-login: Info: Aborted login (auth failed, 1 attempts in 4 secs): user=, method=PLAIN, rip=::1, lip=::1, secured, session=<2pP4xSQzzgAAAAAA$ From edgar at pettijohn-web.com Wed May 18 22:31:26 2016 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Wed, 18 May 2016 17:31:26 -0500 Subject: passwd mismatch In-Reply-To: <573CE78A.2010803@gmail.com> References: <573CE78A.2010803@gmail.com> Message-ID: On 05/18/16 17:07, Mike Ponto wrote: > I have been trying to follow the "instructions" at > "http://blog.iandreev.com/?p=1975#dovecot" and they seem to be pretty > good. everything works until I get to the part were I test the > dovecot installation and it keeps failing. > I get an AUTH fail error at the "telnet localhost 110" login commands > > I am trying to setup 3 domains: > domain1.us > domain2.com > domain3.com > I have 4 users entered in the /etc/dovecot/users file > I followed the instructions for creating an encrypted password but > don't think it is correct and I don't have the knowledge to say why. > Just doesn't feel right. > > I admit I am over my head and the learning curve just got exponential > - any help at all would be really appreciated. > > > System: > QEMU/KVM vitrual install > Centos 7.01 > basic install > 4096 G Ram > 4 proc > 120 G HD > > unit is behind an IPCop firewall with port forwarding. > internal IP 192.168.1.155 > external IP on request > > have username in /etc/dovecot/users as such: > mynamehere at mydomainhere.us: > > Turned on debuggin and log file shows: > May 18 16:45:02 auth: Debug: auth client connected (pid=5639) > May 18 16:45:16 auth: Debug: client in: AUTH 1 PLAIN > service=pop3 secured > session=2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 > lport=110 rport=33742 $ > May 18 16:45:16 auth: Debug: > passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): > lookup: user=mynamehere at mydomainhere.us file=/etc/dovecot/users > May 18 16:45:16 auth: Info: > passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): > Password mismatch > May 18 16:45:18 auth: Debug: client passdb out: FAIL 1 > user=mynameherel at mydomainhere.us > May 18 16:45:20 pop3-login: Info: Aborted login (auth failed, 1 > attempts in 4 secs): user=, method=PLAIN, > rip=::1, lip=::1, secured, session=<2pP4xSQzzgAAAAAA$ doveconf -n would have been helpful man doveadm-pw then edit your passdb { scheme = SHA512-CRYPT } telnet is user-error prone. try mutt -f imap://user at example.com@mail.example.com From lists at lazygranch.com Wed May 18 22:47:38 2016 From: lists at lazygranch.com (lists at lazygranch.com) Date: Wed, 18 May 2016 15:47:38 -0700 Subject: passwd mismatch In-Reply-To: <573CE78A.2010803@gmail.com> References: <573CE78A.2010803@gmail.com> Message-ID: <20160518224738.5468245.31513.3157@lazygranch.com> I used his blog for freebsd. He gets you really close, but I found a few lines deprecated, but IIRC in postfix mostly.? The sha512 scheme works fine. But are you sure you did the postmap earlier in the instructions? What I like about his scheme is it is a stick shift versus automatic. That is there is no port 80 means to add or delete users. This is fine for a mail server with a small number of addresses. I prefer this set up rather than having jerks/bots trying to login to some web based email account management panel. But you need to follow the instructions exactly else you will grind the gears. ?As I add accounts, I occasional miss the postmap. Again to avoid hackers, I didn't add the roundcube feature. ?(Technically you would say this reduces the attack surface.) ? Original Message ? From: Mike Ponto Sent: Wednesday, May 18, 2016 3:09 PM To: dovecot at dovecot.org Subject: passwd mismatch I have been trying to follow the "instructions" at "http://blog.iandreev.com/?p=1975#dovecot" and they seem to be pretty good. everything works until I get to the part were I test the dovecot installation and it keeps failing. I get an AUTH fail error at the "telnet localhost 110" login commands I am trying to setup 3 domains: domain1.us domain2.com domain3.com I have 4 users entered in the /etc/dovecot/users file I followed the instructions for creating an encrypted password but don't think it is correct and I don't have the knowledge to say why. Just doesn't feel right. I admit I am over my head and the learning curve just got exponential - any help at all would be really appreciated. System: QEMU/KVM vitrual install Centos 7.01 basic install 4096 G Ram 4 proc 120 G HD unit is behind an IPCop firewall with port forwarding. internal IP 192.168.1.155 external IP on request have username in /etc/dovecot/users as such: mynamehere at mydomainhere.us: Turned on debuggin and log file shows: May 18 16:45:02 auth: Debug: auth client connected (pid=5639) May 18 16:45:16 auth: Debug: client in: AUTH 1 PLAIN service=pop3 secured session=2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 lport=110 rport=33742 $ May 18 16:45:16 auth: Debug: passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): lookup: user=mynamehere at mydomainhere.us file=/etc/dovecot/users May 18 16:45:16 auth: Info: passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): Password mismatch May 18 16:45:18 auth: Debug: client passdb out: FAIL 1 user=mynameherel at mydomainhere.us May 18 16:45:20 pop3-login: Info: Aborted login (auth failed, 1 attempts in 4 secs): user=, method=PLAIN, rip=::1, lip=::1, secured, session=<2pP4xSQzzgAAAAAA$ From ifrpilot541 at gmail.com Wed May 18 23:04:31 2016 From: ifrpilot541 at gmail.com (Mike Ponto) Date: Wed, 18 May 2016 18:04:31 -0500 Subject: passwd mismatch In-Reply-To: References: <573CE78A.2010803@gmail.com> Message-ID: <573CF4FF.9090003@gmail.com> here is the "doveconf -n" as requested [root at email conf.d]# doveconf -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_debug = yes disable_plaintext_auth = no log_path = /var/log/dovecot.log mail_gid = 1001 mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~ mail_privileged_group = vpostfix mail_uid = 1001 mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } service auth { unix_listener /var/spool/postfix/private/auth { group = vpostfix mode = 0666 user = vpostfix } unix_listener auth-userdb { group = vpostfix mode = 0600 user = vpostfix } } ssl = no ssl_cert = > > On 05/18/16 17:07, Mike Ponto wrote: >> I have been trying to follow the "instructions" at >> "http://blog.iandreev.com/?p=1975#dovecot" and they seem to be pretty >> good. everything works until I get to the part were I test the >> dovecot installation and it keeps failing. >> I get an AUTH fail error at the "telnet localhost 110" login commands >> >> I am trying to setup 3 domains: >> domain1.us >> domain2.com >> domain3.com >> I have 4 users entered in the /etc/dovecot/users file >> I followed the instructions for creating an encrypted password but >> don't think it is correct and I don't have the knowledge to say why. >> Just doesn't feel right. >> >> I admit I am over my head and the learning curve just got exponential >> - any help at all would be really appreciated. >> >> >> System: >> QEMU/KVM vitrual install >> Centos 7.01 >> basic install >> 4096 G Ram >> 4 proc >> 120 G HD >> >> unit is behind an IPCop firewall with port forwarding. >> internal IP 192.168.1.155 >> external IP on request >> >> have username in /etc/dovecot/users as such: >> mynamehere at mydomainhere.us: >> >> Turned on debuggin and log file shows: >> May 18 16:45:02 auth: Debug: auth client connected (pid=5639) >> May 18 16:45:16 auth: Debug: client in: AUTH 1 PLAIN >> service=pop3 secured >> session=2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 >> lport=110 rport=33742 $ >> May 18 16:45:16 auth: Debug: >> passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): >> lookup: user=mynamehere at mydomainhere.us file=/etc/dovecot/users >> May 18 16:45:16 auth: Info: >> passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): >> Password mismatch >> May 18 16:45:18 auth: Debug: client passdb out: FAIL 1 >> user=mynameherel at mydomainhere.us >> May 18 16:45:20 pop3-login: Info: Aborted login (auth failed, 1 >> attempts in 4 secs): user=, method=PLAIN, >> rip=::1, lip=::1, secured, session=<2pP4xSQzzgAAAAAA$ > doveconf -n would have been helpful > > man doveadm-pw > > then edit your passdb { scheme = SHA512-CRYPT } > telnet is user-error prone. try mutt -f > imap://user at example.com@mail.example.com From ifrpilot541 at gmail.com Thu May 19 00:10:07 2016 From: ifrpilot541 at gmail.com (Mike Ponto) Date: Wed, 18 May 2016 19:10:07 -0500 Subject: passwd mismatch In-Reply-To: References: <573CE78A.2010803@gmail.com> Message-ID: <573D045F.5090100@gmail.com> This got it!! Edgar you were right about the doveadm usage. the tutorial leaves out the part about using a username. That is what was bugging me and I never saw it. Thanks for the extra eyes. dud - your are my hero for the 3 hours. On 05/18/2016 05:31 PM, Edgar Pettijohn wrote: > > > On 05/18/16 17:07, Mike Ponto wrote: >> I have been trying to follow the "instructions" at >> "http://blog.iandreev.com/?p=1975#dovecot" and they seem to be pretty >> good. everything works until I get to the part were I test the >> dovecot installation and it keeps failing. >> I get an AUTH fail error at the "telnet localhost 110" login commands >> >> I am trying to setup 3 domains: >> domain1.us >> domain2.com >> domain3.com >> I have 4 users entered in the /etc/dovecot/users file >> I followed the instructions for creating an encrypted password but >> don't think it is correct and I don't have the knowledge to say why. >> Just doesn't feel right. >> >> I admit I am over my head and the learning curve just got exponential >> - any help at all would be really appreciated. >> >> >> System: >> QEMU/KVM vitrual install >> Centos 7.01 >> basic install >> 4096 G Ram >> 4 proc >> 120 G HD >> >> unit is behind an IPCop firewall with port forwarding. >> internal IP 192.168.1.155 >> external IP on request >> >> have username in /etc/dovecot/users as such: >> mynamehere at mydomainhere.us: >> >> Turned on debuggin and log file shows: >> May 18 16:45:02 auth: Debug: auth client connected (pid=5639) >> May 18 16:45:16 auth: Debug: client in: AUTH 1 PLAIN >> service=pop3 secured >> session=2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 >> lport=110 rport=33742 $ >> May 18 16:45:16 auth: Debug: >> passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): >> lookup: user=mynamehere at mydomainhere.us file=/etc/dovecot/users >> May 18 16:45:16 auth: Info: >> passwd-file(mynamehere at mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): >> Password mismatch >> May 18 16:45:18 auth: Debug: client passdb out: FAIL 1 >> user=mynameherel at mydomainhere.us >> May 18 16:45:20 pop3-login: Info: Aborted login (auth failed, 1 >> attempts in 4 secs): user=, method=PLAIN, >> rip=::1, lip=::1, secured, session=<2pP4xSQzzgAAAAAA$ > doveconf -n would have been helpful > > man doveadm-pw > > then edit your passdb { scheme = SHA512-CRYPT } > telnet is user-error prone. try mutt -f > imap://user at example.com@mail.example.com From nick at cpanel.net Thu May 19 00:54:12 2016 From: nick at cpanel.net (J. Nick Koston) Date: Wed, 18 May 2016 14:54:12 -1000 Subject: BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish) Message-ID: <980567F6-A127-4B14-88BC-A5F18537A798@cpanel.net> Hi, It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn?t received the X (done) response. If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. Thanks J. Nick Koston # dovecot --version 2.2.24 (a82c823) dovecot-2.2.24-1.x86_64 F\n is in packet 1 X\n is in packet 2 STRACE of an the unexpected EPIPE: [pid 34753] read(4, "F\n", 524288) = 2 [pid 34753] write(1, "F\n", 2) = 2 [pid 34752] <... epoll_wait resumed> {{EPOLLIN, {u32=42323904, u64=42323904}}}, 3, 3350) = 1 [pid 34753] select(8, [0 4], NULL, NULL, {21600, 0} [pid 34752] read(10, "F\n", 1361) = 2 [pid 34752] write(9, "X\n", 2 [pid 34753] <... select resumed> ) = 1 (in [0], left {21599, 999925}) [pid 34752] <... write resumed> ) = 2 [pid 34753] read(0, [pid 34752] epoll_ctl(6, EPOLL_CTL_DEL, 10, {0, {u32=42323904, u64=42323904}} [pid 34753] <... read resumed> "X\n", 524288) = 2 [pid 34752] <... epoll_ctl resumed> ) = 0 [pid 34753] write(4, "X\n", 2 [pid 34752] close(9) = 0 [pid 34753] <... write resumed> ) = 2 [pid 34752] close(10 [pid 34753] select(8, [0 4], NULL, NULL, {21600, 0} [pid 34752] <... close resumed> ) = 0 [pid 34753] <... select resumed> ) = 2 (in [0 4], left {21599, 999998}) [pid 34752] epoll_wait(6, [pid 34753] read(4, "X\n", 524288) = 2 [pid 34753] write(1, "X\n", 2) = -1 EPIPE (Broken pipe) STRACE where its in one packet: F\nX\n is in a single packet [pid 34833] select(8, [0 4], NULL, NULL, {21600, 0}) = 1 (in [4], left {21599, 999966}) [pid 34833] read(4, "F\nX\n", 524288) = 4 [pid 34833] write(1, "F\nX\n", 4) = 4 [pid 34833] select(8, [0 4], NULL, NULL, {21600, 0} [pid 34832] <... epoll_wait resumed> {{EPOLLIN, {u32=38678464, u64=38678464}}}, 3, 3889) = 1 [pid 34833] <... select resumed> ) = 1 (in [4], left {21599, 999572}) [pid 34832] read(10, "F\nX\n", 1361) = 4 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3569 bytes Desc: not available URL: From skdovecot at smail.inf.fh-brs.de Thu May 19 06:07:57 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 19 May 2016 08:07:57 +0200 (CEST) Subject: mailbox.auto ignores dovecot-uidlist.lock In-Reply-To: <897a96155e9582824b7eb853731d0b73@tomsommer.dk> References: <573C3B57.3030909@dovecot.fi> <897a96155e9582824b7eb853731d0b73@tomsommer.dk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 18 May 2016, Tom Sommer wrote: > On 2016-05-18 11:52, Aki Tuomi wrote: >> On 18.05.2016 12:44, Tom Sommer wrote: >>> I'm trying to lock down a maildir from modifications using >>> dovecot-uidlist.lock, but when a user with mailbox.auto = create logs >>> in, then the folder is created regardless of dovecot-uidlist.lock >>> existing or not. >>> >>> Is there no way to prevent dovecot from touching anything in a maildir >>> folder? >>> >> Could you provide more information, like: >> >> what you are trying to achieve >> what did you do >> what you configuration looks like >> what version you are using > > I'm trying to move all files and folder from /foo.com/bar/ to > /foo.com/bar/Maildir without moving .dovecot.svbin and .dovecot.sieve etc. > (homedir files) > > So I wanted to lock the folder, kick the user and the move the files/folders, > but the user logs in just after I kick him and so the auto-folders are > created and my migration script fails, locking the folder with > dovecot-uidlist.lock and maildirlock does not work in this case. Check out: http://wiki2.dovecot.org/Authentication/RestrictAccess?highlight=%28user%29|%28deny%29 "deny passwd-file" add user into this file, doveadm auth cache flush, # force reread credentials kick user migrate remove user from file either restart with next user or flush cache - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVz1YPXz1H7kL/d9rAQK4nwf/aBuFTOhHS+hrGZS8ivTp/zOB4bfXYqaj LgqnGbivJq9ehkETpFe1ltl1LGs4HcesuEkF7/F4Qr2+HA5OfQ5UQ0HW76GBE9CG RBuQbeQPVWjMX7pjaToDMNkuZAt7KMTAft6uMQXuaSVAo31yR5MZhyBol1b7gfHU BB7MHErGJ2EufGRA2tfNuwnmEzpcIBDtVnUSscQkbozSM6sqkSbenMa/q4iZ0Lqp dw3GF18WZYEMSbYzOLUHpgK/WGfG2rGu8DxFJwB5xNo6tOs2hALSjLcXs7Zowptp Pms9qEAhoj95wjguCR/CYrB8jkoUazDzEiSNsZsG6kc/Uz7asqWgdA== =+v7x -----END PGP SIGNATURE----- From mail at tomsommer.dk Thu May 19 08:37:05 2016 From: mail at tomsommer.dk (Tom Sommer) Date: Thu, 19 May 2016 10:37:05 +0200 Subject: mailbox.auto ignores dovecot-uidlist.lock In-Reply-To: References: <573C3B57.3030909@dovecot.fi> <897a96155e9582824b7eb853731d0b73@tomsommer.dk> Message-ID: <58a6dd3ebac825ad3146b38dfbb3ae80@tomsommer.dk> On 2016-05-19 08:07, Steffen Kaiser wrote: > On Wed, 18 May 2016, Tom Sommer wrote: >> On 2016-05-18 11:52, Aki Tuomi wrote: >>> On 18.05.2016 12:44, Tom Sommer wrote: >>>> I'm trying to lock down a maildir from modifications using >>>> dovecot-uidlist.lock, but when a user with mailbox.auto = create >>>> logs >>>> in, then the folder is created regardless of dovecot-uidlist.lock >>>> existing or not. >>>> >>>> Is there no way to prevent dovecot from touching anything in a >>>> maildir >>>> folder? >>>> >>> Could you provide more information, like: >>> >>> what you are trying to achieve >>> what did you do >>> what you configuration looks like >>> what version you are using >> >> I'm trying to move all files and folder from /foo.com/bar/ to >> /foo.com/bar/Maildir without moving .dovecot.svbin and .dovecot.sieve >> etc. (homedir files) >> >> So I wanted to lock the folder, kick the user and the move the >> files/folders, but the user logs in just after I kick him and so the >> auto-folders are created and my migration script fails, locking the >> folder with dovecot-uidlist.lock and maildirlock does not work in this >> case. > > Check out: > > http://wiki2.dovecot.org/Authentication/RestrictAccess?highlight=%28user%29|%28deny%29 Yea, I was just hoping there was a way without having to mess with passdb From skdovecot at smail.inf.fh-brs.de Thu May 19 08:43:54 2016 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 19 May 2016 10:43:54 +0200 (CEST) Subject: mailbox.auto ignores dovecot-uidlist.lock In-Reply-To: <58a6dd3ebac825ad3146b38dfbb3ae80@tomsommer.dk> References: <573C3B57.3030909@dovecot.fi> <897a96155e9582824b7eb853731d0b73@tomsommer.dk> <58a6dd3ebac825ad3146b38dfbb3ae80@tomsommer.dk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 19 May 2016, Tom Sommer wrote: > On 2016-05-19 08:07, Steffen Kaiser wrote: > >> On Wed, 18 May 2016, Tom Sommer wrote: >>> On 2016-05-18 11:52, Aki Tuomi wrote: >>>> On 18.05.2016 12:44, Tom Sommer wrote: >>>>> I'm trying to lock down a maildir from modifications using >>>>> dovecot-uidlist.lock, but when a user with mailbox.auto = create logs >>>>> in, then the folder is created regardless of dovecot-uidlist.lock >>>>> existing or not. >>>>> >>>>> Is there no way to prevent dovecot from touching anything in a maildir >>>>> folder? >>>>> >>>> Could you provide more information, like: >>>> >>>> what you are trying to achieve >>>> what did you do >>>> what you configuration looks like >>>> what version you are using >>> >>> I'm trying to move all files and folder from /foo.com/bar/ to >>> /foo.com/bar/Maildir without moving .dovecot.svbin and .dovecot.sieve etc. >>> (homedir files) >>> >>> So I wanted to lock the folder, kick the user and the move the >>> files/folders, but the user logs in just after I kick him and so the >>> auto-folders are created and my migration script fails, locking the folder >>> with dovecot-uidlist.lock and maildirlock does not work in this case. >> >> Check out: >> >> http://wiki2.dovecot.org/Authentication/RestrictAccess?highlight=%28user%29|%28deny%29 > > Yea, I was just hoping there was a way without having to mess with passdb If you have easy Unix permission on disk: # user cannot change anything chown -R root:root basedir migrate # make sure userdb is current flush cache # make user re-login to get changes in effect kick user # allow write access to user chown -R vmail:vmail basedir - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVz18ynz1H7kL/d9rAQJ25gf9FY7Qbkpqj0frTQmFgRBbmM9sXwPFropt 5FO2snpNPQYA7Wnu3YjZFkJOpoqJcouC+DX5Vp9HRM2KiCz16o0ANfjf37zF+vVl sAGNvb5gFSL2B0diARqu1idUKlYe++wJ20coXjs+1/gc+IDmsam+RGjddpLqWhuo c+6OhDRYsBhB8OVLvQUwT/B20Q6vmJujZ9i6oGA1jW2Ul7ibiuRXah2dSn98FAlv T1671+Dw2M+Awh8/bftBZiRlZUQ7tXiaDHGgdaBhXorEAXqsGxU/dg+0//Bqyt3w 3Th9h6YO2nfmfBbvyz+mbG34jtdyGrfW3xXpeQmBcTd5M8LaKMZHeQ== =bsGE -----END PGP SIGNATURE----- From dovecot-ml at makomi.de Thu May 19 12:40:19 2016 From: dovecot-ml at makomi.de (M. Koehler) Date: Thu, 19 May 2016 14:40:19 +0200 Subject: sieve-extprogram: How to hand over a variable or pipe a mail so a shell script? Message-ID: <8646a2f3-bfec-5a9e-d48b-4f8a930d8b99@makomi.de> Hi, I?m just playing around with sieve-extprogram but it didn?t works like I think. I want a. execute a shell script that get the subject as parameter my try: require ["fileinto","variables","envelope","vnd.dovecot.execute"]; if header :contains "Subject" "123test" { execute :input "${1}" "exec-test"; keep; } and require ["fileinto","variables","envelope","vnd.dovecot.execute"]; if header :contains "Subject" "123test" { execute "exec-test" "${1}"; keep; } The shell script will create a file with $1 as text. But I didn?t get any output - the created file is empty b. later the complete mail should piped to the shell script require ["fileinto","variables","envelope","vnd.dovecot.execute"]; if header :contains "Subject" "123test" { execute :pipe "exec-test"; keep; } This should (as http://hg.rename-it.nl/dovecot-2.2-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt described) hand over the complete mail to the script But I don?t know how I handle the complete mail with a shell script? I think the mail will not hand over as a parameter to the shell script, right? But how should I store the mail into a file or should I pipe it through a sed command line (and so one) into a shell script? Hope someone could bring some lights into my darkness :) Best regards, Michael From jlambot at gmail.com Thu May 19 14:27:18 2016 From: jlambot at gmail.com (Julien Lambot) Date: Thu, 19 May 2016 16:27:18 +0200 Subject: AD query timeout might be result size limit exceeded Message-ID: Hello list I've been struggling for a while trying to configure multiple domain ldap authentication with full e-mail address authentication. Which in fact was not the issue. There where some discrepancies between the doc and our actual configuration (see appendix A/ ) Seems that pass_filters and user_filters don't need much special settings for our setup. Now it's working correctly at the sole exception that when an OU contains "lots" of users (>200) i suspect that the ldapseach query fails. We can well authenticate when we have 50 users in an OU, but not when the number raises (I don't have the exact number above which it locks). Is there a parameter that we can set to increase the result size limit (as i suspect this to be the cause of this possible bug)? If I query manually it's ok (ldapsearch) if I use "doveadm auth user.name at domain.tld", it succeed also but I wonder if it doesn't use the winbind authentication instead. Here is our ldap-auth configuration hosts = master.domain.local:389 dn = DOMAIN\ro-user dnpass = password debug_level = 2 auth_bind = yes #auth_bind_userdn = cn=%u,OU=_myou,OU=Utilisateurs,OU=ouname,DC=domain,DC=local (tried with and without with no better results) ldap_version = 3 #deref = never #base = OU=InfrastructureManagement,DC=domain,DC=local (works has a few users) base = OU=_myou,OU=Utilisateurs,OU=ouname,DC=domain,DC=local scope = subtree user_filter = (&(objectclass=person)(mail=%u)) pass_filter = (&(objectclass=person)(mail=%u)) and some logs in appendix B/ Thanks for any hints on this. Have a nice day appendix A/ # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.7 auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb default_vsz_limit = 1 G disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 last_valid_gid = 50000 last_valid_uid = 50000 mail_gid = 5000 mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail mail_privileged_group = virtmail mail_uid = 5000 namespace inbox { hidden = no inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = subscriptions = yes } passdb { args = scheme=plain-md5 username_format=%u /etc/dovecot/users driver = passwd-file } passdb { args = scheme=plain-md5 username_format=%u /etc/dovecot/users driver = passwd-file } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/pirisusers-ldap.conf.ext driver = ldap } protocols = imap service auth { unix_listener auth-userdb { mode = 0666 user = virtmail } } ssl_cert = May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: re_encode_request new request is: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_chase_v3referral: msgid 11, url "ldap://ForestDnsZones.domain.local/DC=ForestDnsZones,DC=domain,DC=local" May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_send_server_request May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_new_connection 0 1 1 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_int_open_connection May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_connect_to_host: TCP ForestDnsZones.domain.local:389 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_new_socket: 21 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_prepare_socket: 21 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_connect_to_host: Trying 10.1.2.34:389 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_pvt_connect: fd: 21 tm: -1 async: 0 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: anonymous rebind via ldap_sasl_bind("") May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_sasl_bind May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_send_initial_request May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_send_server_request May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ldap_result ld 0x7fcc0a585fa0 msgid 16 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: wait4msg ld 0x7fcc0a585fa0 msgid 16 (timeout 100000 usec) May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: wait4msg continue ld 0x7fcc0a585fa0 msgid 16 all 1 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ** ld 0x7fcc0a585fa0 Connections: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: * host: ForestDnsZones.domain.local port: 0 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: refcnt: 2 status: Connected May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: last used: Thu May 19 12:57:36 2016 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: rebind in progress May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: queue is empty May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: * host: DomainDnsZones.domain.local port: 0 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: refcnt: 2 status: Connected May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: last used: Thu May 19 12:57:36 2016 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: rebind in progress May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: queue is empty May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: * host: master.domain.local port: 389 (default) May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: refcnt: 4 status: Connected May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: last used: Thu May 19 12:57:36 2016 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: ** ld 0x7fcc0a585fa0 Outstanding Requests: May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: * msgid 16, origid 16, status InProgress May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: outstanding referrals 0, parent count 0 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: * msgid 14, origid 14, status InProgress May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: outstanding referrals 0, parent count 0 May 19 12:57:36 iftstpupimap1 dovecot: auth: Error: * msgid 11, origid 11, status InProgress May 19 13:00:06 iftstpupimap1 dovecot: auth: Error: PLAIN(): Request 0.1 timeouted after 150 secs, state=1 From tss at iki.fi Thu May 19 23:15:18 2016 From: tss at iki.fi (Timo Sirainen) Date: Fri, 20 May 2016 02:15:18 +0300 Subject: BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish) In-Reply-To: <980567F6-A127-4B14-88BC-A5F18537A798@cpanel.net> References: <980567F6-A127-4B14-88BC-A5F18537A798@cpanel.net> Message-ID: On 19 May 2016, at 03:54, J. Nick Koston wrote: > > Hi, > > It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn?t received the X (done) response. > If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. > > This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. Do you see this as causing some problem? I think this is intentional. The F(inish) tells the final response whether everything was OK or not, so all the work is done after that. The 'X' is described in code as: /* notify remote that we're closing. this is mainly to avoid "read() failed: EOF" errors on failing dsyncs */ o_stream_nsend_str(ibc->output, t_strdup_printf("%c\n", items[ITEM_DONE].chr)); and on the receiving side: if (line[0] == items[ITEM_DONE].chr) { /* remote cleanly closed the connection, possibly because of some failure (which it should have logged). we don't want to log any stream errors anyway after this. */ dsync_ibc_stream_stop(ibc); return DSYNC_IBC_RECV_RET_TRYAGAIN; } So I think it only matters in error handling. From nick at cpanel.net Fri May 20 04:12:46 2016 From: nick at cpanel.net (J. Nick Koston) Date: Thu, 19 May 2016 18:12:46 -1000 Subject: BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish) In-Reply-To: References: Message-ID: <72AE10C0-1D2B-4CA5-9C3F-E8EA52E3A1EE@cpanel.net> Hi Timo, The below solved the problem for me as far as I can tell I was getting a EPIPE about 1:30 dsync runs over the socket. I have received 0 in 10000 runs with this change. https://github.com/dovecot/core/pull/10 You know the code much better then I do so you might have a better way? Also I have a commit to address the constant warnings when the prefix is set to INBOX. with the quota_quota plugin here: https://github.com/dovecot/core/pull/7 Thanks -Nick Timo Sirainen tss at iki.fi? Thu May 19 23:15:18 UTC 2016 > > On 19 May 2016, at 03:54, J. Nick Koston > wrote: > > > > Hi, > > > > It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn?t received the X (done) response. > > If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. > > > > This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. > > Do you see this as causing some problem? I think this is intentional. The F(inish) tells the final response whether everything was OK or not, so all the work is done after that. The 'X' is described in code as: > > /* notify remote that we're closing. this is mainly to avoid > "read() failed: EOF" errors on failing dsyncs */ > o_stream_nsend_str(ibc->output, > t_strdup_printf("%c\n", items[ITEM_DONE].chr)); > > and on the receiving side: > > if (line[0] == items[ITEM_DONE].chr) { > /* remote cleanly closed the connection, possibly because of > some failure (which it should have logged). we don't want to > log any stream errors anyway after this. */ > dsync_ibc_stream_stop(ibc); > return DSYNC_IBC_RECV_RET_TRYAGAIN; > } > > So I think it only matters in error handling. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3569 bytes Desc: not available URL: From dovecot at tvetc.de Fri May 20 07:59:53 2016 From: dovecot at tvetc.de (Karim 'Kasi Mir' Senoucci) Date: Fri, 20 May 2016 09:59:53 +0200 Subject: dovecot-sieve: require "mailbox" fails (Unbuntu 16.04) Message-ID: Hello everyone, I'm moving some old procmail recipes to sieve, but having problems with it. The problem lies with the mailbox extension, which I'm trying to use as proposed in an example in the Dovecot wiki: > require ["variables","date","fileinto","mailbox"]; > > # Extract date info > if currentdate :matches "year" "*" { set "year" "${1}"; } > if currentdate :matches "month" "*" { set "month" "${1}"; } > > # Archive Dovecot mailing list items by year and month. > # Create folder when it does not exist. > if header :is "list-id" "dovecot.dovecot.org" { > fileinto :create "INBOX.Lists.${year}.${month}.dovecot"; > } I'm trying a similar rule (different condition, different mail folder), but cannot seen to get it to work. I always get the errors: > .DOVECOT: LINE 16: ERROR: UNEXPECTED '}' FOUND AT (THE PRESUMED) END OF FILE. > .DOVECOT: ERROR: PARSE FAILED. I'm using the standard dovecot packages from Ubuntu 16.04: > II DOVECOT-CORE 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - CORE FILES > II DOVECOT-IMAPD 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - IMAP DAEMON > II DOVECOT-MANAGESIEVED 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - MANAGE SIEVE SERVER > II DOVECOT-SIEVE 1:2.2.22-1UBUNTU2 AMD64 SECURE POP3/IMAP SERVER - SIEVE FILTERS SUPPORT I was under the impression that these sieve extensions, including "mailbox", are builtin into dovecot/pigeonhole, or at least packaged into dovecot-sieve. Is that not the case? How can I get the example working under Ubuntu linux 16.04? Do I need to add or reconfigure something by hand? Thanks in advance for any help on the subject. Greetings Kasi Mir From p.heinlein at heinlein-support.de Fri May 20 12:19:51 2016 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Fri, 20 May 2016 14:19:51 +0200 Subject: Dovecot-FTS: Message-ID: <573F00E7.1030809@heinlein-support.de> We're running Dovecot with SOLR Lucene 5.5 and we have several errors like badMessage: 414 for HttpChannelOverHttp at 5216da8e{r=0,​c=false,​a=IDLE,​uri=-} 20.5.2016, 14:10:16 and URI is too large >8192 in our solr-logfile. I don't know if this is a problem in SOLR or in Dovecot, so I just bring it to your attention. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From mkliewe at gmx.de Fri May 20 14:56:10 2016 From: mkliewe at gmx.de (Michael Kliewe) Date: Fri, 20 May 2016 16:56:10 +0200 Subject: Dovecot-FTS: In-Reply-To: <573F00E7.1030809@heinlein-support.de> References: <573F00E7.1030809@heinlein-support.de> Message-ID: <573F258A.6030203@gmx.de> Hi Peer, On 20.05.2016 14:19, Peer Heinlein wrote: > We're running Dovecot with SOLR Lucene 5.5 and we have several errors like > > badMessage: 414 for > HttpChannelOverHttp at 5216da8e{r=0,​c=false,​a=IDLE,​uri=-} > 20.5.2016, 14:10:16 > > and > > URI is too large >8192 > > in our solr-logfile. > > I don't know if this is a problem in SOLR or in Dovecot, so I just bring > it to your attention. > > Peer > > Maybe increasing maxHttpHeaderSize="65536" in your Tomcat Connector definition in /etc/tomcat7/server.xml helps. But as far as I know the default URI limit should be 32k, not 8k... Do you have a HTTP proxy in front of your SOLR servers? Apache has a request limit of 8k for example... http://httpd.apache.org/docs/current/mod/core.html#limitrequestline Problem seems to be that some clients (iOS?) are doing very strange SEARCH commands, with many OR conditions, which results in very long search requests to SOLR... Michael From anmeyer at mailbox.org Fri May 20 23:49:25 2016 From: anmeyer at mailbox.org (Andreas Meyer) Date: Sat, 21 May 2016 01:49:25 +0200 Subject: solr does not index files in the mailbox Message-ID: <20160521014925.1acff7e7@workstation.bitcorner.intern> Hello! I startet solr and configured dovecot to use it. Dovecot just indexes two files dovecot.index.log and dovecot.mailbox.log but nothing in the cur directory. ./post -c myfiles /home/a.meyer/Postfach java -classpath /home/andreas/solr-5.5.1/dist/solr-core-5.5.1.jar -Dauto=yes -Dc=myfiles -Ddata=files -Drecursive=yes org.apache.solr.util.SimplePostTool /home/a.meyer/Postfach SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/myfiles/update... Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log Entering recursive mode, max depth=999, delay=0s Indexing directory /home/a.meyer/Postfach (2 files, depth=0) POSTing file dovecot.index.log (text/plain) to [base]/extract POSTing file dovecot.mailbox.log (text/plain) to [base]/extract Indexing directory /home/a.meyer/Postfach/cur (0 files, depth=1) Indexing directory /home/a.meyer/Postfach/new (0 files, depth=1) Indexing directory /home/a.meyer/Postfach/tmp (0 files, depth=1) 2 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/myfiles/update... Time spent: 0:00:00.751 If I give the cur folder as startingpoint ./post -c myfiles /home/a.meyer/Postfach/cur java -classpath /home/andreas/solr-5.5.1/dist/solr-core-5.5.1.jar -Dauto=yes -Dc=myfiles -Ddata=files -Drecursive=yes org.apache.solr.util.SimplePostTool /home/a.meyer/Postfach/cur SimplePostTool version 5.0.0 Posting files to [base] url http://localhost:8983/solr/myfiles/update... Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log Entering recursive mode, max depth=999, delay=0s Indexing directory /home/a.meyer/Postfach/cur (0 files, depth=0) 0 files indexed. COMMITting Solr index changes to http://localhost:8983/solr/myfiles/update... Time spent: 0:00:00.102 nothing is indexed although the directory contains 4 mails. What am I doing wrong? Greetings Andreas From hughbragg at dodo.com.au Sat May 21 19:17:23 2016 From: hughbragg at dodo.com.au (Hugh Bragg) Date: Sun, 22 May 2016 05:17:23 +1000 Subject: mail-search backtrace In-Reply-To: <2D05C734-10E5-4F8E-B26C-39B981C18FBC@iki.fi> References: <57094E96.7020501@dodo.com.au> <2D05C734-10E5-4F8E-B26C-39B981C18FBC@iki.fi> Message-ID: <5740B443.9070205@dodo.com.au> On 13/04/16 06:41, Timo Sirainen wrote: > On 09 Apr 2016, at 21:48, Hugh Bragg wrote: >> I'm repeatedly getting this error: >> >> Apr 07 04:37:27 imap(mymail at address): Panic: file mail-search.c: line 84 (mail_search_arg_init): assertion failed: (arg->initialized.keywords == NULL) >> Apr 07 04:37:27 imap(mymail at address): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x827c2) [0x7fcb7f65e7c2] -> /usr/lib64/dovecot/libdovecot.so.0(+0x828ad) [0x7fcb7f65e8ad] -> /usr/lib64/dov >> ecot/libdovecot.so.0(i_fatal+0) [0x7fcb7f605b01] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_search_arg_init+0x228) [0x7fcb7f91a328] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_search_result_updat >> e_flags+0x100) [0x7fcb7f98e470] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_sync_search_results_update+0x52) [0x7fcb7f9983e2] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_mailbox_sync_deinit+0x185 >> ) [0x7fcb7f998bb5] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x32) [0x7fcb7f921222] -> /usr/lib64/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0xab3) [0x7fcb7e9f7313] -> /usr > It's coming from virtual mailboxes. > >> namespace virtual { >> location = virtual:/var/mail/vhosts/%d/%n/virtual >> prefix = virtual. >> separator = . >> } > What do your dovecot-virtual files contain? I guess opening one of those virtual mailboxes crashes always. Related to searching keywords. It still happens once in a while. It just won't expunge old messages from unseen. There is no other trace or log message. I was hoping to isolate the cause, but all I could only say for sure that it happens sometime after Dovecot first starts up and I have to restart to fix it. dovecot-virtual files look like this: # cat virtual/all/dovecot-virtual * all # cat virtual/Unseen/dovecot-virtual virtual.all inthread refs unseen A fresh trace: May 21 00:28:08 imap(x at y): Panic: file mail-search.c: line 84 (mail_search_arg_init): assertion failed: (arg->initialized.keywords == NULL) May 21 00:28:08 imap(x at y): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x85c62) [0x7f4fd8915c62] -> /usr/lib64/dovecot/libdovecot.so.0(+0x85d4d) [0x7f4fd8915d4d] -> /usr/lib64/dov ecot/libdovecot.so.0(i_fatal+0) [0x7f4fd88ba5c1] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_search_arg_init+0x228) [0x7f4fd8bd4b78] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_search_result_updat e_flags+0x100) [0x7f4fd8c49d00] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_sync_search_results_update+0x52) [0x7f4fd8c53ce2] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_mailbox_sync_deinit+0x185 ) [0x7f4fd8c544b5] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x32) [0x7f4fd8bdba82] -> /usr/lib64/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x538) [0x7f4fd7caa428] -> /usr /lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x3b) [0x7f4fd8bdb9fb] -> dovecot/imap(imap_sync_init+0x68) [0x56091d93b078] -> dovecot/imap(+0x1210e) [0x56091d92710e] -> dovecot/imap(+0x1234d) [0x56091 d92734d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xea) [0x7f4fd892984a] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f4fd892ae4b] -> /usr/lib64/dovecot/libdo vecot.so.0(io_loop_handler_run+0x25) [0x7f4fd8929a75] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f4fd8929c18] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4fd88c0123] -> d ovecot/imap(main+0x328) [0x56091d922a98] -> /lib64/libc.so.6(__libc_start_main+0xf0) [0x7f4fd84ef580] -> dovecot/imap(_start+0x29) [0x56091d922c19] From heiken at luis.uni-hannover.de Mon May 23 16:16:34 2016 From: heiken at luis.uni-hannover.de (Karsten Heiken) Date: Mon, 23 May 2016 18:16:34 +0200 Subject: No implicit INBOX with mailbox_list_index = yes? Message-ID: <2fe55897-aa2f-b16b-3e1c-03530a986bd2@luis.uni-hannover.de> Hello all, I am running a Dovecot Cluster (2.2.13 and 2.2.23 show the same behaviour) using dsync replication (as described here: http://wiki.dovecot.org/Replication). In addition to that I allow users to share mailboxes using IMAP ACLs. My problem is that I need to set "mail_shared_explicit_inbox = yes" if I want my users to see Inboxes of other users. If I leave mail_shared_explicit_inbox = no, then the user cannot see any Inboxes of other users: > * LIST (\Subscribed \UnMarked) "/" Sent > * LIST (\Subscribed \UnMarked) "/" Drafts > * LIST (\Subscribed) "/" INBOX When I set explicit_inbox = yes, then I can see the shared mailboxes: > * LIST (\Subscribed \UnMarked) "/" Sent > * LIST (\Subscribed \UnMarked) "/" Drafts > * LIST (\NonExistent) "/" shared > * LIST (\NonExistent) "/" shared/bob at luis.uni-hannover.de > * LIST (\Subscribed) "/" shared/bob at luis.uni-hannover.de/INBOX > * LIST (\Subscribed) "/" INBOX After some digging I found out that if I set mailbox_list_index = NO, then everything works even without explicit_inbox: > * LIST (\Subscribed) "/" Sent > * LIST (\Subscribed) "/" Drafts > * LIST (\NonExistent) "/" shared > * LIST (\Subscribed) "/" shared/bob at luis.uni-hannover.de > * LIST (\Subscribed) "/" INBOX Since we are migrating from Cyrus to Dovecot, our users are used to the behaviour of explicit_inbox = no, so I would like to keep it that way. I would simply leave mailbox_list_index disabled, but the comment states it is required for NOTIFY, which in turn is required by the rolling replication -- am I corrent? > # Mailbox list indexes can be used to optimize IMAP STATUS commands. They are > # also required for IMAP NOTIFY extension to be enabled. > # mailbox_list_index = yes What am I doing wrong or don't understand? Thanks, Karsten -----8<----- # doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-44-pve x86_64 Debian 8.1 default_vsz_limit = 512 M dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } lmtp_rcpt_check_quota = yes mail_gid = 7777 mail_location = maildir:%h/Maildir mail_plugins = " zlib quota acl notify" mail_uid = 7777 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:/var/vmail/%%d/%%n/Maildir:INDEXPVT=~/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox 30dTrash { auto = subscribe special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=5 acl_shared_dict = proxy::acl quota = maildir:Postfach-Limit quota_grace = 2%% quota_rule = *:storage=8G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_redirects = 25 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at luis.uni-hannover.de protocols = " imap lmtp sieve pop3 sieve" replication_dsync_parameters = -d -l 30 -U -n inbox service auth { unix_listener auth-userdb { group = vmail user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 24245 } } service imap-login { process_min_avail = 16 service_count = 0 } service imap { executable = imap postlogin process_limit = 30000 } service lmtp { inet_listener lmtp { port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3 { process_limit = 2500 } service postlogin { executable = script-login -d rawlog /usr/local/bin/dovecot-postlogin.sh } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } ssl_cert = OS is CentOS 6, dovecot-2.0.9-19 Just started using this server as a replacement for an old Debian server running an old dovecot. The number of dovecot processes just grows and grows until there are 100 concurrent login processes (we have maybe 40 or 50 users), and then nobody can log in any more. A reload resets, but the number of stuck processes climbs very quickly. We're authenticating against an OpenLDAP. Unfortunately, this system is not connected to the Internet so I cannot paste logs or output of commands. I've been Googling, but aren't hitting on the right phrase to describe this issue to uncover possible answers :-( If anyone can get me pointed in the right direction, I'd appreciate it. -- *********************************************************************** * John Oliver http://www.john-oliver.net/ * * * *********************************************************************** From alessio at skye.it Tue May 24 06:47:22 2016 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 24 May 2016 08:47:22 +0200 Subject: Growing number of dovecot processes In-Reply-To: <20160523171644.GA30264@ns2.sdsitehosting.net> References: <20160523171644.GA30264@ns2.sdsitehosting.net> Message-ID: <5743F8FA.8060902@skye.it> Il 23/05/2016 19:16, John Oliver ha scritto: > OS is CentOS 6, dovecot-2.0.9-19 > > Just started using this server as a replacement for an old Debian server > running an old dovecot. The number of dovecot processes just grows and > grows until there are 100 concurrent login processes (we have maybe 40 > or 50 users), and then nobody can log in any more. A reload resets, but > the number of stuck processes climbs very quickly. Switch to "High-performance mode" [...] It works by using a number of long running login processes [...] http://wiki.dovecot.org/LoginProcess -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice From ward.romp at gmail.com Tue May 24 11:31:36 2016 From: ward.romp at gmail.com (Ward Romp) Date: Tue, 24 May 2016 13:31:36 +0200 Subject: Error: Filename keeps changing Message-ID: <740aaa1e-207c-09d5-61e9-84b43a27f5a4@gmail.com> Dear all, Two days ago, my ldap server (for authentication) crashed and I had to set up a new one. Everything works fine except the mailbox for one user (let's name him debugger). When I try to access his mail whatever way, dovecot complains about "Error: maildir_file_do(........): Filename keeps changing" For example, a command "doveadm copy -u debugger at boontje.net Archive mailbox Netspar" returns doveadm(debugger at boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/1423052681.M610997P2829.mail,S=1499407,W=1518981:2,S): Filename keeps changing doveadm(debugger at boontje.net): Error: Copying message UID 1 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] doveadm(debugger at boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/1447239217.M843201P13440.mail,S=6052999,W=6131730:2,RSa): Filename keeps changing doveadm(debugger at boontje.net): Error: Copying message UID 2 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] doveadm(debugger at boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/1447239226.M993616P13440.mail,S=13920,W=14288:2,S): Filename keeps changing doveadm(debugger at boontje.net): Error: Copying message UID 3 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] doveadm(debugger at boontje.net): Error: maildir_file_do(/var/mail/vmail/boontje.net/debugger/mail/Netspar/cur/1449826302.M51386P11767.mail,S=1252017,W=1268393:2,RS): Filename keeps changing doveadm(debugger at boontje.net): Error: Copying message UID 4 from 'Netspar' failed: Internal error occurred. Refer to server log for more information. [2016-05-24 13:19:58] It does this on ALL mailboxes of this user and only this user. Other users give no problems. Accessing the mails via a mail client (Roundcube, Thunderbird, etc...) give the same error. Dovecot runs on a virtual machine and the mailboxes are stored via a CIFS-share on a NAS. I rebooted both machines, checked the filesystem (no errors), CIFS-share, directories and mail files are readable and writeable by dovecot user (vmail on the mail server). The mails are intact, cat shows the full mail. I'm baffled, any suggestions? Kind regards, Ward Romp The configuration etc.... ================================================ # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 4.2.0-36-generic x86_64 Ubuntu 14.04.4 LTS cifs auth_mechanisms = plain login auth_verbose = yes listen = * mail_home = /var/mail/vmail/%d/%n mail_location = maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/conf.d/ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_after = /var/mail/vmail/sieve-after sieve_before = /var/mail/vmail/sieve-before sieve_dir = ~/sieve } protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r From p.heinlein at heinlein-support.de Tue May 24 18:20:29 2016 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 24 May 2016 20:20:29 +0200 Subject: New feature: HTTP API In-Reply-To: References: <289895264.13993.22ea9490-41a0-4846-81ec-d4706f3bbfc6.open-xchange@appsuite.open-xchange.com> <1988331466.16402.720408d5-801e-44fb-ac8f-1881f8bd92d1.open-xchange@appsuite.open-xchange.com> <853408930.4781.1459838699525@appsuite-dev.open-xchange.com> <868b534e29558c25035e536a604e7544@tomsommer.dk> <173870327.5192.1459848290040@appsuite-dev.open-xchange.com> <2143737481.6249.1459863315740@appsuite-dev.open-xchange.com> Message-ID: <57449B6D.2090905@heinlein-support.de> Thanks for the new HTTP API. I get the API with some commands up and running, but I'm still not able to create folder. curl -k -H "Content-Type: application/json" -H "Authorization: Basic secret" -d '[["mailboxCreate",{"user":"user at example.org","mailbox":["INBOX/TEST"]},c01]]' https://xxx.xxx.xxx.xxx.:8080/doveadm/v1 and also curl -k -H "Content-Type: application/json" -H "Authorization: Basic secret" -d '[["mailboxList",{"user":"user at example.org","mailboxMask":["INBOX/TEST"]},c01]]' https://xxx.xxx.xxx.xxx.:8080/doveadm/v1 doesn't work. It /should/ create INBOX/TEST -- but doveadm always says it's "Invalid JSON input". What am I doing wrong? Peer From michael.slusarz at dovecot.fi Tue May 24 18:38:55 2016 From: michael.slusarz at dovecot.fi (Michael Slusarz) Date: Tue, 24 May 2016 12:38:55 -0600 (MDT) Subject: New feature: HTTP API In-Reply-To: <57449B6D.2090905@heinlein-support.de> References: <289895264.13993.22ea9490-41a0-4846-81ec-d4706f3bbfc6.open-xchange@appsuite.open-xchange.com> <1988331466.16402.720408d5-801e-44fb-ac8f-1881f8bd92d1.open-xchange@appsuite.open-xchange.com> <853408930.4781.1459838699525@appsuite-dev.open-xchange.com> <868b534e29558c25035e536a604e7544@tomsommer.dk> <173870327.5192.1459848290040@appsuite-dev.open-xchange.com> <2143737481.6249.1459863315740@appsuite-dev.open-xchange.com> <57449B6D.2090905@heinlein-support.de> Message-ID: <2098385212.191.1464115136306@appsuite-dev.open-xchange.com> > > On May 24, 2016 at 12:20 PM Peer Heinlein wrote: > > Thanks for the new HTTP API. > > I get the API with some commands up and running, but I'm still not able > to create folder. > > curl -k -H "Content-Type: application/json" -H "Authorization: Basic > secret" -d > '[["mailboxCreate",{"user":"user at example.org","mailbox":["INBOX/TEST"]},c01]]' > https://xxx.xxx.xxx.xxx.:8080/doveadm/v1 > The identifier is a JSON string (c01) and needs to be in quotes, at a minimum. > > and also > > curl -k -H "Content-Type: application/json" -H "Authorization: Basic > secret" -d > '[["mailboxList",{"user":"user at example.org","mailboxMask":["INBOX/TEST"]},c01]]' > https://xxx.xxx.xxx.xxx.:8080/doveadm/v1 > > doesn't work. > > It /should/ create INBOX/TEST -- but doveadm always says it's "Invalid > JSON input". > > What am I doing wrong? > michael From p.heinlein at heinlein-support.de Tue May 24 22:23:54 2016 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Wed, 25 May 2016 00:23:54 +0200 Subject: New feature: HTTP API In-Reply-To: <2098385212.191.1464115136306@appsuite-dev.open-xchange.com> References: <289895264.13993.22ea9490-41a0-4846-81ec-d4706f3bbfc6.open-xchange@appsuite.open-xchange.com> <1988331466.16402.720408d5-801e-44fb-ac8f-1881f8bd92d1.open-xchange@appsuite.open-xchange.com> <853408930.4781.1459838699525@appsuite-dev.open-xchange.com> <868b534e29558c25035e536a604e7544@tomsommer.dk> <173870327.5192.1459848290040@appsuite-dev.open-xchange.com> <2143737481.6249.1459863315740@appsuite-dev.open-xchange.com> <57449B6D.2090905@heinlein-support.de> <2098385212.191.1464115136306@appsuite-dev.open-xchange.com> Message-ID: <5744D47A.9070005@heinlein-support.de> Am 24.05.2016 um 20:38 schrieb Michael Slusarz: > The identifier is a JSON string (c01) and needs to be in quotes, at a minimum. Yes, that's it. Sorry, I was too blind to see that. Thanks. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From voytek at sbt.net.au Wed May 25 06:06:08 2016 From: voytek at sbt.net.au (voytek at sbt.net.au) Date: Wed, 25 May 2016 16:06:08 +1000 Subject: ot: selfcert IMAP StartTLS 143 Win10 Tbird woes Message-ID: I was trying to set up a replacement laptop running W10/64 to replace a Win7 laptop, installed TBird, and, attempted normal setup: IMAP StartTLS 143, SMTP StartTLS 587 (like on old laptop), and, to cut a long story, eventually discovered it won't work till I changed IMAP SSL/TLS 993 ? SMTP seems OK, only IMAP didn't (I recall earlier similar type of issues with either Outlook or OE or WLive?) is this the self certified part ? or what is it ? what's the best way around ? or just leave it as 993 ? thanks for any hints , pointers or even words of encouragement From voytek at sbt.net.au Wed May 25 06:13:29 2016 From: voytek at sbt.net.au (voytek at sbt.net.au) Date: Wed, 25 May 2016 16:13:29 +1000 Subject: ot: migrating TB user's email to new laptop Message-ID: another OT: setting up new W10 laptop with TBird: user has three accounts, several hundred emails on old laptop BUT NO LONGER on server two of the accounts are on my Dovecot server, one on ISP server what's best way to transfer local emails: transfer TBird files laptop to laptop ? or, is there some way to sync old laptop to server, then retrieve on new laptop ? thanks, V From philip at mcgaw.eu Wed May 25 06:31:18 2016 From: philip at mcgaw.eu (Philip McGaw) Date: Wed, 25 May 2016 07:31:18 +0100 Subject: ot: migrating TB user's email to new laptop In-Reply-To: References: Message-ID: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> Did user historically have POP set up? If you still have access to the old laptop set up imap and move the emails back. Sent from my iPhone > On 25 May 2016, at 07:13, voytek at sbt.net.au wrote: > > another OT: > > setting up new W10 laptop with TBird: > > user has three accounts, several hundred emails on old laptop BUT NO > LONGER on server > > two of the accounts are on my Dovecot server, one on ISP server > > what's best way to transfer local emails: > > transfer TBird files laptop to laptop ? or, is there some way to sync old > laptop to server, then retrieve on new laptop ? > > thanks, > > V > From philip at mcgaw.eu Wed May 25 06:33:16 2016 From: philip at mcgaw.eu (Philip McGaw) Date: Wed, 25 May 2016 07:33:16 +0100 Subject: ot: selfcert IMAP StartTLS 143 Win10 Tbird woes In-Reply-To: References: Message-ID: <03323364-15A8-4817-8CF6-8822EC85938C@mcgaw.eu> Why are you using a self signed? If your machine has an external hostname use lets encrypt to get an SSL cert for it. https://skippy.org.uk/lets-encrypt-postfix-and-dovecot/ Sent from my iPhone > On 25 May 2016, at 07:06, voytek at sbt.net.au wrote: > > I was trying to set up a replacement laptop running W10/64 to replace a > Win7 laptop, installed TBird, and, attempted normal setup: > > IMAP StartTLS 143, SMTP StartTLS 587 (like on old laptop), and, to cut a > long story, eventually discovered it won't work till I changed IMAP > SSL/TLS 993 ? > > SMTP seems OK, only IMAP didn't (I recall earlier similar type of issues > with either Outlook or OE or WLive?) > > is this the self certified part ? or what is it ? > > what's the best way around ? or just leave it as 993 ? > > thanks for any hints , pointers or even words of encouragement > From voytek at sbt.net.au Wed May 25 06:37:59 2016 From: voytek at sbt.net.au (voytek at sbt.net.au) Date: Wed, 25 May 2016 16:37:59 +1000 Subject: ot: migrating TB user's email to new laptop In-Reply-To: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> References: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> Message-ID: <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> On Wed, May 25, 2016 4:31 pm, Philip McGaw wrote: > Did user historically have POP set up? Philip, thanks no, not for a long time, IMAP/143/StartTLS on old laptop > If you still have access to the old laptop set up imap and move the > emails back. yes, I have old laptop here. sorry, not sure how to, is that inside TBird, or how ? (not very familiar with TBird...) >> another OT: >> >> setting up new W10 laptop with TBird: >> >> user has three accounts, several hundred emails on old laptop BUT NO >> LONGER on server >> >> >> two of the accounts are on my Dovecot server, one on ISP server >> >> what's best way to transfer local emails: >> >> transfer TBird files laptop to laptop ? or, is there some way to sync >> old laptop to server, then retrieve on new laptop ? >> >> thanks, >> >> V >> >> > From philip at mcgaw.eu Wed May 25 06:48:37 2016 From: philip at mcgaw.eu (Philip McGaw) Date: Wed, 25 May 2016 07:48:37 +0100 Subject: ot: migrating TB user's email to new laptop In-Reply-To: <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> References: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> Message-ID: <9C47CDC3-D547-4AB0-AACD-FD3333FBFFB8@mcgaw.eu> But the emails are on the old laptop? Can you send a screenshot of the left hand folder tree? Sent from my iPhone > On 25 May 2016, at 07:37, voytek at sbt.net.au wrote: > >> On Wed, May 25, 2016 4:31 pm, Philip McGaw wrote: >> Did user historically have POP set up? > > Philip, thanks > > no, not for a long time, IMAP/143/StartTLS on old laptop > >> If you still have access to the old laptop set up imap and move the >> emails back. > > yes, I have old laptop here. > sorry, not sure how to, is that inside TBird, or how ? (not very familiar > with TBird...) > >>> another OT: >>> >>> setting up new W10 laptop with TBird: >>> >>> user has three accounts, several hundred emails on old laptop BUT NO >>> LONGER on server >>> >>> >>> two of the accounts are on my Dovecot server, one on ISP server >>> >>> what's best way to transfer local emails: >>> >>> transfer TBird files laptop to laptop ? or, is there some way to sync >>> old laptop to server, then retrieve on new laptop ? >>> >>> thanks, >>> >>> V > From sottilette at rfx.it Wed May 25 09:15:41 2016 From: sottilette at rfx.it (Paolo) Date: Wed, 25 May 2016 11:15:41 +0200 (CEST) Subject: ot: migrating TB user's email to new laptop In-Reply-To: <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> References: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> Message-ID: On Wed, 25 May 2016, voytek at sbt.net.au wrote: > On Wed, May 25, 2016 4:31 pm, Philip McGaw wrote: >> Did user historically have POP set up? > > Philip, thanks > > no, not for a long time, IMAP/143/StartTLS on old laptop > >> If you still have access to the old laptop set up imap and move the >> emails back. > > yes, I have old laptop here. > sorry, not sure how to, is that inside TBird, or how ? (not very familiar > with TBird...) Start TB on the new laptop Close it You have created the default folder/settings in the local %appdata% folder copy the %appdata%\Thunderbird folder from the old laptop to the new Finished ;-) The new laptop is now exacly as the old (better if TB same version before migration). -- Regards, Paolo ____________________________________________ From philip at mcgaw.eu Wed May 25 12:59:02 2016 From: philip at mcgaw.eu (Skippy) Date: Wed, 25 May 2016 13:59:02 +0100 Subject: ot: migrating TB user's email to new laptop In-Reply-To: References: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> Message-ID: <000e01d1b685$362f2520$a28d6f60$@eu> That would work well too However make sure you get to the bottom of the mail not being on the server as well > -----Original Message----- > From: dovecot [mailto:dovecot-bounces at dovecot.org] On Behalf Of Paolo > Sent: 25 May 2016 10:16 > To: voytek at sbt.net.au > Cc: Dovecot Mailing List > Subject: Re: ot: migrating TB user's email to new laptop > > On Wed, 25 May 2016, voytek at sbt.net.au wrote: > > > On Wed, May 25, 2016 4:31 pm, Philip McGaw wrote: > >> Did user historically have POP set up? > > > > Philip, thanks > > > > no, not for a long time, IMAP/143/StartTLS on old laptop > > > >> If you still have access to the old laptop set up imap and move the > >> emails back. > > > > yes, I have old laptop here. > > sorry, not sure how to, is that inside TBird, or how ? (not very > > familiar with TBird...) > > Start TB on the new laptop > Close it > You have created the default folder/settings in the local %appdata% > folder > > copy the %appdata%\Thunderbird folder from the old laptop to the new > > Finished ;-) > > The new laptop is now exacly as the old (better if TB same version > before migration). > > -- > > Regards, > Paolo > > ____________________________________________ From david.myers.24j74 at gmail.com Wed May 25 13:52:44 2016 From: david.myers.24j74 at gmail.com (David Myers) Date: Wed, 25 May 2016 15:52:44 +0200 Subject: ot: migrating TB user's email to new laptop In-Reply-To: <000e01d1b685$362f2520$a28d6f60$@eu> References: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> <000e01d1b685$362f2520$a28d6f60$@eu> Message-ID: Just for information to help anyone else who ends up on this thread for similar reasons, here are some links to the relevant pages on the mozilla site. Moving Thunderbird Data to a new computer Which has links to the detalsa about thunderbird profiles , which has details of backup and restore etc. Pertinent to this thread is the restoring to a different location sub section. @voytek : If you continue to have problems I would strongly recommend looking at the mozilla help pages, and if neccessary droping a message to their forums (they are after all the thunderbird experts ;) ), I personally have found them very helpful. Hope that helps David. On Wed, May 25, 2016 at 2:59 PM, Skippy wrote: > That would work well too > > However make sure you get to the bottom of the mail not being on the server > as well > > > -----Original Message----- > > From: dovecot [mailto:dovecot-bounces at dovecot.org] On Behalf Of Paolo > > Sent: 25 May 2016 10:16 > > To: voytek at sbt.net.au > > Cc: Dovecot Mailing List > > Subject: Re: ot: migrating TB user's email to new laptop > > > > On Wed, 25 May 2016, voytek at sbt.net.au wrote: > > > > > On Wed, May 25, 2016 4:31 pm, Philip McGaw wrote: > > >> Did user historically have POP set up? > > > > > > Philip, thanks > > > > > > no, not for a long time, IMAP/143/StartTLS on old laptop > > > > > >> If you still have access to the old laptop set up imap and move the > > >> emails back. > > > > > > yes, I have old laptop here. > > > sorry, not sure how to, is that inside TBird, or how ? (not very > > > familiar with TBird...) > > > > Start TB on the new laptop > > Close it > > You have created the default folder/settings in the local %appdata% > > folder > > > > copy the %appdata%\Thunderbird folder from the old laptop to the new > > > > Finished ;-) > > > > The new laptop is now exacly as the old (better if TB same version > > before migration). > > > > -- > > > > Regards, > > Paolo > > > > ____________________________________________ > From tanstaafl at libertytrek.org Wed May 25 17:13:11 2016 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Wed, 25 May 2016 13:13:11 -0400 Subject: ot: migrating TB user's email to new laptop In-Reply-To: References: <1834110C-6CE1-4E6B-B335-AAD505F81639@mcgaw.eu> <03ad52e14d68d66025731cdcf6609f23.squirrel@sbt.net.au> Message-ID: <813284ba-e046-a5f5-4629-a9cccf688ce8@libertytrek.org> On 5/25/2016 5:15 AM, Paolo wrote: > Start TB on the new laptop > Close it > You have created the default folder/settings in the local %appdata% folder > > copy the %appdata%\Thunderbird folder from the old laptop to the new No need for step 1. Just copy the folder from %appdata% on old PC to the new PC %appdata% and you're done. From devuan.2 at gmail.com Wed May 25 16:03:22 2016 From: devuan.2 at gmail.com (dev) Date: Wed, 25 May 2016 11:03:22 -0500 Subject: Unable to view shared mailbox Message-ID: <34ac1842-2ee2-3322-f2cd-994fa69995e8@gmail.com> Hi All, I have a mailing list I'd like to archive and share with a handful of users. The users are all in OpenLDAP. 10-auth-system.conf.ext is setup with "driver=pam" so it appears anyone in LDAP who already is able to SSH into the server may also connect to dovecot with same username/password they use in LDAP. I have a shared mailbox directory called LISTS_Archive. The system directory is /home/shared_imap. I can connect to dovecot using LDAP credentials (using Thunderbird) and this shared folder (LISTS_Archive) is grayed out. I assume I have acls wrong? What I want is for users in LDAP to access the LISTS_Archive imap folder as a shared folder for everyone. Only R/O access however (which is 'lr' in acl file I think). Would someone please point out what I've missed? Thank you. # # egrep -v '(^$|\#)' /etc/dovecot/conf.d/10-mail.conf # mail_location = maildir:~/.maildir namespace inbox { type = private separator = / inbox = yes } mail_plugins = acl plugin { acl = vfile:/etc/dovecot/acls } namespace { hidden = no inbox = no location = maildir:/home/shared_imap/:INDEX=~/Maildir/shared/%%u:INDEXPVT=~/Maildir/shared/%%u prefix = LISTS_Archive/ separator = / type = public list = yes subscriptions = no } # # egrep -v '(^$|\#)' /etc/dovecot/conf.d/20-imap.conf # protocol imap { mail_plugins = $mail_plugins imap_acl } # # dir /home/shared_imap/ # total 8 drwxr-xr-x 7 root root 4096 May 25 10:26 .. -rw-rw-r-- 1 user384 mail-user 0 May 25 10:43 dovecot-acl-list drwxrwsr-x 2 root mail-user 4096 May 25 10:43 . # # cat /etc/dovecot/acls # * user=user384 lr From dovecot at mx24.net Fri May 27 20:29:12 2016 From: dovecot at mx24.net (Vince42) Date: Fri, 27 May 2016 22:29:12 +0200 Subject: first_valid_uid Message-ID: <30dd144e-0f63-e765-5a0c-af085bd303ec@mx24.net> Hi, I have error log entries like dovecot: lmtp(22155, mail): Error: user mail: Mail access for users with UID 8 not permitted (see first_valid_uid in config file, uid from userdb lookup). This obviously tells me that the user mail (UID 8) is trying to access the LMTP daemon. How can I find out why he is doing so and how I can get rid of this error message? The mail system (Ubuntu 14.04 LTS : Postfix + Dovecot) is working - but probably not properly configured. -- Cheers, \\|// Vince (o o) ----------------------------ooO-(_)-Ooo------------------------- ''' (o)_(o) [ ][0][ ] ??? (=?o?=) World Domination by Copy and Paste [ ][ ][0] - (")_(") [0][0][0] () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Ooo. ---------------------------.ooO----( )------------------------- ( ) (_/ \_) From aki.tuomi at dovecot.fi Fri May 27 20:33:24 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 27 May 2016 23:33:24 +0300 Subject: first_valid_uid In-Reply-To: <30dd144e-0f63-e765-5a0c-af085bd303ec@mx24.net> References: <30dd144e-0f63-e765-5a0c-af085bd303ec@mx24.net> Message-ID: <5748AF14.8060802@dovecot.fi> On 27.05.2016 23:29, Vince42 wrote: > Hi, > > I have error log entries like > > dovecot: lmtp(22155, mail): Error: user mail: Mail access for users with > UID 8 not permitted (see first_valid_uid in config file, uid from userdb > lookup). > > This obviously tells me that the user mail (UID 8) is trying to access > the LMTP daemon. How can I find out why he is doing so and how I can get > rid of this error message? > > The mail system (Ubuntu 14.04 LTS : Postfix + Dovecot) is working - but > probably not properly configured. > Hi! Check your postfix logs? Maybe there is a clue? Grep for the uid 8 username. Aki From dovecot at mx24.net Fri May 27 20:52:08 2016 From: dovecot at mx24.net (Vince42) Date: Fri, 27 May 2016 22:52:08 +0200 Subject: first_valid_uid In-Reply-To: <5748AF14.8060802@dovecot.fi> References: <30dd144e-0f63-e765-5a0c-af085bd303ec@mx24.net> <5748AF14.8060802@dovecot.fi> Message-ID: <63b4dac4-6b87-3c3b-38a5-56b490d472bc@mx24.net> Hi, [Aki Tuomi] - [27.05.2016 22:33] >> I have error log entries like >> >> dovecot: lmtp(22155, mail): Error: user mail: Mail access for users with >> UID 8 not permitted (see first_valid_uid in config file, uid from userdb >> lookup). >> >> This obviously tells me that the user mail (UID 8) is trying to access >> the LMTP daemon. How can I find out why he is doing so and how I can get >> rid of this error message? >> The mail system (Ubuntu 14.04 LTS : Postfix + Dovecot) is working - but >> probably not properly configured. > Check your postfix logs? Maybe there is a clue? Grep for the uid 8 > username. The mail log says May 27 22:14:36 server postfix/qmgr[15415]: B05AC15E25C4: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/trivial-rewrite[26512]: warning: do not list domain domain.tld in BOTH mydestination and virtual_alias_domains May 27 22:14:36 server postfix/qmgr[15415]: A329215E2678: from=, size=707, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 62CAA15E24F6: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: C321615E2883: from=, size=707, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 0815B15E253A: from=, size=707, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 1212315E27A3: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 4374015E25F5: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 4FA1D15E245F: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 4A65815E24ED: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: 21F6C15E2760: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server postfix/qmgr[15415]: D7F9515E24B5: from=, size=708, nrcpt=1 (queue active) May 27 22:14:36 server dovecot: lmtp(26518): Connect from local May 27 22:14:36 server dovecot: lmtp(26518, mail): Error: user mail: Mail access for users with UID 8 not permitted (see first_valid_uid in config file, uid from userdb lookup). May 27 22:14:36 server dovecot: lmtp(26518): Disconnect from local: Successful quit May 27 22:14:36 server dovecot: lmtp(26518): Connect from local May 27 22:14:36 server dovecot: lmtp(26518, mail): Error: user mail: Mail access for users with UID 8 not permitted (see first_valid_uid in config file, uid from userdb lookup). May 27 22:14:36 server dovecot: lmtp(26518): Disconnect from local: Successful quit May 27 22:14:36 server postfix/lmtp[26517]: A329215E2678: to=, orig_to=, relay=server.domain.tld[private/dovecot-lmtp], delay=29904, delays=29904/0.01/0.01/0, ds n=4.3.0, status=deferred (host server.domain.tld[private/dovecot-lmtp] said: 451 4.3.0 Temporary internal error (in reply to end of DATA command)) May 27 22:14:36 server postfix/lmtp[26516]: B05AC15E25C4: to=, orig_to=, relay=server.domain.tld[private/dovecot-lmtp], delay=8910, delays=8910/0.01/0.01/0, dsn= 4.3.0, status=deferred (host server.domain.tld[private/dovecot-lmtp] said: 451 4.3.0 Temporary internal error (in reply to end of DATA command)) which shows more information but leaves me still clueless. Maybe it is a problem that domain.tld is listed in Postfix's mydestination and virtual_alias_domains, maybe it is due a "Connect from local", maybe it's an LMTP misconfiguration ... I hope that somebody will be able to shed some light on this - and maybe even explain the problem to me. :))) -- Cheers, \\|// Vince (o o) ----------------------------ooO-(_)-Ooo------------------------- ''' (o)_(o) [ ][0][ ] ??? (=?o?=) World Domination by Copy and Paste [ ][ ][0] - (")_(") [0][0][0] () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Ooo. ---------------------------.ooO----( )------------------------- ( ) (_/ \_) From aki.tuomi at dovecot.fi Fri May 27 20:59:31 2016 From: aki.tuomi at dovecot.fi (Aki Tuomi) Date: Fri, 27 May 2016 23:59:31 +0300 Subject: first_valid_uid In-Reply-To: <63b4dac4-6b87-3c3b-38a5-56b490d472bc@mx24.net> References: <30dd144e-0f63-e765-5a0c-af085bd303ec@mx24.net> <5748AF14.8060802@dovecot.fi> <63b4dac4-6b87-3c3b-38a5-56b490d472bc@mx24.net> Message-ID: <5748B533.6030101@dovecot.fi> On 27.05.2016 23:52, Vince42 wrote: > Hi, > > [Aki Tuomi] - [27.05.2016 22:33] >>> I have error log entries like >>> >>> dovecot: lmtp(22155, mail): Error: user mail: Mail access for users with >>> UID 8 not permitted (see first_valid_uid in config file, uid from userdb >>> lookup). >>> >>> This obviously tells me that the user mail (UID 8) is trying to access >>> the LMTP daemon. How can I find out why he is doing so and how I can get >>> rid of this error message? >>> The mail system (Ubuntu 14.04 LTS : Postfix + Dovecot) is working - but >>> probably not properly configured. >> Check your postfix logs? Maybe there is a clue? Grep for the uid 8 >> username. > The mail log says > > > May 27 22:14:36 server postfix/qmgr[15415]: B05AC15E25C4: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/trivial-rewrite[26512]: warning: do not > list domain domain.tld in BOTH mydestination and virtual_alias_domains > May 27 22:14:36 server postfix/qmgr[15415]: A329215E2678: > from=, size=707, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 62CAA15E24F6: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: C321615E2883: > from=, size=707, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 0815B15E253A: > from=, size=707, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 1212315E27A3: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 4374015E25F5: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 4FA1D15E245F: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 4A65815E24ED: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: 21F6C15E2760: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server postfix/qmgr[15415]: D7F9515E24B5: > from=, size=708, nrcpt=1 (queue active) > May 27 22:14:36 server dovecot: lmtp(26518): Connect from local > May 27 22:14:36 server dovecot: lmtp(26518, mail): Error: user mail: > Mail access for users with UID 8 not permitted (see first_valid_uid in > config file, uid from userdb lookup). > May 27 22:14:36 server dovecot: lmtp(26518): Disconnect from local: > Successful quit > May 27 22:14:36 server dovecot: lmtp(26518): Connect from local > May 27 22:14:36 server dovecot: lmtp(26518, mail): Error: user mail: > Mail access for users with UID 8 not permitted (see first_valid_uid in > config file, uid from userdb lookup). > May 27 22:14:36 server dovecot: lmtp(26518): Disconnect from local: > Successful quit > May 27 22:14:36 server postfix/lmtp[26517]: A329215E2678: > to=, orig_to=, > relay=server.domain.tld[private/dovecot-lmtp], delay=29904, > delays=29904/0.01/0.01/0, ds > n=4.3.0, status=deferred (host server.domain.tld[private/dovecot-lmtp] > said: 451 4.3.0 Temporary internal error (in reply to > end of DATA command)) > May 27 22:14:36 server postfix/lmtp[26516]: B05AC15E25C4: > to=, orig_to=, > relay=server.domain.tld[private/dovecot-lmtp], delay=8910, > delays=8910/0.01/0.01/0, dsn= > 4.3.0, status=deferred (host server.domain.tld[private/dovecot-lmtp] > said: 451 4.3.0 Temporary internal error (in reply to > end of DATA command)) > > > which shows more information but leaves me still clueless. Maybe it is a > problem that domain.tld is listed in Postfix's mydestination and > virtual_alias_domains, maybe it is due a "Connect from local", maybe > it's an LMTP misconfiguration ... I hope that somebody will be able to > shed some light on this - and maybe even explain the problem to me. :))) > May 27 22:14:36 server postfix/lmtp[26516]: B05AC15E25C4: to=, orig_to=, relay=server.domain.tld[private/dovecot-lmtp], delay=8910, delays=8910/0.01/0.01/0, dsn= 4.3.0, status=deferred (host server.domain.tld[private/dovecot-lmtp] said: 451 4.3.0 Temporary internal error (in reply to end of DATA command)) Your webmin is sending email to domain.tld at domain.tld which is forwarded to mail at domain.tld, which happens to have local UID 8. And you have configured min_uid to prevent this. Maybe fix your transport config? =) Aki From dovecot at mx24.net Fri May 27 21:08:42 2016 From: dovecot at mx24.net (Vince42) Date: Fri, 27 May 2016 23:08:42 +0200 Subject: first_valid_uid In-Reply-To: <5748B533.6030101@dovecot.fi> References: <30dd144e-0f63-e765-5a0c-af085bd303ec@mx24.net> <5748AF14.8060802@dovecot.fi> <63b4dac4-6b87-3c3b-38a5-56b490d472bc@mx24.net> <5748B533.6030101@dovecot.fi> Message-ID: <03535835-9623-10cb-9c45-4fc762c8db46@mx24.net> Hi, [Aki Tuomi] - [27.05.2016 22:59] > May 27 22:14:36 server postfix/lmtp[26516]: B05AC15E25C4: > to=, orig_to=, > relay=server.domain.tld[private/dovecot-lmtp], delay=8910, > delays=8910/0.01/0.01/0, dsn= > 4.3.0, status=deferred (host server.domain.tld[private/dovecot-lmtp] > said: 451 4.3.0 Temporary internal error (in reply to > end of DATA command)) > > Your webmin is sending email to domain.tld at domain.tld which is forwarded > to mail at domain.tld, which happens to have local UID 8. And you have > configured min_uid to prevent this. > > Maybe fix your transport config? =) Ouch ... I guess I missed the wood for the trees ... I hope that I found the reason for this error now ... thank you very much for pointing me into the right direction! -- Cheers, \\|// Vince (o o) ----------------------------ooO-(_)-Ooo------------------------- ''' (o)_(o) [ ][0][ ] ??? (=?o?=) World Domination by Copy and Paste [ ][ ][0] - (")_(") [0][0][0] () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Ooo. ---------------------------.ooO----( )------------------------- ( ) (_/ \_) From marcec at gmx.de Sat May 28 07:16:23 2016 From: marcec at gmx.de (Marc Joliet) Date: Sat, 28 May 2016 09:16:23 +0200 Subject: pread() failures when using mdbox on btrfs In-Reply-To: <52EF3DAD-8747-4B3D-8219-ED9DFE20B9DF@iki.fi> References: <3209263.oXmjp69zg6@thetick> <52EF3DAD-8747-4B3D-8219-ED9DFE20B9DF@iki.fi> Message-ID: <8417532.uloF26s7Pk@thetick> Am Friday 27 May 2016 schrieb Timo Sirainen >On 05 May 2016, at 16:21, Marc Joliet wrote: [...] >> Rebooting the system would make it go away (without loss of mails), though >> remounting the file system should have worked, too. I tried a variety of >> options, but the only one that helped was to convert the mailbox from mdbox >> to maildir. I have not seen any pread() failures in the two weeks since. >I can't really think of why Maildir format would get rid of the problem, >since the dovecot.index.log files still exist and they're accessed the same >way. Maybe with Maildir there are enough locks that it prevents some race >condition that triggers this. Yeah, I was somewhat surprised even without knowing that. And what do you know: two days ago I discovered pread() failures in my system log again, but this time they went away by themselves without me noticing that they even occurred: Mai 23 18:03:45 diefledermaus dovecot[600]: imap(marcec): Error: pread() failed with file /home/marcec/.maildir/.LinuxAudio.Dev/dovecot.index.log: Input/output error Mai 23 18:03:45 diefledermaus dovecot[600]: imap(marcec): Error: pread() failed with file /home/marcec/.maildir/.LinuxAudio.Dev/dovecot.index.log: Input/output error Mai 23 18:13:57 diefledermaus dovecot[600]: imap(marcec): Error: pread() failed with file /home/marcec/.maildir/.LinuxAudio.Dev/dovecot.index.log: Input/output error Mai 23 18:13:57 diefledermaus dovecot[600]: imap(marcec): Error: pread() failed with file /home/marcec/.maildir/.LinuxAudio.Dev/dovecot.index.log: Input/output error There were also a whole bunch affecting the same file on May 20th, but not before. I probably didn't notice those because they started shortly before I rebooted for a kernel upgrade (from gentoo-sources 4.4.10 to 4.5.5). >You could try stress testing with imaptest to see if it's easy to reproduce: >http://imapwiki.org/ImapTest I still haven't had time to do this, but haven't forgotten! Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From dovecot at mx24.net Sat May 28 19:25:19 2016 From: dovecot at mx24.net (Vince42) Date: Sat, 28 May 2016 21:25:19 +0200 Subject: IMAP Timeouts Message-ID: <7954926f-eb37-ef42-cca5-57cccb5bc5e9@mx24.net> Hi, from time to time the IMAP login times out ... my external monitoring says "connection reset by peer (SSL)" and Thunderbird tries to connect forever. After a short while everything returns back to normal operation. The system resources are plenty, there are no error messages and no greedy background tasks running. I read a little about tweaking imap-login - but all the default parameters seem to be set more generously than what was recommended. Where should I start the debugging process? I really wish there was some "dovecot optimizer tool" ... :P -- Cheers, \\|// Vince (o o) ----------------------------ooO-(_)-Ooo------------------------- ''' (o)_(o) [ ][0][ ] ??? (=?o?=) World Domination by Copy and Paste [ ][ ][0] - (")_(") [0][0][0] () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Ooo. ---------------------------.ooO----( )------------------------- ( ) (_/ \_) From ad+lists at uni-x.org Sat May 28 22:43:46 2016 From: ad+lists at uni-x.org (Alexander Dalloz) Date: Sun, 29 May 2016 00:43:46 +0200 Subject: IMAP Timeouts In-Reply-To: <7954926f-eb37-ef42-cca5-57cccb5bc5e9@mx24.net> References: <7954926f-eb37-ef42-cca5-57cccb5bc5e9@mx24.net> Message-ID: Am 28.05.2016 um 21:25 schrieb Vince42: > Hi, > > from time to time the IMAP login times out ... my external monitoring > says "connection reset by peer (SSL)" and Thunderbird tries to connect > forever. After a short while everything returns back to normal operation. > > The system resources are plenty, there are no error messages and no > greedy background tasks running. > > I read a little about tweaking imap-login - but all the default > parameters seem to be set more generously than what was recommended. > > Where should I start the debugging process? > > I really wish there was some "dovecot optimizer tool" ... :P That sounds suspiciously as if you are running out of entropy. Is that a virtual machine you are running with dovecot? At time when the SSL connection cannot be established check the value of available entropy by cat /proc/sys/kernel/random/entropy_avail In case you see a low value you should consider running a daemon like haveged. Alexander From aki.tuomi at dovecot.fi Sun May 29 06:58:47 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Sun, 29 May 2016 09:58:47 +0300 (EEST) Subject: IMAP Timeouts In-Reply-To: References: <7954926f-eb37-ef42-cca5-57cccb5bc5e9@mx24.net> Message-ID: <1273648539.1317.1464505129187@appsuite-dev.open-xchange.com> > On May 29, 2016 at 1:43 AM Alexander Dalloz wrote: > > > Am 28.05.2016 um 21:25 schrieb Vince42: > > Hi, > > > > from time to time the IMAP login times out ... my external monitoring > > says "connection reset by peer (SSL)" and Thunderbird tries to connect > > forever. After a short while everything returns back to normal operation. > > > > The system resources are plenty, there are no error messages and no > > greedy background tasks running. > > > > I read a little about tweaking imap-login - but all the default > > parameters seem to be set more generously than what was recommended. > > > > Where should I start the debugging process? > > > > I really wish there was some "dovecot optimizer tool" ... :P > Dovecot optimization is usually only required for large installations. > That sounds suspiciously as if you are running out of entropy. Is that a > virtual machine you are running with dovecot? > > At time when the SSL connection cannot be established check the value of > available entropy by > > cat /proc/sys/kernel/random/entropy_avail > > In case you see a low value you should consider running a daemon like > haveged. > > Alexander Also make sure you are not facing simple firewall or interconnectivity issue. Also usually it helps if you provide outputs from doveconf -n doveadm log errors --- Aki Tuomi From wrosenauer at gmail.com Sun May 29 07:56:46 2016 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Sun, 29 May 2016 09:56:46 +0200 Subject: fts lucene crashes in 2.2.24 Message-ID: Hi, I've just enabled FTS via Lucene on my Dovecot 2.2.24 installation but I see the indexer crashing ?always?. This simple testcase with a very tiny testing mailbox exposes the issue immediately: doveadm -v index -u anmesse INBOX Program received signal SIGSEGV, Segmentation fault. rescan_clear_unseen_mailbox (rescan_ctx=rescan_ctx at entry=0x0, vname=0x555555839820 "INBOX.Testfolder 2", hdr=hdr at entry=0x7fffffffdaf0) at lucene-wrapper.cc:831 831 (enum mailbox_flags)0); (gdb) bt full #0 rescan_clear_unseen_mailbox (rescan_ctx=rescan_ctx at entry=0x0, vname=0x555555839820 "INBOX.Testfolder 2", hdr=hdr at entry=0x7fffffffdaf0) at lucene-wrapper.cc:831 box = 0x7ffff76fd0ad metadata = {guid = "\230\332\377\377\377\177\000\000\200?UUU\000", virtual_size = 140737340823160, physical_size = 140737488345756, first_save_date = 140737488345840, cache_fields = 0x555500000000, precache_fields = (MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_NUL_STATE | MAIL_FETCH_STREAM_BINARY | MAIL_FETCH_IMAP_BODY | MAIL_FETCH_IMAP_BODYSTRUCTURE | MAIL_FETCH_IMAP_ENVELOPE | MAIL_FETCH_FROM_ENVELOPE | MAIL_FETCH_REFCOUNT | MAIL_FETCH_BODY_SNIPPET | unknown: 1409286144), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 4294957808)} #1 0x00007ffff489aade in rescan_clear_unseen_mailboxes (index=index at entry=0x555555822e20, rescan_ctx=rescan_ctx at entry=0x0) at lucene-wrapper.cc:863 iter = 0x55555588d530 info = vname = hdr = {last_indexed_uid = 0, settings_checksum = 3784394109, unused = 0} ns = 0x55555580f6c0 #2 0x00007ffff489b700 in lucene_settings_check (index=0x555555822e20) at lucene-wrapper.cc:429 set_checksum = ret = #3 lucene_index_build_init (index=0x555555822e20) at lucene-wrapper.cc:448 lock_path = 0x5555557dd320 "/srv/dovecot/anmesse/maildir/lucene-indexes/write.lock" st = {st_dev = 93823560581121, st_ino = 0, st_nlink = 140737344007992, st_mode = 4150696184, st_uid = 32767, st_gid = 5, __pad0 = 0, st_rdev = 93824995156880, st_size = 93824995175648, st_blksize = 140737351975397, st_blocks = 93824995096256, st_atim = {tv_sec = 140737488346056, tv_nsec = 5}, st_mtim = {tv_sec = 5, tv_nsec = 93824995156880}, st_ctim = { tv_sec = 93824995175648, tv_nsec = 140737488346352}, __glibc_reserved = {140737488346352, 93824995267672, 140737344364485}} exists = #4 0x00007ffff4899dbe in fts_backend_lucene_update_set_build_key (_ctx=0x55555583a550, key=0x7fffffffdcf0) at fts-backend-lucene.c:366 ctx = 0x55555583a550 backend = 0x555555822ce0 __FUNCTION__ = "fts_backend_lucene_update_set_build_key" #5 0x00007ffff674c984 in fts_backend_update_set_build_key (ctx=0x55555583a550, key=key at entry=0x7fffffffdcf0) at fts-api.c:175 __FUNCTION__ = "fts_backend_update_set_build_key" #6 0x00007ffff674dbb8 in fts_build_mail_header (block=0x7fffffffdcd0, block=0x7fffffffdcd0, ctx=0x7fffffffdd20) at fts-build-mail.c:174 hdr = key = {uid = 96, type = FTS_BACKEND_BUILD_KEY_HDR, part = 0x5555557dd2c0, hdr_name = 0x5555558396b0 "Return-Path", body_content_type = 0x0, body_content_disposition = 0x0} ret = #7 fts_build_mail_real (mail=0x555555838190, update_ctx=0x55555583a550) at fts-build-mail.c:548 block = {part = 0x5555557dd2c0, hdr = 0x555555839458, data = 0x55550000007c , size = 0} ret = input = 0x555555838eb0 raw_block = {part = 0x5555557dd2c0, hdr = 0x5555558395e0, data = 0x0, size = 0} skip_body = false ctx = {mail = 0x555555838190, update_ctx = 0x55555583a550, content_type = 0x0, content_disposition = 0x0, body_parser = 0x0, word_buf = 0x0, pending_input = 0x0, cur_user_lang = 0x0} prev_part = 0x5555557dd2c0 parser = 0x555555839070 decoder = 0x555555839440 parts = 0x31 body_part = false body_added = false binary_body = error = 0x60 #8 fts_build_mail (update_ctx=0x55555583a550, mail=mail at entry=0x555555838190) at fts-build-mail.c:594 _data_stack_cur_id = 5 #9 0x00007ffff675393e in fts_mail_index (_mail=0x555555838190) at fts-storage.c:501 ft = 0x5555558366a0 flist = 0x55555581ea68 #10 fts_mail_precache (_mail=0x555555838190) at fts-storage.c:520 _data_stack_cur_id = 4 mail = 0x555555838190 fmail = ft = #11 0x00007ffff76987b9 in mail_precache (mail=0x555555838190) at mail.c:401 _data_stack_cur_id = 3 p = 0x555555838190 #12 0x0000555555582fa7 in cmd_index_box_precache (box=0x55555582fd80) at doveadm-mail-index.c:75 ctx = 0x555555839be0 metadata = {guid = '\000' , virtual_size = 0, physical_size = 0, first_save_date = 0, cache_fields = 0x0, precache_fields = (MAIL_FETCH_STREAM_HEADER | MAIL_FETCH_STREAM_BODY | MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_VIRTUAL_SIZE | MAIL_FETCH_UIDL_BACKEND | MAIL_FETCH_GUID | MAIL_FETCH_POP3_ORDER), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 0)} counter = 0 max = 15 status = {messages = 15, recent = 0, unseen = 0, uidvalidity = 1048765368, uidnext = 151, first_unseen_seq = 0, first_recent_uid = 151, last_cached_seq = 0, highest_modseq = 0, highest_pvt_modseq = 0, keywords = 0x0, permanent_flags = 0, permanent_keywords = 0, allow_new_keywords = 0, nonpermanent_modseqs = 0, no_modseq_tracking = 0, have_guids = 1, have_save_guids = 1, have_only_guid128 = 0} mail = 0x555555838190 ret = 0 trans = 0x555555834310 search_args = 0x0 seq = #13 cmd_index_box (info=, info=, ctx=0x555555807410) at doveadm-mail-index.c:130 box = 0x55555582fd80 status = {messages = 130, recent = 0, unseen = 0, uidvalidity = 0, uidnext = 1434552496, first_unseen_seq = 21845, first_recent_uid = 4294958896, last_cached_seq = 32767, highest_modseq = 93824995069728, highest_pvt_modseq = 93824995074385, keywords = 0x555555819868, permanent_flags = 4150947016, permanent_keywords = 1, allow_new_keywords = 1, nonpermanent_modseqs = 1, no_modseq_tracking = 1, have_guids = 1, have_save_guids = 1, have_only_guid128 = 1} ret = 0 #14 cmd_index_run (_ctx=0x555555807410, user=0x5555558188b0) at doveadm-mail-index.c:201 _data_stack_cur_id = 2 iter = 0x55555582b9f0 info = ret = 0 user = 0x5555558188b0 _ctx = 0x555555807410 ctx = 0x555555807410 i = #15 0x000055555557dedf in doveadm_mail_next_user (ctx=ctx at entry=0x555555807410, cctx=cctx at entry=0x7fffffffe2a0, error_r=error_r at entry=0x7fffffffe0b8) at doveadm-mail.c:400 input = {module = 0x0, service = 0x5555555b15a9 "doveadm", username = 0x5555557da298 "anmesse", session_id = 0x0, session_id_prefix = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { ---Type to continue, or q to quit--- __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = { __in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0, debug = 0} error = 0x20 ip = ret = __FUNCTION__ = "doveadm_mail_next_user" #16 0x000055555557e843 in doveadm_mail_single_user (ctx=ctx at entry=0x555555807410, cctx=cctx at entry=0x7fffffffe2a0, error_r=error_r at entry=0x7fffffffe0b8) at doveadm-mail.c:431 __FUNCTION__ = "doveadm_mail_single_user" #17 0x000055555557e8fd in doveadm_mail_cmd_exec (ctx=ctx at entry=0x555555807410, cctx=cctx at entry=0x7fffffffe2a0, wildcard_user=wildcard_user at entry=0x0) at doveadm-mail.c:589 ret = error = 0x7ffff7fe89e8 "" #18 0x000055555557f6ef in doveadm_cmd_ver2_to_mail_cmd_wrapper (cctx=0x7fffffffe2a0) at doveadm-mail.c:1030 mctx = 0x555555807410 wildcard_user = username_args = {0x5555555b1479 "-u", 0x5555557da298 "anmesse", 0x0} fieldstr = 0x7ffff73d7423 "H\203\304\bH\211\330[]\303\017\037" username_args_count = i = mail_cmd = {alloc = 0x555555582ad0 , name = 0x5555555b3e7b "index", usage_args = 0x5555555b2ee8 "[-u |-A] [-S ] [-q] [-n ] "} #19 0x000055555558c93c in doveadm_cmd_run_ver2 (argc=4, argv=0x5555557e13a0, cctx=cctx at entry=0x7fffffffe2a0) at doveadm-cmd.c:524 param = pargv = {arr = {buffer = 0x5555557d9a40, element_size = 104}, v = 0x5555557d9a40, v_modifiable = 0x5555557d9a40} opts = {arr = {buffer = , element_size = 32}, v = , v_modifiable = } pargc = 7 c = -1 li = 21845 pool = 0x5555557d9778 optbuf = 0x5555557d9790 __FUNCTION__ = "doveadm_cmd_run_ver2" #20 0x000055555558c9d7 in doveadm_cmd_try_run_ver2 (cmd_name=, argc=4, argv=0x5555557e13a0, cctx=0x7fffffffe2a0) at doveadm-cmd.c:447 cmd = #21 0x000055555556fb1f in main (argc=4, argv=0x5555557e13a0) at doveadm.c:376 cctx = {cmd = 0x5555557e8208, argc = 7, argv = 0x5555557d9a78, username = 0x5555557da298 "anmesse", cli = true, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = { __in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0} cmd_name = 0x5555557e13dc "index" quick_init = false c = doveconf -n: # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.14 (099a97c) # OS: Linux 4.1.21-14-default x86_64 openSUSE 42.1 (x86_64) auth_mechanisms = plain login imap_id_log = * mail_gid = vmail mail_home = /srv/dovecot/%u mail_location = maildir:~/maildir mail_plugins = acl fts fts_lucene virtual zlib quota mail_uid = vmail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = maildir:/srv/dovecot/%%u/maildir:INDEX=~/maildir/shared/%%u:CONTROL=~/maildir/shared/%%u prefix = shared.%%n. separator = . subscriptions = yes type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Entw?rfe { special_use = \Drafts } mailbox "Gesendete Objekte" { special_use = \Sent } mailbox Sent { special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/srv/dovecot/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars="@" quota = dict:User quota::file:%h/dovecot-quota quota_grace = 10%% quota_rule = *:storage=100M quota_rule2 = INBOX.Trash:storage=+10M quota_status_nosuser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full / Mailbox ist voll quota_status_success = DUNNO sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0777 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = 127.0.0.1,::1 port = 12340 } } ssl_cert = < ssl_dh_parameters_length = 2048 ssl_key = < ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { args = username_format=%n /etc/dovecot/shared-mailboxes default_fields = uid=600 gid=600 home=/srv/dovecot/%n driver = passwd-file } protocol lmtp { mail_plugins = acl fts fts_lucene virtual zlib quota sieve } protocol imap { mail_max_userip_connections = 30 mail_plugins = acl fts fts_lucene virtual zlib quota imap_acl imap_zlib imap_quota namespace inbox { location = mailbox Spam { autoexpunge = 90 days } mailbox Trash { autoexpunge = 370 days } prefix = } } From dovecot at mx24.net Sun May 29 10:50:05 2016 From: dovecot at mx24.net (Vince42) Date: Sun, 29 May 2016 12:50:05 +0200 Subject: IMAP Timeouts In-Reply-To: <1273648539.1317.1464505129187@appsuite-dev.open-xchange.com> References: <7954926f-eb37-ef42-cca5-57cccb5bc5e9@mx24.net> <1273648539.1317.1464505129187@appsuite-dev.open-xchange.com> Message-ID: <330d61e5-a198-3511-e087-830653df23d3@mx24.net> Hi, [aki.tuomi at dovecot.fi] - [29.05.2016 08:58] > Also make sure you are not facing simple firewall or interconnectivity issue. The server is running well all the time, except for these "hickups". I will also try Alexander's hint. > Also usually it helps if you provide outputs from > doveconf -n > doveadm log errors Here is my doveconf -n (doveadm log shows only old errors, nothing in relation to the timeouts): # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-68-generic x86_64 Ubuntu 14.04.4 LTS auth_mechanisms = plain login auth_username_format = %n debug_log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve pop3 service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = Is there a way to log SNI hostname used in TLS session? Info is there in SSL_CTX_set_tlsext_servername_callback, dovecot copies it to ssl_io->host. Unfortunately I don't see it expanded to any variables ( http://wiki.dovecot.org/Variables ). Please consider this to be a feature request. The goal is to be able to see which hostname client used like: May 30 08:21:19 xxx dovecot: pop3-login: Login: user=, method=PLAIN, rip=1.1.1.1, lip=2.2.2.2, mpid=17135, TLS, SNI=pop3.somehost.org, session= -- Arkadiusz Mi?kiewicz, arekm / ( maven.pl | pld-linux.org ) From pali.rohar at gmail.com Mon May 30 12:56:18 2016 From: pali.rohar at gmail.com (Pali =?utf-8?B?Um9ow6Fy?=) Date: Mon, 30 May 2016 14:56:18 +0200 Subject: [PATCH 0/7] Fixes for lib-mail message-address In-Reply-To: <20160512112436.GB29844@pali> References: <1459789983-10827-1-git-send-email-pali.rohar@gmail.com> <1659571648.16346.720408d5-801e-44fb-ac8f-1881f8bd92d1.open-xchange@appsuite.open-xchange.com> <201604041935.23571@pali> <201604061629.43612@pali> <20160411104402.GR8413@pali> <5ECC7DDB-9497-458A-8891-569386939F9D@iki.fi> <20160512112436.GB29844@pali> Message-ID: <20160530125618.GA29844@pali> On Thursday 12 May 2016 13:24:36 Pali Roh?r wrote: > On Thursday 21 April 2016 14:24:09 Timo Sirainen wrote: > > On 11 Apr 2016, at 13:44, Pali Roh?r wrote: > > > > > > On Thursday 07 April 2016 16:11:12 Timo Sirainen wrote: > > >> On 06 Apr 2016, at 17:29, Pali Roh?r wrote: > > >>> > > >>> On Monday 04 April 2016 19:35:22 Pali Roh?r wrote: > > >>>> On Monday 04 April 2016 19:31:06 aki.tuomi at dovecot.fi wrote: > > >>>>> Can you please send them directly to me, please? > > >>>>> --- > > >>>>> Aki Tuomi > > >>>> > > >>>> Sent. > > >>> > > >>> Received? Are patches OK now? > > >> > > >> It would be nice if each commit was accompanied with the corresponding unit test change in test-message-address.c. Now that the unit test changes are done in a separate commit I'm not really sure which test change is testing which commit or if some tests are missing. At least the "Quote and escape strings if needed" patch needs more tests for different kinds of escapes/atext/non-atext chars and =? in strings. > > >> > > > > > > Ok. If you really need that I can rework my patches. But I'm thinking > > > what to do with tests... I have written lot of examples and tests > > > scenarios in perl for that new perl module. But dovecot test framework > > > is not easy to use as perl Test::More/Test::Spec and such complicated > > > tests which I have are really hard to write directory for dovecot... > > > > Although it would be nice to have a lot of unit tests for everything, I think it would be enough to have just some to show what the change actually fixes. Maybe even just in the git commit message. I remember that by looking at the patches I didn't completely understand what all of the changes did. > > Hi! Here are examples/test cases which patches fixes. Patches 3 and 4 > should be self-explained... It is enough? Or do you need more detailed > description about problems in parser/formatter? > > PATCH 1: > > struct: > { { name = NULL, mailbox = "group", domain = NULL}, { name = NULL, mailbox = NULL, domain = NULL } } > > should be formatted as string: > > group:; > > before patch it was: > > group; > > > > PATCH 2: > > struct: > { { name = NULL, mailbox = "", domain = NULL}, { name = NULL, mailbox = NULL, domain = NULL } } > > should be formatted as string: > > "":; > > before patch it was: > > :; > > > > PATCH 5: > > input string: > > test > > will be parsed as struct: > > { name = "test", mailbox = NULL, domain = NULL } > > before patch it was struct: > > { name = NULL, mailbox = "test", domain = NULL } > > > > PATCH 6: > > struct: > { name = "test\"test", mailbox = "user", domain = "host" } > > should be formatted as string: > > "test\"test" > > before patch it was: > > test"test > > Timo, it is enough? Or do you need something more? Please let me know. -- Pali Roh?r pali.rohar at gmail.com From hs at schlittermann.de Mon May 30 15:17:05 2016 From: hs at schlittermann.de (Heiko Schlittermann) Date: Mon, 30 May 2016 17:17:05 +0200 Subject: doveadm-server protocol change? Message-ID: <20160530151705.GF4691@jumper.schlittermann.de> Hi, I'm doing quota checks from a remote machine (the real setup is a bit more complex, if necessary I can explain it in more detail, but I just extracted the bits that are easily reproduceable) # nc backend1 24245 VERSION doveadm-server 1 0 PLAIN agrVMDvHgz0ya2HHzax5svwB2ZHS? + heiko quota get But since the backend is upgraded to 2.2.22 it's not possible anymore. The exuse in the log of the backend is: dovecot: doveadm(149.1.1.1?): Fatal: USER environment is missing and -u option not used Running the `doveadm quota get -u heiko` locally on the backend works as expected. But using the the doveadm-server it doesn't. # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-22-generic x86_64 Ubuntu 16.04 LTS ? auth_cache_negative_ttl = 0 auth_cache_ttl = 0 auth_master_user_separator = * base_dir = /run/dovecot/ imap_metadata = yes lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_attribute_dict = file:%h/dovecot-attributes mail_location = maildir:~:INBOX=/volumes/dovecot/inbox/%2.256Nn/%n:INDEX=/volumes/dovecot/cache/%2.256Nn/%n mail_plugins = quota mbox_md5 = all mmap_disable = yes namespace inbox { ? passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.passdb.conf.ext driver = ldap } plugin { quota = maildir:User quota quota_grace = 10%% sieve = /volumes/dovecot/sieve/%2.256Nn/%n/.dovecot.sieve sieve_dir = /volumes/dovecot/sieve/%2.256Nn/%n } pop3_uidl_format = %v.%u protocols = " imap lmtp pop3" service auth { extra_groups = ssl-cert } service doveadm { inet_listener { port = 24245 } } service lmtp { inet_listener lmtp { port = 2525 } } ssl = required ssl_cert = From aki.tuomi at dovecot.fi Mon May 30 15:49:53 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Mon, 30 May 2016 18:49:53 +0300 (EEST) Subject: doveadm-server protocol change? In-Reply-To: <20160530151705.GF4691@jumper.schlittermann.de> References: <20160530151705.GF4691@jumper.schlittermann.de> Message-ID: <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> > On May 30, 2016 at 6:17 PM Heiko Schlittermann wrote: > > > Hi, > > I'm doing quota checks from a remote machine (the real setup is a bit > more complex, if necessary I can explain it in more detail, but I just > extracted the bits that are easily reproduceable) > > # nc backend1 24245 > VERSION doveadm-server 1 0 > PLAIN agrVMDvHgz0ya2HHzax5svwB2ZHS? > + > heiko quota get > > But since the backend is upgraded to 2.2.22 it's not possible anymore. > The exuse in the log of the backend is: > > dovecot: doveadm(149.1.1.1?): Fatal: USER environment is missing and -u option not used > > Running the `doveadm quota get -u heiko` locally on the backend works as > expected. But using the the doveadm-server it doesn't. > > # 2.2.22 (fe789d2) Hi! This has been fixed in 2.2.24. There was a bug in user passing. We also invite you to have a go at our HTTP based interface, see http://wiki.dovecot.org/Design/DoveadmProtocol/HTTP --- Aki Tuomi From aki.tuomi at dovecot.fi Mon May 30 15:52:10 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Mon, 30 May 2016 18:52:10 +0300 (EEST) Subject: doveadm-server protocol change? In-Reply-To: <20160530151705.GF4691@jumper.schlittermann.de> References: <20160530151705.GF4691@jumper.schlittermann.de> Message-ID: <1458795285.1300.1464623531845@appsuite-dev.open-xchange.com> > On May 30, 2016 at 6:17 PM Heiko Schlittermann wrote: > > > Hi, > > I'm doing quota checks from a remote machine (the real setup is a bit > more complex, if necessary I can explain it in more detail, but I just > extracted the bits that are easily reproduceable) > > # nc backend1 24245 > VERSION doveadm-server 1 0 > PLAIN agrVMDvHgz0ya2HHzax5svwB2ZHS? > + > heiko quota get > > But since the backend is upgraded to 2.2.22 it's not possible anymore. > The exuse in the log of the backend is: > > dovecot: doveadm(149.1.1.1?): Fatal: USER environment is missing and -u option not used > > Running the `doveadm quota get -u heiko` locally on the backend works as > expected. But using the the doveadm-server it doesn't. > > # 2.2.22 (fe789d2) Hi! This has been fixed in 2.2.24. There was a bug in user passing. We also invite you to have a go at our HTTP based interface, see http://wiki.dovecot.org/Design/DoveadmProtocol/HTTP --- Aki Tuomi From hs at schlittermann.de Mon May 30 18:54:50 2016 From: hs at schlittermann.de (Heiko Schlittermann) Date: Mon, 30 May 2016 20:54:50 +0200 Subject: doveadm-server protocol change? In-Reply-To: <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> Message-ID: <20160530185450.GH4691@jumper.schlittermann.de> Hi Aki, thank your for responding that fast. aki.tuomi at dovecot.fi (Mo 30 Mai 2016 17:49:53 CEST): ? > Hi! This has been fixed in 2.2.24. There was a bug in user passing. Ok, thus at least your answer saves me hours of debugging. We upgraded old Ubuntu Boxes (14.04/LTS) to 16.04 to get around some Dovecot limitations/problems. And now we got new ones :( We did the upgrade to avoid self-built binaries and use the distro packages. Probably we're out of luck. > We also invite you to have a go at our HTTP based interface, see http://wiki.dovecot.org/Design/DoveadmProtocol/HTTP Does the 2.2.22 HTTP API suffer from the same bug with user passing? Is the HTTP API useable in a Director/Backend configuration? Our setup is about [ mx ] ---> dovecot-protocol ---> [ director ] ----> [ backend ] `---> [ backend ] `-> [ backend ] The bug w/ user passing is in the backend? Or already in the directors? -- Heiko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From aki.tuomi at dovecot.fi Mon May 30 18:57:58 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Mon, 30 May 2016 21:57:58 +0300 (EEST) Subject: doveadm-server protocol change? In-Reply-To: <20160530185450.GH4691@jumper.schlittermann.de> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> <20160530185450.GH4691@jumper.schlittermann.de> Message-ID: <304527445.583.1464634679358@appsuite-dev.open-xchange.com> > On May 30, 2016 at 9:54 PM Heiko Schlittermann wrote: > > > Hi Aki, > > thank your for responding that fast. > > aki.tuomi at dovecot.fi (Mo 30 Mai 2016 17:49:53 CEST): > ? > > Hi! This has been fixed in 2.2.24. There was a bug in user passing. > > Ok, thus at least your answer saves me hours of debugging. We upgraded > old Ubuntu Boxes (14.04/LTS) to 16.04 to get around some Dovecot > limitations/problems. And now we got new ones :( > > We did the upgrade to avoid self-built binaries and use the distro > packages. Probably we're out of luck. > > > We also invite you to have a go at our HTTP based interface, see http://wiki.dovecot.org/Design/DoveadmProtocol/HTTP > > Does the 2.2.22 HTTP API suffer from the same bug with user passing? > Is the HTTP API useable in a Director/Backend configuration? > > Our setup is about > > [ mx ] ---> dovecot-protocol ---> [ director ] ----> [ backend ] > `---> [ backend ] > `-> [ backend ] > > The bug w/ user passing is in the backend? Or already in the directors? > -- > Heiko Hi! You can get packages from http://xi.dovecot.fi/debian/, if it helps. The HTTP API should not suffer from the username problem. Aki From hs at schlittermann.de Mon May 30 19:18:09 2016 From: hs at schlittermann.de (Heiko Schlittermann) Date: Mon, 30 May 2016 21:18:09 +0200 Subject: doveadm-server protocol change? In-Reply-To: <304527445.583.1464634679358@appsuite-dev.open-xchange.com> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> <20160530185450.GH4691@jumper.schlittermann.de> <304527445.583.1464634679358@appsuite-dev.open-xchange.com> Message-ID: <20160530191809.GI4691@jumper.schlittermann.de> Hi Aki, aki.tuomi at dovecot.fi (Mo 30 Mai 2016 20:57:58 CEST): ? > You can get packages from http://xi.dovecot.fi/debian/, if it helps. The HTTP API should not suffer from the username problem. Thank you. I just used ppa:patrickdk/production, but probably will try the xi.dovecot.fi packages. With 2.2.24 it works as expected. Due to the project state I'll not try the HTTP API right now (as the MTA (Exim) already speaks successful with the directors (via a Perl extension in Exim). Again, thank you for your instant help. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From dovecot at mx24.net Mon May 30 19:22:55 2016 From: dovecot at mx24.net (Vince42) Date: Mon, 30 May 2016 21:22:55 +0200 Subject: IMAP Timeouts In-Reply-To: References: <7954926f-eb37-ef42-cca5-57cccb5bc5e9@mx24.net> Message-ID: Hi, [Alexander Dalloz] - [29.05.2016 00:43] > At time when the SSL connection cannot be established check the value > of available entropy by cat /proc/sys/kernel/random/entropy_avail In > case you see a low value you should consider running a daemon like > haveged. I had the issue yesterday again and the available entropy has values between 1300 and 1500 - seems to be fine to me ... any other ideas, what I could check or monitor? -- Cheers, \\|// Vince (o o) ----------------------------ooO-(_)-Ooo------------------------- ''' (o)_(o) [ ][0][ ] ??? (=?o?=) World Domination by Copy and Paste [ ][ ][0] - (")_(") [0][0][0] () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Ooo. ---------------------------.ooO----( )------------------------- ( ) (_/ \_) From hs at schlittermann.de Mon May 30 19:26:12 2016 From: hs at schlittermann.de (Heiko Schlittermann) Date: Mon, 30 May 2016 21:26:12 +0200 Subject: doveadm-server protocol change? In-Reply-To: <20160530191809.GI4691@jumper.schlittermann.de> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> <20160530185450.GH4691@jumper.schlittermann.de> <304527445.583.1464634679358@appsuite-dev.open-xchange.com> <20160530191809.GI4691@jumper.schlittermann.de> Message-ID: <20160530192612.GJ4691@jumper.schlittermann.de> Heiko Schlittermann (Mo 30 Mai 2016 21:18:09 CEST): > Hi Aki, > > aki.tuomi at dovecot.fi (Mo 30 Mai 2016 20:57:58 CEST): > ? > > You can get packages from http://xi.dovecot.fi/debian/, if it helps. The HTTP API should not suffer from the username problem. > > Thank you. I just used ppa:patrickdk/production, but probably will try > the xi.dovecot.fi packages. The question is, which of these locations is more trustworthy in the sense of 'production ready'? -- Heiko -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From aki.tuomi at dovecot.fi Mon May 30 20:06:29 2016 From: aki.tuomi at dovecot.fi (aki.tuomi at dovecot.fi) Date: Mon, 30 May 2016 23:06:29 +0300 (EEST) Subject: doveadm-server protocol change? In-Reply-To: <20160530192612.GJ4691@jumper.schlittermann.de> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> <20160530185450.GH4691@jumper.schlittermann.de> <304527445.583.1464634679358@appsuite-dev.open-xchange.com> <20160530191809.GI4691@jumper.schlittermann.de> <20160530192612.GJ4691@jumper.schlittermann.de> Message-ID: <1646393565.595.1464638790584@appsuite-dev.open-xchange.com> > On May 30, 2016 at 10:26 PM Heiko Schlittermann wrote: > > > Heiko Schlittermann (Mo 30 Mai 2016 21:18:09 CEST): > > Hi Aki, > > > > aki.tuomi at dovecot.fi (Mo 30 Mai 2016 20:57:58 CEST): > > ? > > > You can get packages from http://xi.dovecot.fi/debian/, if it helps. The HTTP API should not suffer from the username problem. > > > > Thank you. I just used ppa:patrickdk/production, but probably will try > > the xi.dovecot.fi packages. > > The question is, which of these locations is more trustworthy in the > sense of 'production ready'? > > -- > Heiko I'd consider xi.dovecot.fi more reliable myself. AKi From jkamp at amazon.nl Mon May 30 13:41:52 2016 From: jkamp at amazon.nl (van der Kamp, John) Date: Mon, 30 May 2016 13:41:52 +0000 Subject: Bug with shared access to mailbox Message-ID: Hello, I'm testing dovecot with some setups, and one of them is with shared mailboxes. The test I wrote will create and delete mail using multiple connections to the same user and folder. Each connection makes a couple of mails, remembers the uid from APPENDUID, and will delete those emails again. At the end of the test I expect an empty folder. This is not what happens. At the end I still have several mails in the folder. I lack insight in the dovecot source to tell exactly what's going on. I've tested this with different setups: 1) local system user, connecting over localhost -> bug is present 2) local system user, connecting over internet -> bug is present, but is harder to reproduce 3) dovecot as proxy to another imap server -> bug is present In step 3, you can even setup a dovecot to be a proxy to another dovecot server. >From logging in the other imap server I've seen that a client command to the proxy like: TAG UID STORE 1:3 +FLAGS (\Deleted) TAG UID EXPUNGE 1:3 will be sent to the other imap server in 3 steps, one for each message. When running the test with multiple threads, that logging shows that some uids are never sent to the other imap server, and some uids are sent over different connections than they original were sent to. (Thread 1 deletes 1:3, Thread 2 deletes 4:6, the proxy of Thread 1 might expunge messages from Thread 2 and vice versa). Attached is a python script which tests the behavior. The script expects a file named "testmail.eml" to upload to the imap server. I used an email which was about 75 kB. I tested using version: 2.2.22 (fe789d2). Let me know if I can help in any other way too. John -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dovecot-threading-bug.py URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5332 bytes Desc: not available URL: From pch at myzel.net Tue May 31 08:31:50 2016 From: pch at myzel.net (Peter Chiochetti) Date: Tue, 31 May 2016 10:31:50 +0200 Subject: Ubuntu package - Was: Re: doveadm-server protocol change? In-Reply-To: <1646393565.595.1464638790584@appsuite-dev.open-xchange.com> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> <20160530185450.GH4691@jumper.schlittermann.de> <304527445.583.1464634679358@appsuite-dev.open-xchange.com> <20160530191809.GI4691@jumper.schlittermann.de> <20160530192612.GJ4691@jumper.schlittermann.de> <1646393565.595.1464638790584@appsuite-dev.open-xchange.com> Message-ID: <574D4BF6.3060400@myzel.net> >>> ? >>>> You can get packages from http://xi.dovecot.fi/debian/, if it helps. The HTTP API should not suffer from the username problem. >>> >>> Thank you. I just used ppa:patrickdk/production, but probably will try >>> the xi.dovecot.fi packages. >> >> The question is, which of these locations is more trustworthy in the >> sense of 'production ready'? >> >> Heiko > > I'd consider xi.dovecot.fi more reliable myself. > > AKi > Not having installed any of the two, I can say, as a Ubuntu user: In ppa "/etc/init.d/dovecot" is a symlink to "/lib/init/upstart-job" While xi packages places its own init script there. Curiously, dpkg on installation seems not to unlink the existing one first, but overwrite it with the new contents, thereby destroying upstart -- This happened to me last year, I noticed early :) Last checked in 2.2.22 xi package as seen from the contents, did that change? Possible workaround: remove the stock init file link ahead of installation? -- peter From hs at schlittermann.de Tue May 31 12:32:41 2016 From: hs at schlittermann.de (Heiko Schlittermann) Date: Tue, 31 May 2016 14:32:41 +0200 Subject: Ubuntu package - Was: Re: doveadm-server protocol change? In-Reply-To: <574D4BF6.3060400@myzel.net> References: <20160530151705.GF4691@jumper.schlittermann.de> <1264900304.1298.1464623395647@appsuite-dev.open-xchange.com> <20160530185450.GH4691@jumper.schlittermann.de> <304527445.583.1464634679358@appsuite-dev.open-xchange.com> <20160530191809.GI4691@jumper.schlittermann.de> <20160530192612.GJ4691@jumper.schlittermann.de> <1646393565.595.1464638790584@appsuite-dev.open-xchange.com> <574D4BF6.3060400@myzel.net> Message-ID: <20160531123241.GN4691@jumper.schlittermann.de> Hi, Peter Chiochetti (Di 31 Mai 2016 10:31:50 CEST): > Not having installed any of the two, I can say, as a Ubuntu user: > In ppa "/etc/init.d/dovecot" is a symlink to "/lib/init/upstart-job" The 2.2.24 on 16.04 installs both /etc/init.d/dovecot /lib/systemd/system/dovecot.service > While xi packages places its own init script there. The xi packages I didn't check yet. Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From help at pdscc.com Tue May 31 15:50:27 2016 From: help at pdscc.com (Harondel J. Sibble) Date: Tue, 31 May 2016 08:50:27 -0700 Subject: sieve vacation script exclude based on sender email address Message-ID: <20160531155031.49E415A1C81@sinclaire.sibble.net> I thought I'd asked this question a few years ago but can't seem to find any eveidence of that so here goes. I've been looking at the sieve docs and recipes, done a lot of googling but no joy so far. Using stanard vacation script and that works great, however I want to exclude certain sender email addressess from ever receiving a vacation autoresponse, how do I go about adding that to my existing vacation recipe. I suspect my search terminology is what is causing me not to find anything as I typically am using exclude and similar search terms. -- Harondel J. Sibble Sibble Computer Consulting Ltd. Creating Solutions for the small and medium business computer user. harondel at pdscc.com (use pgp keyid 0x3CC3CFCE not 0x3AD5C11D) http://www.pdscc.com Blog: http://www.pdscc.com/blog (604) 739-3709 (voice) From thenut at bytemonkey.net Tue May 31 20:57:57 2016 From: thenut at bytemonkey.net (dovecot) Date: Tue, 31 May 2016 23:57:57 +0300 Subject: cool stuff Message-ID: <00001dafa9a7$585b07b7$5b75d367$@bytemonkey.net> Hi, I've read about some cool things recently and I though you might be inerested too, here is the link Yours, dovecot From rs at sys4.de Tue May 31 22:56:56 2016 From: rs at sys4.de (Robert Schetterer) Date: Wed, 1 Jun 2016 00:56:56 +0200 Subject: stat .../.dovecot.sieve/tmp failed: Not a directory Message-ID: <574E16B8.5050604@sys4.de> Hi, i have stat .../.dovecot.sieve/tmp failed: Not a directory using sieve = file:~/sieve;active=~/.dovecot.sieve dovecot_2.2.24-1-auto-38 cause i can subscribe to dovecot sieve cant remember to see this in previous versions any idea ? Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Schlei?heimer Stra?e 26/MG, 80333 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From stephan at rename-it.nl Tue May 31 23:04:26 2016 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 1 Jun 2016 01:04:26 +0200 Subject: stat .../.dovecot.sieve/tmp failed: Not a directory In-Reply-To: <574E16B8.5050604@sys4.de> References: <574E16B8.5050604@sys4.de> Message-ID: Op 6/1/2016 om 12:56 AM schreef Robert Schetterer: > Hi, i have > > stat .../.dovecot.sieve/tmp failed: Not a directory > > using > > sieve = file:~/sieve;active=~/.dovecot.sieve > > dovecot_2.2.24-1-auto-38 > > cause i can subscribe to dovecot sieve > > cant remember to see this in previous versions That doesn't make much sense at all. Can you enable mail_debug and show the log for one full delivery attempt? Regards, Stephan.