January 2017 - dovecot

fts_solr and connection via https://
by Jan Vonde 04 Mar '17

04 Mar '17

Hi, I am trying to get fts_solr working and my index server is available via HTTPS only. Dovecot is running on a Debian Jessie system and the Solr server has a letsencrypt certificate. My dovecot version is: 2.2.devel (a9ed8ae) The current setup is: 10-mail.conf: mail_plugins = fts fts_solr 90-fts.conf: plugin { fts = solr fts_autoindex = yes fts_solr = url=https://foo.example.com/solr/dovecot/ } When I try to index the mailboxes I am getting error messages like this: doveadm(user@host): Error: fts_solr: Lookup failed: 9002 Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) doveadm(user@host): Error: Mailbox INBOX: Status lookup failed: Internal error occurred. Refer to server log for more information. [2017-01-22 09:52:38] Segmentation fault Contacting the index server via curl on the command line on the same host works, it returns HTTP 200: user@host ~ $ curl -s -o /dev/null -w "%{http_code}" https://foo.example.com/solr/ 200 user@host ~ $ Currently I have the following ssl related settings: user@host ~ $ doveconf -n -P | grep -i ssl ssl_cert = </etc/ssl/certs/mail.example.org.crt ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_key = </etc/ssl/private/mail.example.org.key ssl_protocols = !SSLv2 !SSLv3 I tried adding the following settings but that didn't help: ssl_ca = < /etc/ssl/certs/ca-certificates.crt ssl_client_ca_dir = /etc/ssl/certs Can you give me a hint how I can get the ssl certificate accepted? Thanks in advance and have a nice day, Jan :-)

2 10

Sieve removeflag Action
by Thomas Leuxner 26 Feb '17

26 Feb '17

Hi, I recently noticed that some of my automatically processed mails do no longer show flags with current Dovecot builds. This used to work before, not sure what broke it or whether I used it in the wrong way. When the removeflag line is dropped *both* messages get flagged in the client, one in INBOX and one in Trash. With removeflag in place, *both* messages are unflagged, the one in INBOX and the one in Trash. This used to work differently before if memory doesn't fail me. #Test if address :is "From" "user(a)example.com" { addflag "\\Flagged $MailFlagBit1"; keep; removeflag "\\Flagged $MailFlagBit1"; fileinto "Trash"; } # 2.2.devel (01867a4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (3c071a4) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 Regards Thomas

4 18

Sieve Script Replication Gliches (Report #2)
by Reuben Farrelly 26 Feb '17

26 Feb '17

Hi, I've observed some odd behaviour with dsync replication between two hosts, specifically to do with sieve script replication. In short, I have two hosts which replicate in a master-master type setup where almost all of the reads and writes happen to just one of the two hosts. They are both running 2.2.devel (9dc6403), which is close to the latest 2.2 -git . Pigeonhole is running master-0.4 . This is on x86_64 Gentoo. Normal mail replication between Maildir's for all users works fine, however it appears that something recently committed to the code has broken sieve script replication between the two. I am sure this did once work. Replication is via tcps: . Sieve scripts on the lesser-used host are not up to date by an order of days/weeks with the main host and they don't seem to re-replicate - even if the rules don't exist at all on the replica. The symptoms and effects look to be the same as this (unanswered) post from December: http://dovecot.org/list/dovecot/2015-December/102690.html I am not sure how to view the transaction log files, but I am seeing the same symptoms, ie no live replication, and on the lesser-used host almost all the scripts were old and some had the 1970 date on them. Even after forcing a [dsync replication replicate '*'] the scripts are not replicated. As it stands now there are no sieve scripts on one of the two members and the system seems unable to replicate by itself. Secondly, I am also seeing some doubled up outputs if I run 'doveadm sieve list -A': thunderstorm reuben # doveadm sieve list -A reuben rules ACTIVE liam rules ACTIVE kaylene rules ACTIVE reuben rules ACTIVE liam rules ACTIVE kaylene rules ACTIVE ... Has anyone else experienced the replication problem? Are sieve scripts actually replicating in live time for other 2.2.24/2.2.25 users as well? For me I didn't notice this till I went looking so I wonder if other people are experiencing this but just not aware of it yet...? Reuben

4 9

Replication sieve scripts.
by Luescher Claude 26 Feb '17

26 Feb '17

Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html I have the exact same issue with 2.2.10: ii dovecot-antispam 2.0+20130822-2 amd64 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.10-1 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.10-1 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.10-1 amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.10-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.10-1 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.10-1 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.10-1 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.10-1 amd64 secure POP3/IMAP server - Sieve filters support As even his version should already support sieve replication through dsync mine would have to support it for sure but I would like to hear a confirmation from the developers. The mail sync just works perfectly between the 2 nodes, it is only the sieve scripts which don't get replicated. I have tried to add additional sieve related options to the replicator: service replicator { process_min_avail = 1 } plugin { mail_replica = tcp:1.2.3.4 replication_full_sync_interval = 1 hours #Added (makes no difference) sieve = ~/.dovecot.sieve sieve_dir = ~/sieve_after sieve_extensions = +vnd.dovecot.duplicate sieve_duplicate_period = 1h #Added (makes no difference) } service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } It did not help. If there is one way I would like to get this work without upgrading dovecot. Thank you.

4 12

Pigeonhole External pipe script going zombie?
by Matt Simpson 23 Feb '17

23 Feb '17

I’m running dovecot 2.2.27 and pigeonhole 0.4.16 on FreeBSD 11. I’m using the pigeonhole/sieve external pipe plugin to run a Perl program to send a Pushover notification when certain messages are received. The Perl script is executed, and the notification is sent. But then the script task seems to go zombie until it is killed after a timeout. In the user’s sieve log, I get a message like error: msgid=<20170112191921.66140.qmail(a)v1.redhorse.me>: pipe action: failed to pipe message to program `sievepush.pl': refer to server log for more information. [2017-01-12 14:19:36]. (even though the message really was piped to the program successfully) In the dovecot server log, I see Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: sieve: Executing script from `/usr/home/matt/maildoms/.dovecot.svbin' Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: sieve: action pipe: running program: sievepush.pl Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: Mailbox stdin: Opened mail UID=1 because: mail stream Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: waiting for program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl' to finish after 0 msecs Jan 12 14:19:31 v1 dovecot: lda(matt): Debug: program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl'(66145) execution timed out after 10000 milliseconds: sending TERM signal Jan 12 14:19:36 v1 dovecot: lda(matt): Debug: program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl' (66145) did not die after 5000 milliseconds: sending KILL signal In the process list during that 10 second interval, I see matt 66142 29972 801 801 0 S - 0:00.00 bin/qmail-local -- matt /home/matt/maildoms jmn-matt - jmn-m matt 66143 66142 801 801 0 S - 0:00.00 /var/qmail/bin/preline -f /usr/local/libexec/dovecot/dovecot matt 66144 66143 801 801 0 S - 0:00.01 /usr/local/libexec/dovecot/dovecot-lda matt 66145 66144 801 801 0 Z - 0:00.65 <defunct> I’m not a Unix programming ace, but from what I’ve been able to find out, this seems to mean that the lda process is forking another process to run the pipe script, and not getting the proper notification when it finishes (not issuing a wait?). So after 10 seconds, it sends a TERM to the task which is no longer running, and when that doesn’t work, it sends a KILL. Anybody know what’s happening here? doveconf -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 11.0-RELEASE-p2 amd64 auth_verbose = yes default_vsz_limit = 128 M lock_method = flock mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = prefix = } passdb { args = imap driver = pam } plugin { recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_pipe_exec_timeout = 10s sieve_plugins = sieve_extprograms } protocols = imap service auth { unix_listener auth-master { group = qnofiles mode = 0660 user = alias } user = root } service imap-login { process_min_avail = 3 vsz_limit = 94 M } ssl_cert = </usr/local/etc/letsencrypt/live/dummy.redhorse.me/cert.pem ssl_key = # hidden, use -P to show it syslog_facility = local0 userdb { driver = passwd } verbose_proctitle = yes protocol lda { mail_plugins = " sieve" }

2 3

antispam plugin pipe backend error when moving multiple emails
by Stéphane Cottin 12 Feb '17

12 Feb '17

Hi, I've got some trouble with the dovecot antispam plugin and the pipe backend. I'm using dovecot 2.2.18 with maildirs and zlib compression enabled. When moving 2 or more emails at once from the Junk folder to another one, I always have the following error : "Failed to copy to temporary file" In the server logs : imap(vvv(a)vvv.vvv): Error: read(zlib(/data/Maildir/.test/tmp/1446974366.M123890P936.vvv)) failed: Cached message size smaller than expected (13553 < 13562, box=test, UID=0) The same operation with one email at a time, on the same emails, works as expected. Stéphane

2 2

Prohibit dots in folder names
by Marek Grondecki 10 Feb '17

10 Feb '17

Hello Dovecot Community, I am using Dovecot 2.2.13-12~deb8u1 (Debian Jessie). separator = / listescape plugin is NOT active LAYOUT=fs is NOT active I would like to prohibit a creation of folders with dot "." in the names - so creation of a folder "foo.bar" should NOT be possible. Currently, when I create "foo.bar" in Outlook 2013 it will be created but won't be synchronised with the mail server - Outlook shows it as "foo.bar (Only this computer)". Ideally the user should receive an error message informing that "the folder foo.bar could not be created." Thank you for your help. Regards, Marek Grondecki

3 4

Dovecot auth-worker error after cram-md5 auth
by Poliman - Serwis 03 Feb '17

03 Feb '17

I set up cram-md5 using this tutorial https://wiki2.dovecot.org/HowTo/CRAM-MD5 in /etc/dovecot/dovecot.conf in passdb code block: listen = *,[::] protocols = imap pop3 #auth_mechanisms = plain login cram-md5 auth_mechanisms = cram-md5 plain login #dodana nizej linia ssl = required disable_plaintext_auth = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_privileged_group = vmail postmaster_address = postmaster(a)vps342401.ovh.net ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key ssl_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384[image: :D]HE-RSA-AES128-GCM-SHA256[image: :D]HE-DSS-AES$ ssl_prefer_server_ciphers = yes ssl_dh_parameters_length = 2048 mail_max_userip_connections = 100 passdb { # args = /etc/dovecot/dovecot-sql.conf # driver = sql driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd } userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } Of course I created cram-md5.pwd file. All mails go out and come nicely. But after I want to do default settings by commented out these two lines: driver = passwd-file args = scheme=cram-md5 /etc/dovecot/cram-md5.pwd and uncomment # args = /etc/dovecot/dovecot-sql.conf # driver = sql I can't send emails - I use Thunderbird - get error "logging on server mail.example.com not work out". Error in logs: dovecot: auth-worker(22698): Error: Auth worker sees different passdbs/userdbs than auth server. dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF Is it possible that hashed password from cram-md5.pwd file was written to database (if yes then where - I have ISPconfig)? I wasn't change any userdb {} block and this second userdb block has this same lines like default settings in passdb block. -- *Pozdrawiam / Best Regards* *Piotr Bracha* *tel. 534 555 877* *serwis(a)poliman.pl <serwis(a)poliman.pl>*

4 25

Moving to new password scheme
by ＠lbutlr 02 Feb '17

02 Feb '17

dovecot is setup on a system with MD5-CRYPT password scheme for all users, and I would like to update this to something that is secure, probably SSHA256-CRYPT, but I want to do this seamlessly without the users having to jump through any hoops. The users are in mySQL (managed via postfixadmin) and the mailbox record simply stores the hash in the password field. Users access their accounts though IMAP MUAs or Roundcube. How would I setup my system so that if a user logs in and still has a $1$ password (MD5-CRYPT) their password will be encoded to the new SHCEME and then the SQL row updated with the $5$ password instead? Something where they are redirected after authentication to a page that forces them to renter their password (or choose a new one) is acceptable. And, while I am here, is it worthwhile to set the -r flag to a large number (like something over 100,000 which sets takes about 0.25 seconds to do on my machine)? -- Apple broke AppleScripting signatures in Mail.app, so no random signatures.

4 8

quota-status returns quota_status_success when email would put user over quota
by Kristian Pedersen 02 Feb '17

02 Feb '17

Hi list, I am attempting to get quota-status service working, so I can deny email at the initial smtp dialog instead of generating bounces with lda. I can't seem to get quota-status to return quota_status_overquota, even when an email would put an account over quota. Quota in general works fine: Jan 22 06:39:23 mail dovecot: lda(xx(a)yy.dk): msgid=<25c5bdb20d58fc4f649f716a947613dc(a)zz.org>: save failed to INBOX: Quota exceeded (mailbox for user is full) Jan 22 06:39:23 mail dovecot: lda(xx(a)yy.dk): msgid=<25c5bdb20d58fc4f649f716a947613dc(a)zz.org>: rejected: Quota exceeded (mailbox for user is full) The system is a pretty standard debian 8 box running virtually on a kvm/ovirt cluster: root@mail:~# dpkg --list | grep dovec ii dovecot-core 1:2.2.13-12~deb8u1 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.13-12~deb8u1 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-lmtpd 1:2.2.13-12~deb8u1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-mysql 1:2.2.13-12~deb8u1 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.13-12~deb8u1 amd64 secure POP3/IMAP server - POP3 daemon Mail storage is local XFS partitions as far as the dovecot server knows. Behind the scene its actually raw disk images on NFS shares from a SAN towards the kvm/ovirt cluster. I test by using nc towards the quota-status instance: root@mail:~# printf "recipient=kptest(a)asom-net.dk\nsize=1000000000\n\n" | nc -q1 localhost 12340 action=DUNNO User has 500MB quota: mysql> SELECT CONCAT('/data/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 110 AS uid, 110 AS gid, CONCAT('*:storage=',mailquota,'M') AS quota_rule FROM virtual_users WHERE email='kptest(a)asom-net.dk'; +--------------------------------+-----+-----+----------------+ | home | uid | gid | quota_rule | +--------------------------------+-----+-----+----------------+ | /data/vmail/asom-net.dk/kptest | 110 | 110 | *:storage=500M | +--------------------------------+-----+-----+----------------+ 1 row in set (0.00 sec) maildirsize seems ok: root@mail:~# cat /data/vmail/asom-net.dk/kptest/maildirsize 524288000S 685 1 690 1 Quota-status will return unknown user if that is the case: root@mail:~# printf "recipient=kptest2(a)asom-net.dk\nsize=1000000000\n\n" | nc -q1 localhost 12340 action=551 5.5.1 User not found Log file (with mail_debug = yes): Jan 12 16:52:21 mail dovecot: quota-status: Debug: auth input: kptest(a)asom-net.dk home=/data/vmail/asom-net.dk/kptest uid=110 gid=110 quota_rule=*:storage=500M Jan 12 16:52:21 mail dovecot: quota-status: Debug: Added userdb setting: plugin/quota_rule=*:storage=500M Jan 12 16:52:21 mail dovecot: quota-status(kptest(a)asom-net.dk): Debug: Effective uid=110, gid=110, home=/data/vmail/asom-net.dk/kptest Jan 12 16:52:21 mail dovecot: quota-status(kptest(a)asom-net.dk): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/data/vmail/asom-net.dk/kptest/ Jan 12 16:52:21 mail dovecot: quota-status(kptest(a)asom-net.dk): Debug: maildir++: root=/data/vmail/asom-net.dk/kptest, index=, indexpvt=, control=, inbox=/data/vmail/asom-net.dk/kptest, alt= dovecot -n: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 ext4 auth_default_realm = vejen-net.dk auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no first_valid_uid = 110 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = maildir:/data/vmail/%d/%n/ mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/local-sql.conf driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=200M quota_status_nouser = 551 5.5.1 User not found quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1024 process_limit = 256 process_min_avail = 8 service_count = 0 vsz_limit = 512 M } service imap { process_limit = 10240 } service pop3-login { client_limit = 512 process_limit = 256 process_min_avail = 8 service_count = 0 vsz_limit = 512 M } service pop3 { process_limit = 10240 } service quota-status { client_limit = 1 executable = /usr/lib/dovecot/quota-status -p postfix inet_listener { port = 12340 } } ssl_cert = </etc/letsencrypt/live/mail.asom-net.dk/fullchain.pem ssl_key = </etc/letsencrypt/live/mail.asom-net.dk/privkey.pem ssl_prefer_server_ciphers = yes userdb { args = /etc/dovecot/local-sql.conf driver = sql } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota postmaster_address = postmaster(a)asom-net.dk } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } local-sql.conf: driver = mysql connect = host=xyz dbname=xyz user=xyz password=xyz default_pass_scheme = CRYPT password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; user_query = SELECT CONCAT('/data/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 110 AS uid, 110 AS gid, CONCAT('*:storage=',mailquota,'M') AS quota_rule FROM virtual_users WHERE email='%u'; If I do a strace on the quota-status PID, it seems to do a stat on the directory and then give up? Not sure Im interpreting it correct (only including the last few lines): ... lseek(14, 833, SEEK_SET) = 833 munmap(0x7f165d32a000, 833) = 0 close(14) = 0 geteuid() = 0 getegid() = 110 getgid() = 110 getegid() = 110 setgroups(1, [110]) = 0 setresuid(-1, 110, -1) = 0 prctl(PR_SET_DUMPABLE, 1) = 0 stat("/data/vmail/asom-net.dk/kptest", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 prctl(PR_SET_DUMPABLE, 1) = 0 setsockopt(12, SOL_TCP, TCP_CORK, [1], 4) = 0 write(12, "action=DUNNO\n\n", 14) = 14 setsockopt(12, SOL_TCP, TCP_CORK, [0], 4) = 0 epoll_wait(11, {{EPOLLIN, {u32=1593554016, u64=139734059562080}}}, 5, 59999) = 1 read(12, "", 8146) = 0 epoll_ctl(11, EPOLL_CTL_DEL, 12, 7fff0be817a0) = 0 close(12) = 0 epoll_wait(11, {}, 5, 1000) = 0 write(5, "\35q\1\0007\10\0\0\1\0\0\0", 12) = 12 epoll_wait(11, ... It seems like sort of permission issue? root@mail:~# su - vmail No directory, logging in with HOME=/ $ id -a uid=110(vmail) gid=110(vmail) groups=110(vmail) $ cat /data/vmail/asom-net.dk/kptest/maildirsize 524288000S 685 1 690 1 /data/vmail/asom-net.dk is actually a symlink, maybe that could be of importance?: root@mail:~# ls -ld /data/vmail/asom-net.dk lrwxrwxrwx 1 root root 19 Jan 9 11:18 /data/vmail/asom-net.dk -> ../mnt1/asom-net.dk root@mail:~# ls -ld /data/mnt1/asom-net.dk/ drwxrwx--- 45 vmail vmail 4096 Dec 15 10:54 /data/mnt1/asom-net.dk/ root@mail:~# ls -ld /data/mnt1/asom-net.dk/kptest/ drwx------ 9 vmail vmail 4096 Jan 23 08:55 /data/mnt1/asom-net.dk/kptest/ root@mail:~# ls -ld /data/mnt1/asom-net.dk/kptest/maildirsize -rw------- 1 vmail vmail 23 Jan 12 16:50 /data/mnt1/asom-net.dk/kptest/maildirsize Anyone have any idea what might be wrong here? Regards, -- Kristian Pedersen ASOM-Net Systemadministrator www.asom-net.dk Telefon: 44 400 970

3 5