February 2019 - dovecot

2.3.3: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed
by Tom Sommer 13 Mar '19

13 Mar '19

I see this in my logs after 2.3.3: using zlib plugin, ofc. Oct 02 10:01:39 imap(user(a)example.com)<50643></2k4Sjp3vMqC496W>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) Oct 02 10:01:39 imap(user(a)example.com)<50643></2k4Sjp3vMqC496W>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xce56a) [0x7f442487556a] -> /usr/lib64/dovecot/libdovecot.so.0(+0xce5b1) [0x7f44248755b1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3d941) [0x7f44247e4941] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x5cdf) [0x7f44233c4cdf] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf4b46) [0x7f442489bb46] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_destroy+0x13) [0x7f442489be83] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x173) [0x7f4424b93973] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x36) [0x7f4424b68696] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xede9) [0x55a9d7e86de9] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](command_exec+0x65) [0x55a9d7e956e5] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xf9e6) [0x55a9d7e879e6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f442488c275] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbf) [0x7f442488e13f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x55) [0x7f442488c365] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f442488c588] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4424808053] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](main+0x32d) [0x55a9d7ea20dd] -> /lib64/libc.so.6(__libc_start_main+0x100) [0x7f4424431d20] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xe419) [0x55a9d7e86419] No clue how to reproduce -- Tom

3 2

Assistance with doveadm backup...
by SH Development 11 Mar '19

11 Mar '19

I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south. We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server? Can someone help get me started here? Jeff

7 13

Re: Removing a mailbox from a dovecot cluster
by Francis 08 Mar '19

08 Mar '19

Le mar. 26 févr. 2019 à 22:58, David Myers <david.myers.24j74(a)gmail.com> a écrit : > Hello Francis, > > I wonder if this is due to how a cluster is configured to function > internally. > > Tell us more about the cluster, is it one of those ‘fancy pants’ high > availability, auto backup heart beat things, or is it more a traditional > multi server (master slave style) setup. > > Either way you may need to disconnect the servers from one another and > delete the offending files / directories either via dove or or via the os > (although reading your original email it sounds like you are already > attempting this). > > If you have a fancy cluster this may actually be more difficult than it > sounds and have interesting (unwanted) side effects, also the underlying > database (if you are storing emails that way) may have a method to remove > data > > I assume you are keeping back up copies of all those emails somewhere, > just in case you need them in the future. > > See this wiki article to better understand what I mean by the ‘fancy > pants’ clusters : > https://en.m.wikipedia.org/wiki/High-availability_cluster > They sound very cool, but I suspect are overkill for a mail server, unless > your database is already inside one then it would make sense I guess. > > Hello, I just use the cluster/replication functionality integrated into dovecot, nothing more. There is no database involved. I use LDAP for the authentication. The mails are stored locally on each server and replicated with the replication feature of dovecot. I followed this wiki article: https://wiki.dovecot.org/Replication Thanks! -- Francis

2 8

Pigeonhome Sieve: check existence of a folder?
by AvV 03 Mar '19

03 Mar '19

Dear All, Thanks ofr the great job so far. I have crawled the doc & web, and did not find oh to check for the presence of a folder ina a mailbox? I know about "fileinto :create" of course, but the purpose is slightly different: I want to automate the move into a folder based on some rules but *only if* an associated folder is present (which name is based on the rule), otherwise I will do a form of catch-all. Example: - INBOX +- abc +- def - Rule detects "aaa" in someheader field: folder "aaa" not present" -> continue; - Rule detect "def" in some header field: "def exists" -> fileinto "def" ; stop; Any help appreciated. Cheers, A/

2 2

Unexpected behavior with sieve_vacation_to_header_ignore_envelope = yes
by Matthias Petermann 03 Mar '19

03 Mar '19

Hello, we face an issue with the vacation extension of pigeonhole. Our upstream mail server (from which we pull our E-Mails with POP3) uses SPF and performs a SRS re-write of the Return Path. Once the mail is processed via our local postfix, it is delivered via dovecot deliver (using the lda). When the recipient has a vacation rule enabled in sieve, the out-of-office note cannot be sent, as pigeonhole tries to use the Return-Path (which is not accepted as sender by the mail server). We are using the following dovecot / pigeonhole versions: dovecot-2.3.2.1 dovecot-pigeonhole-0.5.2 As of my understanding, the problem was addressed in pigeonhole version v0.4.21 by introducing the sieve_vacation_to_header_ignore_envelope setting. Anyway - even when this is set to "yes" pigeonhole still uses the Return-Path instead of the "From:" header which I had expected: Feb 27 13:22:26 mail dovecot: lda(customer)<48383><3o+yOQGBdlz/vAAAZU03Dg>: Error: sieve: msgid=<e081d2ba-21d5-c32b-3750-69fe0c8a76d2(a)petermann-it.de>: failed to send vacation response to SRS0=gwGXg6ad=RC=petermann-it.de=matthias(a)example.com: <Sendmail program returned error> (temporary error) Are my expectations wrong or did I miss some precondition? Best regards, Matthias 1) This is the email which I use to test (I replaced the actual domain with example.com) ------>8------------------------------------------------------------------------------ Return-Path: <SRS0=gwGXg6ad=RC=petermann-it.de=matthias(a)example.com> Delivered-To: customer[[(a)example.com]] To: Customer <customer(a)example.com> From: Matthias Petermann <matthias(a)petermann-it.de> Subject: Test Message-ID: <e081d2ba-21d5-c32b-3750-69fe0c8a76d2(a)petermann-it.de> Date: Wed, 27 Feb 2019 05:21:45 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 Test message ------8<------------------------------------------------------------------------------ 2) The related excerpt of the configuration: ------>8------------------------------------------------------------------------------ mail /usr/local/etc/dovecot/conf.d >doveconf | grep -A 10 -B 10 to_header_ignore_envelope result_failure = continue result_internalfail = continue result_success = return-ok skip = never username_filter = } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes sieve = file:/srv/vmail/%u/sieve;active=/srv/vmail/%u/active.sieve sieve_vacation_to_header_ignore_envelope = yes } pop3_client_workarounds = pop3_delete_type = default pop3_deleted_flag = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no ------8<------------------------------------------------------------------------------ -- Matthias Petermann <matthias(a)petermann-it.de>

2 2

dovecot[1511]: auth-worker: Fatal: master: service(auth-worker): child 17009 killed with signal 11
by Patrik 28 Feb '19

28 Feb '19

This bug report is similar to this issue: https://www.dovecot.org/list/dovecot/2017-May/108116.html <https://www.dovecot.org/list/dovecot/2017-May/108116.html> I know it is not enough info, but every kill follows with this info: Feb 28 16:57:50 server dovecot[1511]: auth-worker: Fatal: master: service(auth-worker): child 17009 killed with signal 11 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Feb 28 16:57:50 server kernel: traps: auth[17009] general protection ip:7fd5d3ae9a64 sp:7fff0a781dc0 error:0 in libmariadb.so.3[7fd5d3add000+26000] It looks like the error is related to the libmariadb.so, of course I am using integration with auth sql. The configuration is like this: # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-2-amd64 x86_64 Debian buster/sid ext4 # Hostname: p3x-dc.example.com auth_mechanisms = plain login mail_location = maildir:/var/mail/vhosts/%d/%n mail_plugins = quota mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 30 days special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_grace = 10%% quota_max_mail_size = 100M quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M sieve = /etc/dovecot/sieve.conf } protocols = imap lmtp pop3 service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0666 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } ssl = required ssl_ca = </etc/ssl/example.com/ca.cer ssl_cert = </etc/ssl/example.com/fullchain.cer ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota sieve } protocol imap { mail_plugins = quota imap_quota } *Patrik* WWW <https://patrikx3.com> | GitHub <https://github.com/patrikx3/> | NPM <https://www.npmjs.com/~patrikx3> | Corifeus <https://corifeus.com> | +36 20 342 8046

2 2

Dovecot Logging and Shell Script Hang
by Asai 28 Feb '19

28 Feb '19

Greetings, We have a PHP based shell script that runs a doveadm backup routine. This script was running just fine, and then we turned on more logging for Dovecot (enabled the mail_log plugin). For some reason, once this was done, the shell script would run for a few minutes and then hang, as it added more log messages to the dsync STDOUT. Once the mail_log plugin was disabled, it ran without hanging. Is there any way to run doveadm and temporarily suspend the mail_log plugin? Or is there a better way to do this? Thanks for the insights. Asai

3 3

Dovecot 2.3.3 Mailbox does not exist
by Kunal A. 27 Feb '19

27 Feb '19

HI, I would deeply aprechiate if someone here could help me address a problem with ACL. I would personally refrain from e-mailing ever so often as I understand other users have more important issues to discuss about. Anyways I hope someone here could help. When I run ;- doveadm acl get -A Public/Archive I keep getting an error that says Mailbox does not exist . See erorr message below : - Error: Can't open mailbox Public/Archive: Mailbox doesn't exist: Public/Archive In the fastmail folder there is a folder called Archive, why isn't it being read? The folder is drwxr-xr-x. with vmail as owner. Deeply appreciate if someone could help with this. Many thanks dovecot -n output :- # 2.3.3 (dcead646b): /etc/dovecot/dovecot.conf # OS: Linux 4.20.10-200.fc29.x86_64 x86_64 Fedora release 29 (Twenty Nine) # Hostname: machine auth_mechanisms = plain login mail_location = maildir:~/Maildir mail_plugins = acl mail_privileged_group = mail mbox_write_locks = fcntl namespace { list = children location = maildir:/run/media/computer/Storage/Email/fastmail/ prefix = Public/ separator = / subscriptions = yes type = public } namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile } postmaster_address = postmaster at example.com protocols = imap pop3 service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_cipher_list = PROFILE=SYSTEM ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } protocol imap { mail_plugins = acl imap_acl }

2 2

Re: performance issue with UID SEARCH
by Aki Tuomi 27 Feb '19

27 Feb '19

Without FTS, dovecot needs to open *each* and *every* email when doing text searches, which is understandably rather slow process. Aki On 27.2.2019 10.43, Marc Roos wrote: > > > I am not sure if this any help. From what I understand of maildir it has > lots of separate files, thus uid/gid lookups. Try running something like > nscd, that will cache these lookups? > > > > > -----Original Message----- > From: Aki Tuomi via dovecot [mailto:dovecot@dovecot.org] > Sent: 27 February 2019 06:24 > To: Ben Burke; Dovecot Mailing List > Subject: Re: performance issue with UID SEARCH > > > On 27 February 2019 03:27 Ben Burke via dovecot < > dovecot(a)dovecot.org> wrote: > > > Hi, > > I'm running dovecot 2.2.x and I'm having an issue where I see many > dovecot processes use all the available IO on a server. According > to > iotop the worst offenders seem to be in this state (NOTE: I swapped > in > phony username & IP info): > > dovecot/imap [someusername 123.456.789.012 UID SEARCH] > > The server in question is running with Maildirs on top of an XFS > filesystem. Is there anything I can do to optimize "UID SEARCH" or > find > out why it's being a problem? I've read > https://wiki2.dovecot.org/PerformanceTuning and the linked pages. > > By "being a problem" I mean iostat -xmt 1 /dev/diskdevice shows > 100% > utilization for long periods and in some cases io service times are > > taking many seconds... which causes thunderbird to timeout when > doing > things like appending messages to user "Sent" mailboxes. > > Any ideas? > > Thanks, > Ben Burke > > > Are you using FTS? If not, you should. See > https://wiki.dovecot.org/Plugins/FTS > --- > Aki Tuomi > >

1 0

CASifying dovecot and webmail
by Juan Carlos Sánchez 27 Feb '19

27 Feb '19

Hello: I'm trying to authenticate webmail (roundcube) and dovecot with Apereo CAS. I don't find any good guide and everything I have found is some years old. Has any of you done a recent integration with CAS and can share the details ? Best regards -- ------------------------------------------------------ Juan Carlos Sanchez Hernandez Servicio de Infraestructura e Innovación Universidad Politecnica de Madrid - www.upm.es Rectorado Avda. Ramiro de Maeztu 7 28040 Madrid ------------------------------------------------------

3 2