July 2018 - dovecot - dovecot.org

expunge not removing attachments?
by Ralf Hildebrandt 12 Feb '19

12 Feb '19

I have a large mail backup folder backup(a)backup.invalid; I'm cleaning up daily like this: infimum=`date -d "-4 day" +"%Y-%m-%d"` doveadm expunge -u backup(a)backup.invalid mailbox INBOX SAVEDBEFORE $infimum doveadm purge -u backup(a)backup.invalid yet I see this: # find attachments/ -type f -ctime +5 | wc -l 7522 # find attachments/ -type f | wc -l 127579 # find attachments/ -type f -mtime +5 | wc -l 14361 # find attachments/ -type f | wc -l 127793 About 5.9% of the files in attachments and below are older than 5 days. Why? Is that normal? using dovecot 2:2.3.1-1 from the official repos. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 3

SELinux policy to allow Dovecot to connect to Mysql
by Robert Moskowitz 05 Feb '19

05 Feb '19

As I have noted in previous messages, I been getting the following on my new mailserver: Apr 7 10:17:27 z9m9z dovecot: dict: Error: mysql(localhost): Connect failed to database (postfix): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13) - waiting for 25 seconds before retry They go away when I setenforce 0. It is not a timing issue as I earlier thought. So I googled dovecot mysql selinux and the only worthwhile hit was: http://zszsit.blogspot.com/2012/12/dovecot-mysql-selinux-issue-on-centos6.h… that provides a /etc/selinux/dovecot2mysql.te and other selinux stuff. Is there a simpler way like a setsbool option? With all the howtos on dovecot with mysql, it is interesting that none of them seem to have this problem. Maybe because they connect to mysql through TCP port 3306 which has ITS set of problems (like MariaDB defaults to not listening on TCP). thanks!

1 3

auth: Error - Request timed out
by Hajo Locke 01 Feb '19

01 Feb '19

Hello List, i use dovecot 2.2.22 and have the same problem described here: https://dovecot.org/pipermail/dovecot/2017-November/110020.html I can confirm that sometimes there is a problem with connection to mysql-db, but sometimes not. Reasons for failing are still under investigation by my mates. My current main problem is, that this fail seems to be a one way ticket for dovecot. Even if mysql is verifyable working again and waiting for connection dovecot stucks furthermore with errors like this: May 29 07:00:49 hostname dovecot: auth: Error: plain(mail(a)example.com,xxx.xxx.xx.xxx,<HGI4FVFt2Oy80j3P>): Request 999.7 timed out after 150 secs, state=1 When restarting dovecot all is immediately working again. Is there a way to tell dovecot to restart auth services or reinitialize mysql-connection after these hard fails? I could insert "idle_kill = 1 mins" into service auth and service auth-worker, but i dont know if this would work. Unfortunately i am not able to reproduce this error and there are always a couple of days between errors. Thanks, Hajo

4 6

Freebsd: Fatal error - Support not compiled in for passdb driver 'sql'
by Graham S. Jarvis 14 Jan '19

14 Jan '19

Hello, I have the following fatal error when I uncomment #!include auth-sql.conf.ext in .../conf.d/10-auth.conf dovecot: auth: Fatal: Support not compiled in for passdb driver 'sql' I have installed via 'pkg'. Do I have to install the ports tree to get sql support ? Thanks, -Graham-

4 9

Dovecot Submission Proxy Auth
by Paul Hecker 12 Jan '19

12 Jan '19

Hi, Dovecot 2.3.1 (8e2f634). Could not get Dovecot to forward the (plain) authentication to the SMTP server using submission. Reason why I need it is sender spoofing (do not want my employees to send messages in behalf of me). In exim I can disable sender spoofing with the authenticated user. When sending through dovecot, exim either does not accept the email (need auth) or relay every sender address (because relaying from localhost). Am I missing a setting or do I need any additional field in the (MySQL) user_query/password_query to forward the password? You can find my config here: https://gist.github.com/lluuaapp/7daddf761131da47237b0f45e6bab5a8 Thanks, Paul

5 12

RE: Dovecot (doveadm, ssl, sync) - SSL error
by Arkadiusz Majewski 08 Jan '19

08 Jan '19

Hello! dsync SSL still doesn't work for replication, so I've disabled it and tried to sync without. But I have a problem with temp directory. Is it possible to change path to temp folder? I don't want to set permissions but change temporary folder for replication. Thanks in advance. dovecot: doveadm: Error: safe_mkstemp(/tmp/dovecot.doveadm.) failed: Permission denied Arkadiusz Majewski From: Arkadiusz Majewski <majewsa(a)gmail.com> Sent: Sunday, December 3, 2017 7:38 PM To: 'dovecot(a)dovecot.org' <dovecot(a)dovecot.org> Subject: Dovecot (doveadm, ssl, sync) - SSL error Hello! I've got a problem to run syncing between both dovecot services on the separate servers. The error indicates to the problem with SSL. Directly using openssl command to connect from one server to other and vice versa is passed without any errors. OS: FreeBSD 11.1-RELEASE-p4 Dovecot: 2.2.33.2_2 and the older one dovecot-2.2.32.1_1 (or similar) - build by ports. OpenSSL: 1.0.2k-freebsd 26 Jan 2017 dovecot: doveadm(10.18.1.15): Error: doveadm client disconnected before handshake: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol dovecot.conf (on both servers): mail_plugins = $mail_plugins notify replication service replicator { process_min_avail = 1 } service aggregator { fifo_listener replication-notify-fifo { user = dovecot } unix_listener replication-notify { user = dovecot } } service replicator { unix_listener replicator-doveadm { mode = 0600 } } replication_max_conns = 10 service doveadm { inet_listener { port = 12130 ssl = yes } } ssl = required ssl_protocols = SSLv3 TLSv1 TLSv1.1 TLSv1.2 ssl_cert = </usr/local/etc/dovecot/ssl/cert.crt ssl_key = </usr/local/etc/dovecot/ssl/cert.key ssl_client_ca_file = </usr/local/etc/dovecot/ssl/ca.pem ssl_client_ca_dir = /usr/local/etc/dovecot/ssl Thank you in advance for any help. Kind Regards, Arkadiusz Majewski

2 1

Limit IMAP/IMAPS listening IPs
by Dudi Goldenberg 26 Nov '18

26 Nov '18

Hello list. I am running dovecot v2.2.13 on debian 8.8; I tried to add a "address =" statement for only localhost and one private IP, but then I got a complaint from system: dovecot[20515]: Error: systemd listens on port 143, but it's not configured in Dovecot. Closing. What is the proper way to resolve this? TIA & Regards, Dudi

9 23

doveconf error upon boot up
by Jerry 31 Oct '18

31 Oct '18

FreeBSD 11.1-RELEASE-p9 amd64 doveconf # 2.3.1 (8e2f634): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.1 (d9bc6dfe) # OS: FreeBSD 11.1-RELEASE-p9 amd64 zfs # Hostname: localhost I can send the entire output if necessary. Error message upon boot up: doveconf: error: t_readlink(/var/run/dovecot/dovecot.conf) failed: readlink() failed: No such file or directory Starting dovecot. Dovecot appears to start correctly There is a link in the "/var/run/dovecot directory for dovecot.conf" lrwx------ 1 root wheel 35B 2018-04-10 10:21:52 EDT dovecot.conf@-> /usr/local/etc/dovecot/dovecot.conf This error message has only started since the update to the latest version of dovecot. I am assuming it is just a harmless error message; however, I would like to confirm that hypothesis. -- Jerry

3 4

OCSP Stapling and Certificate Transparency
by Felipe Gasper 31 Oct '18

31 Oct '18

Hi, For CAs that do not include a signed certificate timestamp in their newly-issued certificates, does Dovecot support either OCSP stapling or the Certificate Transparency TLS extension? If the TLS extension is supported, how does the admin configure the timestamp for each certificate? I’m wondering if any MUAs will follow Google’s lead and insist on CT. Thank you! -Felipe Gasper Mississauga, Ontario

4 3

Re: [ceph-users] dovecot + cephfs - sdbox vs mdbox
by Webert de Souza Lima 04 Oct '18

04 Oct '18

Thanks Jack. That's good to know. It is definitely something to consider. In a distributed storage scenario we might build a dedicated pool for that and tune the pool as more capacity or performance is needed. Regards, Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil* *IRC NICK - WebertRLZ* On Wed, May 16, 2018 at 4:45 PM Jack <ceph(a)jack.fr.eu.org> wrote: > On 05/16/2018 09:35 PM, Webert de Souza Lima wrote: > > We'll soon do benchmarks of sdbox vs mdbox over cephfs with bluestore > > backend. > > We'll have to do some some work on how to simulate user traffic, for > writes > > and readings. That seems troublesome. > I would appreciate seeing these results ! > > > Thanks for the plugins recommendations. I'll take the change and ask you > > how is the SIS status? We have used it in the past and we've had some > > problems with it. > > I am using it since Dec 2016 with mdbox, with no issue at all (I am > currently using Dovecot 2.2.27-3 from Debian Stretch) > The only config I use is mail_attachment_dir, the rest lies as default > (mail_attachment_min_size = 128k, mail_attachment_fs = sis posix, > ail_attachment_hash = %{sha1}) > The backend storage is a local filesystem, and there is only one Dovecot > instance > > > > > Regards, > > > > Webert Lima > > DevOps Engineer at MAV Tecnologia > > *Belo Horizonte - Brasil* > > *IRC NICK - WebertRLZ* > > > > > > On Wed, May 16, 2018 at 4:19 PM Jack <ceph(a)jack.fr.eu.org> wrote: > > > >> Hi, > >> > >> Many (most ?) filesystems does not store multiple files on the same > block > >> > >> Thus, with sdbox, every single mail (you know, that kind of mail with 10 > >> lines in it) will eat an inode, and a block (4k here) > >> mdbox is more compact on this way > >> > >> Another difference: sdbox removes the message, mdbox does not : a single > >> metadata update is performed, which may be packed with others if many > >> files are deleted at once > >> > >> That said, I do not have experience with dovecot + cephfs, nor have made > >> tests for sdbox vs mdbox > >> > >> However, and this is a bit out of topic, I recommend you look at the > >> following dovecot's features (if not already done), as they are awesome > >> and will help you a lot: > >> - Compression (classic, https://wiki.dovecot.org/Plugins/Zlib) > >> - Single-Instance-Storage (aka sis, aka "attachment deduplication" : > >> https://www.dovecot.org/list/dovecot/2013-December/094276.html) > >> > >> Regards, > >> On 05/16/2018 08:37 PM, Webert de Souza Lima wrote: > >>> I'm sending this message to both dovecot and ceph-users ML so please > >> don't > >>> mind if something seems too obvious for you. > >>> > >>> Hi, > >>> > >>> I have a question for both dovecot and ceph lists and below I'll > explain > >>> what's going on. > >>> > >>> Regarding dbox format (https://wiki2.dovecot.org/MailboxFormat/dbox), > >> when > >>> using sdbox, a new file is stored for each email message. > >>> When using mdbox, multiple messages are appended to a single file until > >> it > >>> reaches/passes the rotate limit. > >>> > >>> I would like to understand better how the mdbox format impacts on IO > >>> performance. > >>> I think it's generally expected that fewer larger file translate to > less > >> IO > >>> and more troughput when compared to more small files, but how does > >> dovecot > >>> handle that with mdbox? > >>> If dovecot does flush data to storage upon each and every new email is > >>> arrived and appended to the corresponding file, would that mean that it > >>> generate the same ammount of IO as it would do with one file per > message? > >>> Also, if using mdbox many messages will be appended to a said file > >> before a > >>> new file is created. That should mean that a file descriptor is kept > open > >>> for sometime by dovecot process. > >>> Using cephfs as backend, how would this impact cluster performance > >>> regarding MDS caps and inodes cached when files from thousands of users > >> are > >>> opened and appended all over? > >>> > >>> I would like to understand this better. > >>> > >>> Why? > >>> We are a small Business Email Hosting provider with bare metal, self > >> hosted > >>> systems, using dovecot for servicing mailboxes and cephfs for email > >> storage. > >>> > >>> We are currently working on dovecot and storage redesign to be in > >>> production ASAP. The main objective is to serve more users with better > >>> performance, high availability and scalability. > >>> * high availability and load balancing is extremely important to us * > >>> > >>> On our current model, we're using mdbox format with dovecot, having > >>> dovecot's INDEXes stored in a replicated pool of SSDs, and messages > >> stored > >>> in a replicated pool of HDDs (under a Cache Tier with a pool of SSDs). > >>> All using cephfs / filestore backend. > >>> > >>> Currently there are 3 clusters running dovecot 2.2.34 and ceph Jewel > >>> (10.2.9-4). > >>> - ~25K users from a few thousands of domains per cluster > >>> - ~25TB of email data per cluster > >>> - ~70GB of dovecot INDEX [meta]data per cluster > >>> - ~100MB of cephfs metadata per cluster > >>> > >>> Our goal is to build a single ceph cluster for storage that could > expand > >> in > >>> capacity, be highly available and perform well enough. I know, that's > >> what > >>> everyone wants. > >>> > >>> Cephfs is an important choise because: > >>> - there can be multiple mountpoints, thus multiple dovecot instances > on > >>> different hosts > >>> - the same storage backend is used for all dovecot instances > >>> - no need of sharding domains > >>> - dovecot is easily load balanced (with director sticking users to the > >>> same dovecot backend) > >>> > >>> On the upcoming upgrade we intent to: > >>> - upgrade ceph to 12.X (Luminous) > >>> - drop the SSD Cache Tier (because it's deprecated) > >>> - use bluestore engine > >>> > >>> I was said on freenode/#dovecot that there are many cases where SDBOX > >> would > >>> perform better with NFS sharing. > >>> In case of cephfs, at first, I wouldn't think that would be true > because > >>> more files == more generated IO, but thinking about what I said in the > >>> beginning regarding sdbox vs mdbox that could be wrong. > >>> > >>> Any thoughts will be highlt appreciated. > >>> > >>> Regards, > >>> > >>> Webert Lima > >>> DevOps Engineer at MAV Tecnologia > >>> *Belo Horizonte - Brasil* > >>> *IRC NICK - WebertRLZ* > >>> > >>> > >>> > >>> _______________________________________________ > >>> ceph-users mailing list > >>> ceph-users(a)lists.ceph.com > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>> > >> > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users(a)lists.ceph.com > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >> > > > >

1 1