October 2018 - dovecot

iterate query and dsync replication 2.2.33.2
by Matt Bryant 08 Oct '18

08 Oct '18

All, According to doco replication gets its list of users to replicate from the iterate query but this doesn't seem to be the case or its just being totally ignored .. Have ldap passbd and sql userdb, with the iterate query in the userdb sql file. -The query itself returns the handful of users I want to replicate BUT if I run the command dovecot replicator status '*' it shows pretty much all users in the system .. is there a bug here or is documentation just completely wrong when it comes to what is/is not replicated. rgds Matt

1 0

VOLATILEDIR not really used?
by Tom Sommer 08 Oct '18

08 Oct '18

Hi According to the docs, setting VOLATILEDIR will improve I/O performance when using NFS - but as far as I can see, only vsize lock-files are put here, and little else? Is this correct, and if so are there any plans to move dotlocks etc. to this directory? Thanks. -- Tom

2 7

Dovecot -v 2.2.27 (c0f36b0): Outgoing mail server fails
by Tommy Lane 07 Oct '18

07 Oct '18

Hello Everyone, I'm having an issue with a recent install of postfix and Dovecot. I can authenticate without any issues for "incoming" mail server, but when I attempt to send an email from the account my client gives an auth error. the server shows that I was authed using plain-auth, which I would also like to change as these machines are accross the open internet. Any help would be greatly appreciated. -Tommy

3 3

TLS handshake failure - Client Helo rejected
by VB 07 Oct '18

07 Oct '18

Hi, I can no longer connect to Dovecot (IMAP). The connection is terminated by Dovecot after Client Helo. My server: Dovecot 2.3.3 Debian buster/sid Architecture: ppc My problems started in late August after upgrading Dovecot. SSL settings: ssl_dh = </etc/ssl/dh2048.pem ssl_min_protocol = TLSv1.2 ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 ssl_prefer_server_ciphers = yes Client: OS Android 6.0.1 Aquamail Log from Dovecot: Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL: where=0x10, ret=1: before SSL initialization Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before SSL initialization Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: before SSL initialization Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before SSL initialization Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL alert: where=0x4008, ret=598: fatal unknown Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL error: SSL_accept() failed: error:14209175:SSL routines:tls_early_post_process_client_hello:inappropriate fallback Sep 15 23:19:02 debian2 dovecot: imap-login: Debug: SSL error: SSL_accept() syscall failed: Invalid argument Sep 15 23:19:02 debian2 dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=XXX.XXX.XXX.XXX, lip=XXX.XXX.XXX.XXX,TLS handshaking: SSL_accept() syscall failed: Invalid argument, session=<XXXXXXXXXXX> Log from client (Aquamail) is a bit longer (see attachment). I have also listened to the communication using Wireshark. The last piece of communication is Client Helo. After the client sends Client Helo, there is no reply from Dovecot and the server closes the communication. This is the Client Helo, in the "structured view" iin Wireshark: |Secure Sockets Layer TLSv1 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 176 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 172 Version: TLS 1.2 (0x0303) Random: 2b7af5ba92040f081a5a3621e9d9cbab2d50b829b7fe755f... Session ID Length: 0 Cipher Suites Length: 62 Cipher Suites (31 suites) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f) Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e) Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d) Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a) Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d) Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023) Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009) Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c) Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b) Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039) Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067) Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033) Cipher Suite: TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011) Cipher Suite: TLS_ECDHE_ECDSA_WITH_RC4_128_SHA (0xc007) Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005) Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004) Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a) Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff) Cipher Suite: TLS_FALLBACK_SCSV (0x5600) Compression Methods Length: 1 Compression Methods (1 method) Extensions Length: 69 Extension: server_name (len=17) Extension: extended_master_secret (len=0) Extension: signature_algorithms (len=22) Type: signature_algorithms (13) Length: 22 Signature Hash Algorithms Length: 20 Signature Hash Algorithms (10 algorithms) Signature Algorithm: rsa_pkcs1_sha512 (0x0601) Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603) Signature Algorithm: rsa_pkcs1_sha384 (0x0501) Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503) Signature Algorithm: rsa_pkcs1_sha256 (0x0401) Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403) Signature Algorithm: SHA224 RSA (0x0301) Signature Algorithm: SHA224 ECDSA (0x0303) Signature Algorithm: rsa_pkcs1_sha1 (0x0201) Signature Algorithm: ecdsa_sha1 (0x0203) | |What I tried: | * |change all possible settings in Dovecot (ssl_min_protocol, ssl_cipher_list ...)| * |change certificate I use| |I also got in touch with the developer of Aquamail (see our discussion here: https://www.aqua-mail.com/forum/index.php?topic=6824.0 ).| |The developer was able to reproduce the handshake error. We believe that the problem is that Dovecot rejects ClientHello as long as it is wrapped in the TLSv1 Record Layer (see the second lilne in the Wireshark log). According to the developer, Dovecot should accept Client Helo wrapped in the TLSv1 Record Layer.| |Thank you very much for your help. Best regards VB |

2 1

doveadm quota error output in some cases
by Kai Schaetzl 05 Oct '18

05 Oct '18

I am testing a special setup on one machine where I want to put some local users that get mail locally to /home user directories via postfix transport map local: for historical and practical reasons. Other users are setup as virtual users. The local users are *also* setup as virtual users for dovecot so that I can retrieve the mail via dovecot. I changed the SQL for dovecot user_query, so that it gets the local uid and gid (and doesn't use vmail) and also gets the home directory in /home for only those users. After a bit of experimenting this works fine and I can retrieve mail via dovecot. However, there are two problems with quota and one of them has "doveadm get quota" spew out a lot of garbage that I think should actually go to a log and not to the program output. When I run a doveadm quota get -A or -u user I get four errors of this kind (for user/domain storage and message limit) for each special user: doveadm(user): Error: Failed to get quota resource STORAGE: quota-dict: dict_lookup(priv/quota/storage) failed: net_connect_unix (/var/run/dovecot/dict) failed: Permission denied (euid=500(user) egid=200(vmail) missing +r perm: /var/run/dovecot/dict, dir owned by 0:0 mode=0755) (reply took 0.000 secs (0.000 in dict wait, 0.000 in other ioloops, 0.000 in locks)) Then the normal program output follows, with error lines for these special users, of course: user STORAGE error error error The reason is that /var/run/dovecot/dict is owned by vmail:vmail (not root:root as the 0:0 suggests). I don't see a way to change this without blocking the "real" virtual users that use vmail:vmail. Shouldn't this massive error output rather go to a log? (dovecot/error.log) Or: is there a way to avoid this? /var/run/dovecot/dict is a socket. I didn't want to fiddle with it. Can I change the permissions so that it can also be accessed by other system users? Probably only to make it world-readable? If so, what are the implications of doing that? Thanks, Kai

1 1

Username aliases
by Eirik Rye 05 Oct '18

05 Oct '18

Hi, Our organization is currently using a passwd-file user database for our Dovecot installs. However, we now want to provide simple username aliases. We currently use usernames that are -not- equal to the email address, but due to the fact that many major email clients/providers do not even allow you to specifiy a username, we now want to provide aliases where the username is equal to the email address. A simple solution would be to duplicate the lines in the passwd-file, with the email address as the username, but this does not seem be ideal due to data duplication. Ideally, there would be a secondary 'alias' userdb, which on a match would then restart the search in the primary passwd-file userdb using the new username. Is this possible? What would be the best way to go about this?

4 6

vpopmail
by Eric Broch 04 Oct '18

04 Oct '18

Hello list, I run Dovecot with the vpopmail driver and have found that it authenticates against the clear text password in the vpopmail database. Is there a configuration option either at compile time, link time, or a setting in one of the configuration files that tells the program to authenticate against the hash instead of the clear text? -- Eric Broch White Horse Technical Consulting (WHTC)

3 9

Re: [ceph-users] dovecot + cephfs - sdbox vs mdbox
by Webert de Souza Lima 04 Oct '18

04 Oct '18

Thanks Jack. That's good to know. It is definitely something to consider. In a distributed storage scenario we might build a dedicated pool for that and tune the pool as more capacity or performance is needed. Regards, Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil* *IRC NICK - WebertRLZ* On Wed, May 16, 2018 at 4:45 PM Jack <ceph(a)jack.fr.eu.org> wrote: > On 05/16/2018 09:35 PM, Webert de Souza Lima wrote: > > We'll soon do benchmarks of sdbox vs mdbox over cephfs with bluestore > > backend. > > We'll have to do some some work on how to simulate user traffic, for > writes > > and readings. That seems troublesome. > I would appreciate seeing these results ! > > > Thanks for the plugins recommendations. I'll take the change and ask you > > how is the SIS status? We have used it in the past and we've had some > > problems with it. > > I am using it since Dec 2016 with mdbox, with no issue at all (I am > currently using Dovecot 2.2.27-3 from Debian Stretch) > The only config I use is mail_attachment_dir, the rest lies as default > (mail_attachment_min_size = 128k, mail_attachment_fs = sis posix, > ail_attachment_hash = %{sha1}) > The backend storage is a local filesystem, and there is only one Dovecot > instance > > > > > Regards, > > > > Webert Lima > > DevOps Engineer at MAV Tecnologia > > *Belo Horizonte - Brasil* > > *IRC NICK - WebertRLZ* > > > > > > On Wed, May 16, 2018 at 4:19 PM Jack <ceph(a)jack.fr.eu.org> wrote: > > > >> Hi, > >> > >> Many (most ?) filesystems does not store multiple files on the same > block > >> > >> Thus, with sdbox, every single mail (you know, that kind of mail with 10 > >> lines in it) will eat an inode, and a block (4k here) > >> mdbox is more compact on this way > >> > >> Another difference: sdbox removes the message, mdbox does not : a single > >> metadata update is performed, which may be packed with others if many > >> files are deleted at once > >> > >> That said, I do not have experience with dovecot + cephfs, nor have made > >> tests for sdbox vs mdbox > >> > >> However, and this is a bit out of topic, I recommend you look at the > >> following dovecot's features (if not already done), as they are awesome > >> and will help you a lot: > >> - Compression (classic, https://wiki.dovecot.org/Plugins/Zlib) > >> - Single-Instance-Storage (aka sis, aka "attachment deduplication" : > >> https://www.dovecot.org/list/dovecot/2013-December/094276.html) > >> > >> Regards, > >> On 05/16/2018 08:37 PM, Webert de Souza Lima wrote: > >>> I'm sending this message to both dovecot and ceph-users ML so please > >> don't > >>> mind if something seems too obvious for you. > >>> > >>> Hi, > >>> > >>> I have a question for both dovecot and ceph lists and below I'll > explain > >>> what's going on. > >>> > >>> Regarding dbox format (https://wiki2.dovecot.org/MailboxFormat/dbox) > >> when > >>> using sdbox, a new file is stored for each email message. > >>> When using mdbox, multiple messages are appended to a single file until > >> it > >>> reaches/passes the rotate limit. > >>> > >>> I would like to understand better how the mdbox format impacts on IO > >>> performance. > >>> I think it's generally expected that fewer larger file translate to > less > >> IO > >>> and more troughput when compared to more small files, but how does > >> dovecot > >>> handle that with mdbox? > >>> If dovecot does flush data to storage upon each and every new email is > >>> arrived and appended to the corresponding file, would that mean that it > >>> generate the same ammount of IO as it would do with one file per > message? > >>> Also, if using mdbox many messages will be appended to a said file > >> before a > >>> new file is created. That should mean that a file descriptor is kept > open > >>> for sometime by dovecot process. > >>> Using cephfs as backend, how would this impact cluster performance > >>> regarding MDS caps and inodes cached when files from thousands of users > >> are > >>> opened and appended all over? > >>> > >>> I would like to understand this better. > >>> > >>> Why? > >>> We are a small Business Email Hosting provider with bare metal, self > >> hosted > >>> systems, using dovecot for servicing mailboxes and cephfs for email > >> storage. > >>> > >>> We are currently working on dovecot and storage redesign to be in > >>> production ASAP. The main objective is to serve more users with better > >>> performance, high availability and scalability. > >>> * high availability and load balancing is extremely important to us * > >>> > >>> On our current model, we're using mdbox format with dovecot, having > >>> dovecot's INDEXes stored in a replicated pool of SSDs, and messages > >> stored > >>> in a replicated pool of HDDs (under a Cache Tier with a pool of SSDs). > >>> All using cephfs / filestore backend. > >>> > >>> Currently there are 3 clusters running dovecot 2.2.34 and ceph Jewel > >>> (10.2.9-4). > >>> - ~25K users from a few thousands of domains per cluster > >>> - ~25TB of email data per cluster > >>> - ~70GB of dovecot INDEX [meta]data per cluster > >>> - ~100MB of cephfs metadata per cluster > >>> > >>> Our goal is to build a single ceph cluster for storage that could > expand > >> in > >>> capacity, be highly available and perform well enough. I know, that's > >> what > >>> everyone wants. > >>> > >>> Cephfs is an important choise because: > >>> - there can be multiple mountpoints, thus multiple dovecot instances > on > >>> different hosts > >>> - the same storage backend is used for all dovecot instances > >>> - no need of sharding domains > >>> - dovecot is easily load balanced (with director sticking users to the > >>> same dovecot backend) > >>> > >>> On the upcoming upgrade we intent to: > >>> - upgrade ceph to 12.X (Luminous) > >>> - drop the SSD Cache Tier (because it's deprecated) > >>> - use bluestore engine > >>> > >>> I was said on freenode/#dovecot that there are many cases where SDBOX > >> would > >>> perform better with NFS sharing. > >>> In case of cephfs, at first, I wouldn't think that would be true > because > >>> more files == more generated IO, but thinking about what I said in the > >>> beginning regarding sdbox vs mdbox that could be wrong. > >>> > >>> Any thoughts will be highlt appreciated. > >>> > >>> Regards, > >>> > >>> Webert Lima > >>> DevOps Engineer at MAV Tecnologia > >>> *Belo Horizonte - Brasil* > >>> *IRC NICK - WebertRLZ* > >>> > >>> > >>> > >>> _______________________________________________ > >>> ceph-users mailing list > >>> ceph-users(a)lists.ceph.com > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>> > >> > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users(a)lists.ceph.com > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >> > > > >

1 1

Re: Bug reports - auth is broken in Dovecot 2.3.3
by Berindeie Avram-Teodor 04 Oct '18

04 Oct '18

> > I do not have downloaded the source from GitHub. > > On Wed, Oct 3, 2018 at 1:53 PM Timo Sirainen <tss(a)iki.fi> wrote: > >> On 3 Oct 2018, at 13.22, Berindeie Avram-Teodor <berindeie.teo(a)gmail.com> >> wrote: >> > >> > I applied the patch and recompiled but nothing resolved. >> > What else can I do? >> >> You also need to run autogen.sh & configure again. >> >>

4 5

Re: Dovecot User Listing Error - getpwent() failed: Invalid Argument
by Reuben Farrelly 04 Oct '18

04 Oct '18

Sure: https://www.reub.net/files/dovecot/lightning-dovecot.conf https://www.reub.net/files/dovecot/thunderstorm-dovecot.conf Updated nightly. Reuben On 1/09/2018 12:26 am, Aki Tuomi wrote: > Can you provide doveconf -n? > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Reuben Farrelly <reuben-dovecot(a)reub.net> > Date: 31/08/2018 17:12 (GMT+02:00) > To: Aki Tuomi <aki.tuomi(a)dovecot.fi>, Dovecot Mailing List > <dovecot(a)dovecot.org> > Subject: Re: Dovecot User Listing Error - getpwent() failed: Invalid > Argument > > No. Neither of those are installed on either system. > > Reuben > > On 1/09/2018 12:09 am, Aki Tuomi wrote: > > Could apparmor or selinux be causing this! > > > > > > > > --- > > Aki Tuomi > > Dovecot oy > > > > -------- Original message -------- > > From: Reuben Farrelly <reuben-dovecot(a)reub.net> > > Date: 31/08/2018 16:50 (GMT+02:00) > > To: Dovecot Mailing List <dovecot(a)dovecot.org> > > Subject: Dovecot User Listing Error - getpwent() failed: Invalid Argument > > > > Hi, > > > > I'm running dovecot-2.3 git and seeing on 2 of my Dovecot installations, > > the following message logged quite frequently: > > > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: auth-worker(7707): > > Error: getpwent() failed: Invalid argument > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: > > User listing returned failure > > Aug 31 16:55:53 lightning.reub.net dovecot[7698]: replicator: Error: > > listing users failed, can't replicate existing data > > > > It appears that this could be having an effect on replication, but it's > > unclear because the secondary/replica doesn't see much traffic. > > > > Authentication is via PAM and the system is Gentoo Linux x86_64. passb > > driver = pam, userdb driver = static . > > > > doveadm user -u '*' shows the system users listed twice like this (is > > this a problem?) and then prints: > > > > ---- > > reuben > > liam > > reuben > > liam > > <some other users> > > Error: User listing returned failure > > Fatal: user listing failed > > ---- > > > > To me that doesn't look right. > > > > Can anyone suggest what could be causing this? I don't recall seeing > > this message some time ago so I suspect it's a recent change in either > > dovecot, or pam or glibc etc.. > > > > Thanks, > > Reuben

2 10