- dovecot - dovecot.org

Question: Directory hashing to support large domains | Dovecot & MySQL
by Altin Ukshini 26 Feb '16

26 Feb '16

Hello there, I have a postfix + dovecot + mysql installation in an ubuntu server. Virtual users and domains in a small mysql db that I manage via a web based mailadmin I wrote myself. The mysql db I'm talking about has a qmail structure that used to be managed by qmailadmin and worked with vpopmail. As for information, I cannot move to a different structure (exc postfixadmin/postfix) for many reasons, so I was forced to stick with it and configure postfix and dovecot to read that structure... If you may know, qmail (depends on the configuration) uses a directory hashing like /home/vpopmail/DOMAIN/P/7/USERNAME (something similar) in order to manage large domains, and this is handled by vpopmail. Please refer to these two links for details: - http://free.acrconsulting.co.uk/email/qmh.html - https://sourceforge.net/p/vpopmail/mailman/message/2710408/ I saw that dovecot uses something similar which I found in dovecot documentation : http://wiki2.dovecot.org/MailLocation#Directory_hashing Unfortunately, I cannot find more details, so I'd be thankful if you help me out with the following: By reading the documentation, I can tell that I can use the dir hashing variables in my dovecot configuration. But, in my case, I get the directory path from my mysql db via dovecot sql user_query - not from mail_location variable in 10-mail.conf Therefore if in case of a first received email for a user: dovecot checks in the db, gets the path and creates the maildir from the given path. In case the maildir column is empty, it fails, and it also fails to authenticate... So, can dovecot write to mysql? I would like to use directory hashing of dovecot, but from what I can find: If dovecot has mail_location variable set to do the dir hashing, it will not then go and add the created path in my mysql db. While if I try to implement something similar from my web gui application, I'd probably have to rewrite the algorithm that does this!? Can anyone help me figure out what's the best way to implement this? Thank you very much in advance for your help! Best, Altin Ukshini

1 0

[Feature Request] doveadm option to return number of messages acted upon
by Haravikk 26 Feb '16

26 Feb '16

So I have a script for handling my specific archive and expunge needs, but it’d be nice to be able to track how many messages are being affected. Currently I’m doing it by firing the same search queries into doveadm search and counting the lines of the result with wc -l, but that’s not a very pretty solution. While I’m mostly doing it out of interest on a personal server, I can’t imagine it’s a very scalable way to do it if you wanted to gather some kind of metrics for example. What I think would make sense would be for relevant doveadm commands such as move, expunge (probably purge too) and others I haven’t thought of to have a new option that, if enabled, will cause the command to output the number of messages affected as the final line of output (most of these commands don’t have any output anyway). A doveadm count command could also be a convenient, more efficient, alternative to doveadm search | wc -l.

2 1

Replication between two ISPCOnfig servers
by Otto Saayman 26 Feb '16

26 Feb '16

Hello I have set up two servers, one master and one slave, using ISPConfig. I have been using rsync to synchronise mails between the two machines, but this has some unstable results. I had a look at setting up replication: http://wiki.dovecot.org/Replication When I run the command, doveadm user '*', only two users are returned. So when I run this command: doveadm replicator replicate \* I get: 2 users updated I run this command: doveadm sync -u [e-mail address] tcp:[2nd/ slave dovecot server IP address] Here I can input any valid e-mail address in the ISPConfig MySQL database and it works. How do I tell the dovecote server to replicate all the users configured in the MySQL database it uses when it handles logins via POP/ IMAP, all of which is working fine? Is there a command I need to run at a set interval via cron to synchronise the two machines' mailboxes, which are identical in every way except the mails the users' mailboxes contain? OR Is there a way that I can instruct the two machines to keep each other's user mailbox contents "up to date"? The main idea is to have a hot backup of my "main" dovecot server and also to enable a lower mail priority for the second server so that mails are still received when the main server becomes busy, or is taken offline for some reason. Any help, or link to a HOWTO would be appreciated. -- Best Regards Otto Saayman

2 1

Checking index sanity
by Emmanuel Dreyfus 25 Feb '16

25 Feb '16

Hello We experienced corrupted dovecot indexes, probably caused by a server crash. This does not cause a lot of harm, just annoyance. For instance, LMTP raises an assertion during delivery, which causes mail to multiple recipients e-mail to remain in the queue if a single of them hits a corrupted index. The workaround for now is to detect the situation in the logs and to remove corrupted indexes when the problem arise. A better fix would be to sanity check all user's index on startup. Is there a command line tool to do this? -- Emmanuel Dreyfus manu(a)netbsd.org

1 0

VS: Re: SQLite driver and auth-worker credentials
by Aki Tuomi 24 Feb '16

24 Feb '16

this is probably something that ought to be fixed. Read only scenario for auth/userdb is quite plausible. ---Aki TuomiDovecot oy-------- Alkuperäinen viesti --------Lähettäjä: Lev Serebryakov <lev(a)serebryakov.spb.ru> Päivämäärä: 24.2.2016 23.32 (GMT+02:00) Saaja: james(a)lottspot.com Kopio: dovecot(a)dovecot.org Aihe: Re: SQLite driver and auth-worker credentials Hello James, Wednesday, February 24, 2016, 11:17:12 PM, you wrote: >> But system should assign all secondary GIDs to effective UID? > separate 'group =' directive in addition to the 'user =' directive > (http://wiki2.dovecot.org/Services) Oh, I missed this one, thank you -- Best regards, Lev mailto:lev@serebryakov.spb.ru

1 0

SQLite driver and auth-worker credentials
by Lev Serebryakov 24 Feb '16

24 Feb '16

I want to use SQLite database as storage for auth and user databases. I've encountered two problems here: (1) There is no way to open SQLite database read-only (via sqlite3_open_v2() call with SQLITE_OPEN_READONLY flag). It looks bad. I don't need (and want) to give dovecot rights to write to this database. (2) I've created system group "hostingdb", added "dovecot" user to it and gives 660 rights to database file, but still "auth-worker" could not open database and complains to log file. Now I'm set "user = root" for auth-worker, but I don't like it! Why auth-worker doesn't belong to "hostingdb" group? -- // Black Lion AKA Lev Serebryakov

2 4

Script dovecot ACLs, Quota and doveadm
by Chris 24 Feb '16

24 Feb '16

Dear All, I'd like to set a) Quota for mailboxes in private and public namespace b) ACLs for both. What's the easiest way to do this? Do I have to call doveadm for every mailbox (private and public)? Is it required to provide a username to doveadm? Can I use the same user for all if it is defined in global ACLs? For Cyrus there are perl modules which have the same functions as cyradm. Is there something like this for dovecot? Are there any modules available? Haven't found anything in CPAN. - Chris

1 0

Streaming MOVE commands
by gallego＠cri.ensmp.fr 24 Feb '16

24 Feb '16

Dear Dovecot devs, is streaming multiple MOVE commands by clients allowed? I am getting duplicated messages with the GNUS mail client, the interchange looks like this: *stream two moves to different folders* > 9019 UID MOVE 4062,4066,4068 "folder0" > 9020 UID MOVE 4063:4064,4067,4069:4072 "folder1" *the messages are copied* > * OK [COPYUID 1424475218 4062,4066,4068 376:378] Moved UIDs. > * OK [COPYUID 1424475231 4063:4064,4067,4069:4072 26:32] Moved UIDs. *however expunge fails to clean 4063, 4064, and 4067* > * VANISHED 4062,4066,4068:4072 thus 4063, 4064, and 4067 end both in inbox and folder1 producing duplicate messages (more details at [1]). At the GNUS mailing list, we were wondering about what should be the correct reading of RFC6851. Version and config information below. Best regards, Emilio [1] More details in the thread http://permalink.gmane.org/gmane.emacs.gnus.general/86813 [2] Version $ /usr/sbin/dovecot --version 2.2.13 [3] Config $ /usr/sbin/dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 8.3 ext4 mail_location = maildir:/home/%u/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap sieve" ssl_cert = ... ssl_key = ... userdb { driver = passwd }

3 6

Automatically inferring %d on multi-domain virtual install ?
by Gabriel L. Somlo 24 Feb '16

24 Feb '16

Hi, I'm trying to allow domain-less logins for a multi-domain virtual IMAP server, and wondering if I can automatically infer the domain (value of variable %d) from the local IP (%l) or the hostname used by the client when connecting to my server. Let's say I have two host names: mail.foo.org (10.0.0.100) and mail.bar.com (10.0.0.200), with forward and reverse DNS configured to resolve A and PTR records in either direction. Let's also say I have 10.0.0.100 and 10.0.0.200 set up as secondaries on my server's loopback interface, and routing is set up to bring client traffic to me for both of those IP addresses. The relevant bits of my dovecot.conf are: ---%<------------------------------------------------------------------ mail_location = maildir:/var/lib/vmail/%d/%n passdb { driver = passwd-file args = /var/lib/vmail/etc/postfix/userdb } userdb { driver = static args = uid=dovenull gid=dovenull home=/var/lib/vmail/%d/%n } ---%<------------------------------------------------------------------ And my userdb passwd-file right now includes: ---%<------------------------------------------------------------------ user1@foo.org:{PLAIN}user1foo user2@foo.org:{PLAIN}user2foo user1@bar.com:{PLAIN}user1bar user2@bar.com:{PLAIN}user2bar ---%<------------------------------------------------------------------ Right now, user1(a)foo.org must configure their imap client like so: IMAP server: mail.foo.org username: user1(a)foo.org password: user1foo I would like to require this (and other) users to only have to set: IMAP server: mail.foo.org username: user1 password: ... and have dovecot somehow infer the "@foo.org" domain based on the fact that the connection was made to 10.0.0.100, which is mail.foo.org, and therefore the domain can *only* be "@foo.org". I could start out by splitting my user database into two files: userdb.foo.org ---%<------------------------------------------------------------------ user1:{PLAIN}user1foo user2:{PLAIN}user2foo ---%<------------------------------------------------------------------ userdb.bar.com ---%<------------------------------------------------------------------ user1:{PLAIN}user1bar user2:{PLAIN}user2bar ---%<------------------------------------------------------------------ ... then modify dovecot.conf's passdb setup like so: ---%<------------------------------------------------------------------ passdb { driver = passwd-file args = /var/lib/vmail/etc/postfix/userdb.%d } ---%<------------------------------------------------------------------ ... but how would I insure that %d is set to the proper value based on e.g. a reverse lookup of %l, which, in foo.org's case would be 10.0.0.100, and resolve to mail.foo.org, and *somehow* that would match %d == "@foo.org" ? Is this even possible in the first place, or am I just being too fussy about the aesthetics of my users' imap client config files ? :) :) Thanks much, --Gabriel

5 9

Dovecot + Ceph Cluster Sizing
by Michele Soragni 24 Feb '16

24 Feb '16

Hi all. We are searching for information about how to size dovecot servers in a clustered architecture (dovecot + postfix + roundcube + mysql). We need to migrate mailboxes from an old Exchange installation: 60k mailbox 900 GB total storage 14 MB/mailbox About 133 email/mailbox Low usage: about 2k simultaneous imap/pop3 connections This is how we are thinking to build our new architecture 3 x Dell R630 or R530 servers with 6x300SAS disks for Ceph storage with 1 replica . Total storage = 2,5 TB for mailboxes 3 x Dell R630 or R530 with 2x CPU 6core,64GB RAM, 4x 300SAS disks hosting the following VMs in a VMware cluster: - 3 x VM Dovecot - 3 x VM Postfix - 3 x VM Roundcube webmail - 3 x VM MySQL All VMs are behind a balancer All OS disks are on local storage Dovecot VMs share Ceph storage with OCFS2 filesystem for the mailboxes. I'd like to have some hints about CPU and memory for Dovecot VMs. 2 vCPU and 8 GB RAM each sould be enough? Is there a way to calculate this? Do you think the storage performance could be a problem using Ceph + OCFS2? Is Ceph + XFS + Dsync replication a better choice? Thanks! Michele

1 0