February 2016 - dovecot

[Feature Request] doveadm option to return number of messages acted upon
by Haravikk 26 Feb '16

26 Feb '16

So I have a script for handling my specific archive and expunge needs, but it’d be nice to be able to track how many messages are being affected. Currently I’m doing it by firing the same search queries into doveadm search and counting the lines of the result with wc -l, but that’s not a very pretty solution. While I’m mostly doing it out of interest on a personal server, I can’t imagine it’s a very scalable way to do it if you wanted to gather some kind of metrics for example. What I think would make sense would be for relevant doveadm commands such as move, expunge (probably purge too) and others I haven’t thought of to have a new option that, if enabled, will cause the command to output the number of messages affected as the final line of output (most of these commands don’t have any output anyway). A doveadm count command could also be a convenient, more efficient, alternative to doveadm search | wc -l.

2 1

Replication between two ISPCOnfig servers
by Otto Saayman 26 Feb '16

26 Feb '16

Hello I have set up two servers, one master and one slave, using ISPConfig. I have been using rsync to synchronise mails between the two machines, but this has some unstable results. I had a look at setting up replication: http://wiki.dovecot.org/Replication When I run the command, doveadm user '*', only two users are returned. So when I run this command: doveadm replicator replicate \* I get: 2 users updated I run this command: doveadm sync -u [e-mail address] tcp:[2nd/ slave dovecot server IP address] Here I can input any valid e-mail address in the ISPConfig MySQL database and it works. How do I tell the dovecote server to replicate all the users configured in the MySQL database it uses when it handles logins via POP/ IMAP, all of which is working fine? Is there a command I need to run at a set interval via cron to synchronise the two machines' mailboxes, which are identical in every way except the mails the users' mailboxes contain? OR Is there a way that I can instruct the two machines to keep each other's user mailbox contents "up to date"? The main idea is to have a hot backup of my "main" dovecot server and also to enable a lower mail priority for the second server so that mails are still received when the main server becomes busy, or is taken offline for some reason. Any help, or link to a HOWTO would be appreciated. -- Best Regards Otto Saayman

2 1

Checking index sanity
by Emmanuel Dreyfus 25 Feb '16

25 Feb '16

Hello We experienced corrupted dovecot indexes, probably caused by a server crash. This does not cause a lot of harm, just annoyance. For instance, LMTP raises an assertion during delivery, which causes mail to multiple recipients e-mail to remain in the queue if a single of them hits a corrupted index. The workaround for now is to detect the situation in the logs and to remove corrupted indexes when the problem arise. A better fix would be to sanity check all user's index on startup. Is there a command line tool to do this? -- Emmanuel Dreyfus manu(a)netbsd.org

1 0

VS: Re: SQLite driver and auth-worker credentials
by Aki Tuomi 24 Feb '16

24 Feb '16

this is probably something that ought to be fixed. Read only scenario for auth/userdb is quite plausible. ---Aki TuomiDovecot oy-------- Alkuperäinen viesti --------Lähettäjä: Lev Serebryakov <lev(a)serebryakov.spb.ru> Päivämäärä: 24.2.2016 23.32 (GMT+02:00) Saaja: james(a)lottspot.com Kopio: dovecot(a)dovecot.org Aihe: Re: SQLite driver and auth-worker credentials Hello James, Wednesday, February 24, 2016, 11:17:12 PM, you wrote: >> But system should assign all secondary GIDs to effective UID? > separate 'group =' directive in addition to the 'user =' directive > (http://wiki2.dovecot.org/Services) Oh, I missed this one, thank you -- Best regards, Lev mailto:lev@serebryakov.spb.ru

1 0

SQLite driver and auth-worker credentials
by Lev Serebryakov 24 Feb '16

24 Feb '16

I want to use SQLite database as storage for auth and user databases. I've encountered two problems here: (1) There is no way to open SQLite database read-only (via sqlite3_open_v2() call with SQLITE_OPEN_READONLY flag). It looks bad. I don't need (and want) to give dovecot rights to write to this database. (2) I've created system group "hostingdb", added "dovecot" user to it and gives 660 rights to database file, but still "auth-worker" could not open database and complains to log file. Now I'm set "user = root" for auth-worker, but I don't like it! Why auth-worker doesn't belong to "hostingdb" group? -- // Black Lion AKA Lev Serebryakov

2 4

Script dovecot ACLs, Quota and doveadm
by Chris 24 Feb '16

24 Feb '16

Dear All, I'd like to set a) Quota for mailboxes in private and public namespace b) ACLs for both. What's the easiest way to do this? Do I have to call doveadm for every mailbox (private and public)? Is it required to provide a username to doveadm? Can I use the same user for all if it is defined in global ACLs? For Cyrus there are perl modules which have the same functions as cyradm. Is there something like this for dovecot? Are there any modules available? Haven't found anything in CPAN. - Chris

1 0

Streaming MOVE commands
by gallego＠cri.ensmp.fr 24 Feb '16

24 Feb '16

Dear Dovecot devs, is streaming multiple MOVE commands by clients allowed? I am getting duplicated messages with the GNUS mail client, the interchange looks like this: *stream two moves to different folders* > 9019 UID MOVE 4062,4066,4068 "folder0" > 9020 UID MOVE 4063:4064,4067,4069:4072 "folder1" *the messages are copied* > * OK [COPYUID 1424475218 4062,4066,4068 376:378] Moved UIDs. > * OK [COPYUID 1424475231 4063:4064,4067,4069:4072 26:32] Moved UIDs. *however expunge fails to clean 4063, 4064, and 4067* > * VANISHED 4062,4066,4068:4072 thus 4063, 4064, and 4067 end both in inbox and folder1 producing duplicate messages (more details at [1]). At the GNUS mailing list, we were wondering about what should be the correct reading of RFC6851. Version and config information below. Best regards, Emilio [1] More details in the thread http://permalink.gmane.org/gmane.emacs.gnus.general/86813 [2] Version $ /usr/sbin/dovecot --version 2.2.13 [3] Config $ /usr/sbin/dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 8.3 ext4 mail_location = maildir:/home/%u/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap sieve" ssl_cert = ... ssl_key = ... userdb { driver = passwd }

3 6

Automatically inferring %d on multi-domain virtual install ?
by Gabriel L. Somlo 24 Feb '16

24 Feb '16

Hi, I'm trying to allow domain-less logins for a multi-domain virtual IMAP server, and wondering if I can automatically infer the domain (value of variable %d) from the local IP (%l) or the hostname used by the client when connecting to my server. Let's say I have two host names: mail.foo.org (10.0.0.100) and mail.bar.com (10.0.0.200), with forward and reverse DNS configured to resolve A and PTR records in either direction. Let's also say I have 10.0.0.100 and 10.0.0.200 set up as secondaries on my server's loopback interface, and routing is set up to bring client traffic to me for both of those IP addresses. The relevant bits of my dovecot.conf are: ---%<------------------------------------------------------------------ mail_location = maildir:/var/lib/vmail/%d/%n passdb { driver = passwd-file args = /var/lib/vmail/etc/postfix/userdb } userdb { driver = static args = uid=dovenull gid=dovenull home=/var/lib/vmail/%d/%n } ---%<------------------------------------------------------------------ And my userdb passwd-file right now includes: ---%<------------------------------------------------------------------ user1@foo.org:{PLAIN}user1foo user2@foo.org:{PLAIN}user2foo user1@bar.com:{PLAIN}user1bar user2@bar.com:{PLAIN}user2bar ---%<------------------------------------------------------------------ Right now, user1(a)foo.org must configure their imap client like so: IMAP server: mail.foo.org username: user1(a)foo.org password: user1foo I would like to require this (and other) users to only have to set: IMAP server: mail.foo.org username: user1 password: ... and have dovecot somehow infer the "@foo.org" domain based on the fact that the connection was made to 10.0.0.100, which is mail.foo.org, and therefore the domain can *only* be "@foo.org". I could start out by splitting my user database into two files: userdb.foo.org ---%<------------------------------------------------------------------ user1:{PLAIN}user1foo user2:{PLAIN}user2foo ---%<------------------------------------------------------------------ userdb.bar.com ---%<------------------------------------------------------------------ user1:{PLAIN}user1bar user2:{PLAIN}user2bar ---%<------------------------------------------------------------------ ... then modify dovecot.conf's passdb setup like so: ---%<------------------------------------------------------------------ passdb { driver = passwd-file args = /var/lib/vmail/etc/postfix/userdb.%d } ---%<------------------------------------------------------------------ ... but how would I insure that %d is set to the proper value based on e.g. a reverse lookup of %l, which, in foo.org's case would be 10.0.0.100, and resolve to mail.foo.org, and *somehow* that would match %d == "@foo.org" ? Is this even possible in the first place, or am I just being too fussy about the aesthetics of my users' imap client config files ? :) :) Thanks much, --Gabriel

5 9

Dovecot + Ceph Cluster Sizing
by Michele Soragni 24 Feb '16

24 Feb '16

Hi all. We are searching for information about how to size dovecot servers in a clustered architecture (dovecot + postfix + roundcube + mysql). We need to migrate mailboxes from an old Exchange installation: 60k mailbox 900 GB total storage 14 MB/mailbox About 133 email/mailbox Low usage: about 2k simultaneous imap/pop3 connections This is how we are thinking to build our new architecture 3 x Dell R630 or R530 servers with 6x300SAS disks for Ceph storage with 1 replica . Total storage = 2,5 TB for mailboxes 3 x Dell R630 or R530 with 2x CPU 6core,64GB RAM, 4x 300SAS disks hosting the following VMs in a VMware cluster: - 3 x VM Dovecot - 3 x VM Postfix - 3 x VM Roundcube webmail - 3 x VM MySQL All VMs are behind a balancer All OS disks are on local storage Dovecot VMs share Ceph storage with OCFS2 filesystem for the mailboxes. I'd like to have some hints about CPU and memory for Dovecot VMs. 2 vCPU and 8 GB RAM each sould be enough? Is there a way to calculate this? Do you think the storage performance could be a problem using Ceph + OCFS2? Is Ceph + XFS + Dsync replication a better choice? Thanks! Michele

1 0

Segfault using doveadm dsync
by fabio.onorini＠secoges.com 24 Feb '16

24 Feb '16

I'm trying to migrate a imap mailbox from dovecot 1.2.15 to a new imap server with dovecot 2.2.13. I would preserving the IMAP UIDs and POP3 UIDLs. So I'm using doveadm backup command but after folders analyze, command return segmentation fault. Any hint? GDB result: sync(easymail): Debug: brain S: Deleting mailbox 'Bozze' (GUID f0836f3222c335cfa383cb38ff0e3183): UIDNEXT is too high (9 > 1) dsync(easymail): Debug: brain S: Deleting mailbox 'Cestino' (GUID cd0edec22e40323d878a39dc8e432e5a): UIDNEXT is too high (628 > 1) dsync(easymail): Debug: brain S: Deleting mailbox 'INBOX' (GUID c92f64f79f0d1ed01e6d5b314f04886c): UIDVALIDITY changed (1456239585 -> 1456310264) dsync(easymail): Debug: brain S: Deleting mailbox 'Modelli' (GUID f0933f8b90e6eaf3a00a0220b173f3b4): UIDNEXT is too high (2 > 1) dsync(easymail): Debug: brain S: Deleting mailbox 'Sent' (GUID 7d3c7eaa71cdf47ee8a1192687cda8cd): UIDNEXT is too high (39 > 1) Program received signal SIGSEGV, Segmentation fault. imapc_client_mailbox_cmd (box=0x0, callback=callback@entry=0x7ffff76d7140 <imapc_simple_callback>, context=context@entry=0x7fffffffe2d0) at imapc-client.c:351 351 imapc-client.c: No such file or directory. (gdb) bt full #0 imapc_client_mailbox_cmd (box=0x0, callback=callback@entry=0x7ffff76d7140 <imapc_simple_callback>, context=context@entry=0x7fffffffe2d0) at imapc-client.c:351 cmd = <optimized out> __FUNCTION__ = "imapc_client_mailbox_cmd" #1 0x00007ffff76d7915 in imapc_mailbox_noop (mbox=mbox@entry=0x5555558049e0) at imapc-storage.c:154 cmd = <optimized out> sctx = {client = 0x5555557e89b0, ret = -2} #2 0x00007ffff76d5c18 in imapc_mailbox_sync_init (box=0x5555558049e0, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at imapc-sync.c:476 mbox = 0x5555558049e0 list = <optimized out> capabilities = <optimized out> changes = false ret = 0 #3 0x00007ffff76e9289 in mailbox_sync_init (box=0x5555558049e0, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at mail-storage.c:1677 _data_stack_cur_id = 3 ctx = <optimized out> #4 0x00007ffff76e93d7 in mailbox_sync (box=box@entry=0x5555558049e0, flags=<optimized out>, flags@entry=MAILBOX_SYNC_FLAG_FULL_READ) at mail-storage.c:1725 ctx = 0x555555804ea8 status = {sync_delayed_expunges = 0} #5 0x00007ffff7715bab in mailbox_expunge_all_data (box=0x5555558049e0) at index-storage.c:648 ctx = 0x7ffff7714e57 <index_storage_mailbox_open+615> t = 0x5555558049e0 mail = 0x7fffffffe4b4 search_args = 0x7fffffffe558 #6 index_storage_mailbox_delete (box=0x5555558049e0) at index-storage.c:701 metadata = {guid = "\264\344\377\377\377\177\000\000\000\000\000\000\000\000\000", virtual_size = 140737488348504, cache_fields = 0x555555807258, precache_fields = (MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_VIRTUAL_SIZE | MAIL_FETCH_IMAP_ENVELOPE | MAIL_FETCH_REFCOUNT | unknown: 1426065408), backend_ns_prefix = 0x7ffff73b1e59 <t_pop_check+9> ";\003u\v\307\003", backend_ns_type = (unknown: 1434470880)} status = {messages = 4294960472, recent = 32767, unseen = 0, uidvalidity = 0, uidnext = 1434582544, first_unseen_seq = 0, first_recent_uid = 232594432, last_cached_seq = 3612951791, highest_modseq = 93824995052000, highest_pvt_modseq = 93824992578308, keywords = 0x1, permanent_flags = 4151223616, permanent_keywords = 1, allow_new_keywords = 1, nonpermanent_modseqs = 1, no_modseq_tracking = 1, have_guids = 1, have_save_guids = 1, have_only_guid128 = 1} ret_guid = <optimized out> #7 0x00007ffff76ea737 in mailbox_delete (box=box@entry=0x5555558049e0) at mail-storage.c:1319 ret = <optimized out> #8 0x00005555555895d4 in dsync_brain_mailbox_tree_sync_change (brain=brain@entry=0x5555557fcd00, change=<optimized out>) at dsync-brain-mailbox-tree-sync.c:182 box = 0x5555558049e0 destbox = 0x5555558007c8 errstr = 0x5555557ff750 "\030\366\177UUU" func_name = 0x0 ---Type <return> to continue, or q <return> to quit--- storage_name = <optimized out> error = 32767 ret = <optimized out> __FUNCTION__ = "dsync_brain_mailbox_tree_sync_change" #9 0x00005555555892db in dsync_brain_mailbox_trees_sync (brain=0x5555557fcd00) at dsync-brain-mailbox-tree.c:291 ctx = 0x55555580ab40 change = <optimized out> sync_type = <optimized out> sync_flags = <optimized out> #10 dsync_brain_recv_mailbox_tree_deletes (brain=brain@entry=0x5555557fcd00) at dsync-brain-mailbox-tree.c:440 deletes = 0x5555557fa0a0 i = <optimized out> count = 0 sep = 47 '/' __FUNCTION__ = "dsync_brain_recv_mailbox_tree_deletes" #11 0x00005555555860e8 in dsync_brain_run_real (changed_r=0x7fffffffe74b, brain=0x5555557fcd00) at dsync-brain.c:565 ret = true orig_state = DSYNC_STATE_RECV_MAILBOX_TREE_DELETES orig_box_recv_state = DSYNC_BOX_STATE_MAILBOX orig_box_send_state = DSYNC_BOX_STATE_MAILBOX changed = false #12 dsync_brain_run (brain=0x5555557fcd00, changed_r=changed_r@entry=0x7fffffffe74b) at dsync-brain.c:610 _data_stack_cur_id = 2 changed_r = 0x7fffffffe74b brain = 0x5555557fcd00 #13 0x00005555555846f7 in cmd_dsync_run_local (changes_during_sync_r=<synthetic pointer>, ibc2=<optimized out>, brain=0x5555557e1860, user=<optimized out>, ctx=0x5555557cd590) at doveadm-dsync.c:384 brain2 = 0x5555557fcd00 set_parser = <optimized out> location = <optimized out> user2 = 0x0 brain1_running = true changed1 = true ret = <optimized out> set_line = <optimized out> brain2_running = true changed2 = false #14 cmd_dsync_run (_ctx=0x5555557cd590, user=<optimized out>) at doveadm-dsync.c:596 ---Type <return> to continue, or q <return> to quit--- ctx = 0x5555557cd590 ibc = 0x5555557d50d0 ibc2 = 0x5555557e14d0 brain = 0x5555557e1860 set = {process_title_prefix = 0x0, sync_namespaces = {arr = {buffer = 0x5555557c0378, element_size = 8}, v = 0x5555557c0378, v_modifiable = 0x5555557c0378}, sync_box = 0x0, sync_box_guid = '\000' <repeats 15 times>, exclude_mailboxes = 0x0, lock_timeout_secs = 0, state = 0x0} ns = 0x5555557cd670 strp = <optimized out> brain_flags = <optimized out> remote_errors_logged = false changes_during_sync = false status = 0 ret = 0 #15 0x0000555555569e4b in doveadm_mail_next_user (ctx=0x5555557cd590, input=<optimized out>, error_r=0x7fffffffe898) at doveadm-mail.c:314 error = 0x5555557cd590 "p\325|UUU" ret = <optimized out> error_r = 0x7fffffffe898 input = <optimized out> ctx = 0x5555557cd590 ip = <optimized out> ret = 0 #16 0x000055555556ac2c in doveadm_mail_cmd (argv=0x5555557c73b8, argc=4, cmd=0x5555557cb140) at doveadm-mail.c:525 input = {module = 0x0, service = 0x5555555a7367 "doveadm", username = 0x5555557c73e7 "easymail", session_id = 0x0, local_ip = {family = 0, u = {ip6 = { __in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0} ctx = 0x5555557cd590 getopt_args = 0x5555557bf660 "+1dEfg:l:m:n:NPr:Rs:Ux:RAS:u:" ret = <optimized out> c = <optimized out> wildcard_user = 0x0 error = 0x7ffff73b7122 <master_service_init_finish+226> "H\213\204$\230" #17 doveadm_mail_try_run (cmd_name=<optimized out>, argc=4, argv=0x5555557c73a0) at doveadm-mail.c:617 cmd = 0x5555557cb140 #18 0x00005555555699a4 in main (argc=4, argv=0x5555557c73a0) at doveadm.c:398 cmd_name = 0x5555557c73dd "backup" i = <optimized out> ---Type <return> to continue, or q <return> to quit--- quick_init = false c = <optimized out> ----------------------------------------------- Fabio Onorini ----------------------------------------------- Gruppo SeCoGes srl Via Enzo Ferrari, 4/6 - 25134 - Brescia - Italy Tel. +39 030 23 06 877 Fax +39 030 23 15 512 Sito Web: http://www.secoges.com Indirizzo EMail (assistenza): easymail(a)secoges.com ------------------------------------------------

1 0