February 2016 - dovecot

VS: Re: SQLite driver and auth-worker credentials
by Aki Tuomi 24 Feb '16

24 Feb '16

this is probably something that ought to be fixed. Read only scenario for auth/userdb is quite plausible. ---Aki TuomiDovecot oy-------- Alkuperäinen viesti --------Lähettäjä: Lev Serebryakov <lev(a)serebryakov.spb.ru> Päivämäärä: 24.2.2016 23.32 (GMT+02:00) Saaja: james(a)lottspot.com Kopio: dovecot(a)dovecot.org Aihe: Re: SQLite driver and auth-worker credentials Hello James, Wednesday, February 24, 2016, 11:17:12 PM, you wrote: >> But system should assign all secondary GIDs to effective UID? > separate 'group =' directive in addition to the 'user =' directive > (http://wiki2.dovecot.org/Services) Oh, I missed this one, thank you -- Best regards, Lev mailto:lev@serebryakov.spb.ru

1 0

SQLite driver and auth-worker credentials
by Lev Serebryakov 24 Feb '16

24 Feb '16

I want to use SQLite database as storage for auth and user databases. I've encountered two problems here: (1) There is no way to open SQLite database read-only (via sqlite3_open_v2() call with SQLITE_OPEN_READONLY flag). It looks bad. I don't need (and want) to give dovecot rights to write to this database. (2) I've created system group "hostingdb", added "dovecot" user to it and gives 660 rights to database file, but still "auth-worker" could not open database and complains to log file. Now I'm set "user = root" for auth-worker, but I don't like it! Why auth-worker doesn't belong to "hostingdb" group? -- // Black Lion AKA Lev Serebryakov

2 4

Script dovecot ACLs, Quota and doveadm
by Chris 24 Feb '16

24 Feb '16

Dear All, I'd like to set a) Quota for mailboxes in private and public namespace b) ACLs for both. What's the easiest way to do this? Do I have to call doveadm for every mailbox (private and public)? Is it required to provide a username to doveadm? Can I use the same user for all if it is defined in global ACLs? For Cyrus there are perl modules which have the same functions as cyradm. Is there something like this for dovecot? Are there any modules available? Haven't found anything in CPAN. - Chris

1 0

Streaming MOVE commands
by gallego＠cri.ensmp.fr 24 Feb '16

24 Feb '16

Dear Dovecot devs, is streaming multiple MOVE commands by clients allowed? I am getting duplicated messages with the GNUS mail client, the interchange looks like this: *stream two moves to different folders* > 9019 UID MOVE 4062,4066,4068 "folder0" > 9020 UID MOVE 4063:4064,4067,4069:4072 "folder1" *the messages are copied* > * OK [COPYUID 1424475218 4062,4066,4068 376:378] Moved UIDs. > * OK [COPYUID 1424475231 4063:4064,4067,4069:4072 26:32] Moved UIDs. *however expunge fails to clean 4063, 4064, and 4067* > * VANISHED 4062,4066,4068:4072 thus 4063, 4064, and 4067 end both in inbox and folder1 producing duplicate messages (more details at [1]). At the GNUS mailing list, we were wondering about what should be the correct reading of RFC6851. Version and config information below. Best regards, Emilio [1] More details in the thread http://permalink.gmane.org/gmane.emacs.gnus.general/86813 [2] Version $ /usr/sbin/dovecot --version 2.2.13 [3] Config $ /usr/sbin/dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 8.3 ext4 mail_location = maildir:/home/%u/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap sieve" ssl_cert = ... ssl_key = ... userdb { driver = passwd }

3 6

Automatically inferring %d on multi-domain virtual install ?
by Gabriel L. Somlo 24 Feb '16

24 Feb '16

Hi, I'm trying to allow domain-less logins for a multi-domain virtual IMAP server, and wondering if I can automatically infer the domain (value of variable %d) from the local IP (%l) or the hostname used by the client when connecting to my server. Let's say I have two host names: mail.foo.org (10.0.0.100) and mail.bar.com (10.0.0.200), with forward and reverse DNS configured to resolve A and PTR records in either direction. Let's also say I have 10.0.0.100 and 10.0.0.200 set up as secondaries on my server's loopback interface, and routing is set up to bring client traffic to me for both of those IP addresses. The relevant bits of my dovecot.conf are: ---%<------------------------------------------------------------------ mail_location = maildir:/var/lib/vmail/%d/%n passdb { driver = passwd-file args = /var/lib/vmail/etc/postfix/userdb } userdb { driver = static args = uid=dovenull gid=dovenull home=/var/lib/vmail/%d/%n } ---%<------------------------------------------------------------------ And my userdb passwd-file right now includes: ---%<------------------------------------------------------------------ user1@foo.org:{PLAIN}user1foo user2@foo.org:{PLAIN}user2foo user1@bar.com:{PLAIN}user1bar user2@bar.com:{PLAIN}user2bar ---%<------------------------------------------------------------------ Right now, user1(a)foo.org must configure their imap client like so: IMAP server: mail.foo.org username: user1(a)foo.org password: user1foo I would like to require this (and other) users to only have to set: IMAP server: mail.foo.org username: user1 password: ... and have dovecot somehow infer the "@foo.org" domain based on the fact that the connection was made to 10.0.0.100, which is mail.foo.org, and therefore the domain can *only* be "@foo.org". I could start out by splitting my user database into two files: userdb.foo.org ---%<------------------------------------------------------------------ user1:{PLAIN}user1foo user2:{PLAIN}user2foo ---%<------------------------------------------------------------------ userdb.bar.com ---%<------------------------------------------------------------------ user1:{PLAIN}user1bar user2:{PLAIN}user2bar ---%<------------------------------------------------------------------ ... then modify dovecot.conf's passdb setup like so: ---%<------------------------------------------------------------------ passdb { driver = passwd-file args = /var/lib/vmail/etc/postfix/userdb.%d } ---%<------------------------------------------------------------------ ... but how would I insure that %d is set to the proper value based on e.g. a reverse lookup of %l, which, in foo.org's case would be 10.0.0.100, and resolve to mail.foo.org, and *somehow* that would match %d == "@foo.org" ? Is this even possible in the first place, or am I just being too fussy about the aesthetics of my users' imap client config files ? :) :) Thanks much, --Gabriel

5 9

Dovecot + Ceph Cluster Sizing
by Michele Soragni 24 Feb '16

24 Feb '16

Hi all. We are searching for information about how to size dovecot servers in a clustered architecture (dovecot + postfix + roundcube + mysql). We need to migrate mailboxes from an old Exchange installation: 60k mailbox 900 GB total storage 14 MB/mailbox About 133 email/mailbox Low usage: about 2k simultaneous imap/pop3 connections This is how we are thinking to build our new architecture 3 x Dell R630 or R530 servers with 6x300SAS disks for Ceph storage with 1 replica . Total storage = 2,5 TB for mailboxes 3 x Dell R630 or R530 with 2x CPU 6core,64GB RAM, 4x 300SAS disks hosting the following VMs in a VMware cluster: - 3 x VM Dovecot - 3 x VM Postfix - 3 x VM Roundcube webmail - 3 x VM MySQL All VMs are behind a balancer All OS disks are on local storage Dovecot VMs share Ceph storage with OCFS2 filesystem for the mailboxes. I'd like to have some hints about CPU and memory for Dovecot VMs. 2 vCPU and 8 GB RAM each sould be enough? Is there a way to calculate this? Do you think the storage performance could be a problem using Ceph + OCFS2? Is Ceph + XFS + Dsync replication a better choice? Thanks! Michele

1 0

Segfault using doveadm dsync
by fabio.onorini＠secoges.com 24 Feb '16

24 Feb '16

I'm trying to migrate a imap mailbox from dovecot 1.2.15 to a new imap server with dovecot 2.2.13. I would preserving the IMAP UIDs and POP3 UIDLs. So I'm using doveadm backup command but after folders analyze, command return segmentation fault. Any hint? GDB result: sync(easymail): Debug: brain S: Deleting mailbox 'Bozze' (GUID f0836f3222c335cfa383cb38ff0e3183): UIDNEXT is too high (9 > 1) dsync(easymail): Debug: brain S: Deleting mailbox 'Cestino' (GUID cd0edec22e40323d878a39dc8e432e5a): UIDNEXT is too high (628 > 1) dsync(easymail): Debug: brain S: Deleting mailbox 'INBOX' (GUID c92f64f79f0d1ed01e6d5b314f04886c): UIDVALIDITY changed (1456239585 -> 1456310264) dsync(easymail): Debug: brain S: Deleting mailbox 'Modelli' (GUID f0933f8b90e6eaf3a00a0220b173f3b4): UIDNEXT is too high (2 > 1) dsync(easymail): Debug: brain S: Deleting mailbox 'Sent' (GUID 7d3c7eaa71cdf47ee8a1192687cda8cd): UIDNEXT is too high (39 > 1) Program received signal SIGSEGV, Segmentation fault. imapc_client_mailbox_cmd (box=0x0, callback=callback@entry=0x7ffff76d7140 <imapc_simple_callback>, context=context@entry=0x7fffffffe2d0) at imapc-client.c:351 351 imapc-client.c: No such file or directory. (gdb) bt full #0 imapc_client_mailbox_cmd (box=0x0, callback=callback@entry=0x7ffff76d7140 <imapc_simple_callback>, context=context@entry=0x7fffffffe2d0) at imapc-client.c:351 cmd = <optimized out> __FUNCTION__ = "imapc_client_mailbox_cmd" #1 0x00007ffff76d7915 in imapc_mailbox_noop (mbox=mbox@entry=0x5555558049e0) at imapc-storage.c:154 cmd = <optimized out> sctx = {client = 0x5555557e89b0, ret = -2} #2 0x00007ffff76d5c18 in imapc_mailbox_sync_init (box=0x5555558049e0, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at imapc-sync.c:476 mbox = 0x5555558049e0 list = <optimized out> capabilities = <optimized out> changes = false ret = 0 #3 0x00007ffff76e9289 in mailbox_sync_init (box=0x5555558049e0, flags=(MAILBOX_SYNC_FLAG_FULL_READ | MAILBOX_SYNC_FLAG_FIX_INCONSISTENT)) at mail-storage.c:1677 _data_stack_cur_id = 3 ctx = <optimized out> #4 0x00007ffff76e93d7 in mailbox_sync (box=box@entry=0x5555558049e0, flags=<optimized out>, flags@entry=MAILBOX_SYNC_FLAG_FULL_READ) at mail-storage.c:1725 ctx = 0x555555804ea8 status = {sync_delayed_expunges = 0} #5 0x00007ffff7715bab in mailbox_expunge_all_data (box=0x5555558049e0) at index-storage.c:648 ctx = 0x7ffff7714e57 <index_storage_mailbox_open+615> t = 0x5555558049e0 mail = 0x7fffffffe4b4 search_args = 0x7fffffffe558 #6 index_storage_mailbox_delete (box=0x5555558049e0) at index-storage.c:701 metadata = {guid = "\264\344\377\377\377\177\000\000\000\000\000\000\000\000\000", virtual_size = 140737488348504, cache_fields = 0x555555807258, precache_fields = (MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_VIRTUAL_SIZE | MAIL_FETCH_IMAP_ENVELOPE | MAIL_FETCH_REFCOUNT | unknown: 1426065408), backend_ns_prefix = 0x7ffff73b1e59 <t_pop_check+9> ";\003u\v\307\003", backend_ns_type = (unknown: 1434470880)} status = {messages = 4294960472, recent = 32767, unseen = 0, uidvalidity = 0, uidnext = 1434582544, first_unseen_seq = 0, first_recent_uid = 232594432, last_cached_seq = 3612951791, highest_modseq = 93824995052000, highest_pvt_modseq = 93824992578308, keywords = 0x1, permanent_flags = 4151223616, permanent_keywords = 1, allow_new_keywords = 1, nonpermanent_modseqs = 1, no_modseq_tracking = 1, have_guids = 1, have_save_guids = 1, have_only_guid128 = 1} ret_guid = <optimized out> #7 0x00007ffff76ea737 in mailbox_delete (box=box@entry=0x5555558049e0) at mail-storage.c:1319 ret = <optimized out> #8 0x00005555555895d4 in dsync_brain_mailbox_tree_sync_change (brain=brain@entry=0x5555557fcd00, change=<optimized out>) at dsync-brain-mailbox-tree-sync.c:182 box = 0x5555558049e0 destbox = 0x5555558007c8 errstr = 0x5555557ff750 "\030\366\177UUU" func_name = 0x0 ---Type <return> to continue, or q <return> to quit--- storage_name = <optimized out> error = 32767 ret = <optimized out> __FUNCTION__ = "dsync_brain_mailbox_tree_sync_change" #9 0x00005555555892db in dsync_brain_mailbox_trees_sync (brain=0x5555557fcd00) at dsync-brain-mailbox-tree.c:291 ctx = 0x55555580ab40 change = <optimized out> sync_type = <optimized out> sync_flags = <optimized out> #10 dsync_brain_recv_mailbox_tree_deletes (brain=brain@entry=0x5555557fcd00) at dsync-brain-mailbox-tree.c:440 deletes = 0x5555557fa0a0 i = <optimized out> count = 0 sep = 47 '/' __FUNCTION__ = "dsync_brain_recv_mailbox_tree_deletes" #11 0x00005555555860e8 in dsync_brain_run_real (changed_r=0x7fffffffe74b, brain=0x5555557fcd00) at dsync-brain.c:565 ret = true orig_state = DSYNC_STATE_RECV_MAILBOX_TREE_DELETES orig_box_recv_state = DSYNC_BOX_STATE_MAILBOX orig_box_send_state = DSYNC_BOX_STATE_MAILBOX changed = false #12 dsync_brain_run (brain=0x5555557fcd00, changed_r=changed_r@entry=0x7fffffffe74b) at dsync-brain.c:610 _data_stack_cur_id = 2 changed_r = 0x7fffffffe74b brain = 0x5555557fcd00 #13 0x00005555555846f7 in cmd_dsync_run_local (changes_during_sync_r=<synthetic pointer>, ibc2=<optimized out>, brain=0x5555557e1860, user=<optimized out>, ctx=0x5555557cd590) at doveadm-dsync.c:384 brain2 = 0x5555557fcd00 set_parser = <optimized out> location = <optimized out> user2 = 0x0 brain1_running = true changed1 = true ret = <optimized out> set_line = <optimized out> brain2_running = true changed2 = false #14 cmd_dsync_run (_ctx=0x5555557cd590, user=<optimized out>) at doveadm-dsync.c:596 ---Type <return> to continue, or q <return> to quit--- ctx = 0x5555557cd590 ibc = 0x5555557d50d0 ibc2 = 0x5555557e14d0 brain = 0x5555557e1860 set = {process_title_prefix = 0x0, sync_namespaces = {arr = {buffer = 0x5555557c0378, element_size = 8}, v = 0x5555557c0378, v_modifiable = 0x5555557c0378}, sync_box = 0x0, sync_box_guid = '\000' <repeats 15 times>, exclude_mailboxes = 0x0, lock_timeout_secs = 0, state = 0x0} ns = 0x5555557cd670 strp = <optimized out> brain_flags = <optimized out> remote_errors_logged = false changes_during_sync = false status = 0 ret = 0 #15 0x0000555555569e4b in doveadm_mail_next_user (ctx=0x5555557cd590, input=<optimized out>, error_r=0x7fffffffe898) at doveadm-mail.c:314 error = 0x5555557cd590 "p\325|UUU" ret = <optimized out> error_r = 0x7fffffffe898 input = <optimized out> ctx = 0x5555557cd590 ip = <optimized out> ret = 0 #16 0x000055555556ac2c in doveadm_mail_cmd (argv=0x5555557c73b8, argc=4, cmd=0x5555557cb140) at doveadm-mail.c:525 input = {module = 0x0, service = 0x5555555a7367 "doveadm", username = 0x5555557c73e7 "easymail", session_id = 0x0, local_ip = {family = 0, u = {ip6 = { __in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0} ctx = 0x5555557cd590 getopt_args = 0x5555557bf660 "+1dEfg:l:m:n:NPr:Rs:Ux:RAS:u:" ret = <optimized out> c = <optimized out> wildcard_user = 0x0 error = 0x7ffff73b7122 <master_service_init_finish+226> "H\213\204$\230" #17 doveadm_mail_try_run (cmd_name=<optimized out>, argc=4, argv=0x5555557c73a0) at doveadm-mail.c:617 cmd = 0x5555557cb140 #18 0x00005555555699a4 in main (argc=4, argv=0x5555557c73a0) at doveadm.c:398 cmd_name = 0x5555557c73dd "backup" i = <optimized out> ---Type <return> to continue, or q <return> to quit--- quick_init = false c = <optimized out> ----------------------------------------------- Fabio Onorini ----------------------------------------------- Gruppo SeCoGes srl Via Enzo Ferrari, 4/6 - 25134 - Brescia - Italy Tel. +39 030 23 06 877 Fax +39 030 23 15 512 Sito Web: http://www.secoges.com Indirizzo EMail (assistenza): easymail(a)secoges.com ------------------------------------------------

1 0

Require different setting for mail_location for each of POP3S and IMAPS protocols
by Andrew McGlashan 23 Feb '16

23 Feb '16

Okay, Let's try this a different way. I've been able to configure past dovecot to have a different mail_location setting for the same user, but with different protocols. When the user requests POP3S, I adjusted the mail_location to /var/mail/$USER (with mbox in use). When that *same* user requested IMAPS protocol, the required mail_location is the Maildir folder. This allows for independent two mail storage locations, one for the use of mbox and the other for the use of Maildir *and* for the same user. Some users only require mbox with others only requiring Maildir. So, I need to cater for each of these situations. How might I adjust current dovecot configuration to provide different mail_location settings for different protocols? Old dovecot allowed me to run a script before POP3S processing that gave me the chance to adjust the mail_location variable. That is what I need now, just for POP3S. Thanks AndrewM

2 2

syntax for overriding userdb fields from passdb config
by Thomas Möhle 23 Feb '16

23 Feb '16

Dear Dovecot developers, how exactly is the syntax to override a userdb field within the passdb configuration? I cannot find any documentation on this in the wiki. Let's say, I'd like to override the userdb extra field "chroot" with /var/mail. How exactly would be the notation? My auth-sql.conf.ext looks like this: passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } I already tried these modifications within the passdb area: override_fields = userdb_chroot=/var/mail override_fields = userdb:chroot=/var/mail override_fields = %{userdb:chroot}=/var/mail override_fields = %{userdb_chroot}=/var/mail but none of the above returned /var/mail when I tried to hand %{userdb:chroot} to an external script. Thanks for your help, Thomas Möhle

2 1

RE: Dovecot Bulletin
by Kevin Kershner 23 Feb '16

23 Feb '16

It also got the bulletin out to new users without admin intervention. Sent from Mobile ________________________________ From: Doug Hardie<mailto:bc979@lafn.org> Sent: ‎2/‎22/‎2016 4:02 To: Dovecot Mailing List<mailto:dovecot@dovecot.org>; Timo Sirainen<mailto:tss@iki.fi> Cc: Kevin Kershner<mailto:cstkersh@outlook.com> Subject: Re: Dovecot Bulletin > On 20 February 2016, at 18:14, Timo Sirainen <tss(a)iki.fi> wrote: > > On 21 Feb 2016, at 02:50, Kevin Kershner <cstkersh(a)outlook.com> wrote: >> >> I'd like to revisit and old post if I may, will/does Dovecot support the old >> qpopper "Bulletin" ability? >> >> Basically I need a simple way of posting bulletins to all domain users. >> Qpopper maintained a bulletin db for each user and sent them the next >> bulletin in sequence. > > I guess there could be a plugin that does this check on each login. But would it actually be useful? Why would it be better than simply sending the mail to all the users? For example: > > doveadm save -A < bulletin.txt The reasons for bulletins as I see it are: 1. The doveadm save command is undocumented. It does show a cryptic line in the output of the command "doveadm". However, it doesn't give any clue what it does or how to provide the message. Your note above provides considerably more information on that command. I tested it and it works as you have indicated though. 2. The doveadm save command causes the email to be saved in each user's mailbox. If you have a lot of users, thats a lot of wasted disk space. Qpopper's bulletins only kept one copy and every user downloaded from that copy. All that was retained per user was a counter of the last bulletin's sequence number that was downloaded. — Doug

3 3