October 2014 - dovecot

mmap_disable=yes not honored always
by Peter Benko 15 Oct '14

15 Oct '14

Hi all, I'm experimenting with having the mail store on a 9p file system that lacks mmap() functionality. So I disabled it in dovecot: mmap_disable = yes However, I keep getting the following error messages in my log: Oct 15 16:55:00 computer-name dovecot: imap user(a)domain.com[192.168.1.3] Error: mmap() failed with file /mail/mailboxes/domain.com/user/indexpvt-accounts/other-user/mailboxes/INBOX/dovecot.index.pvt.log: Invalid argument I only get those messages for the shared folder's private indexes. This setup works perfectly on an ext2/3/4 fs. Any opinions, is this a bug? Is mmap_disable not mandatory for all operations? Best regards, Peter

1 0

Request to list owner
by Jorge Bastos 15 Oct '14

15 Oct '14

Hi list owner, Sorry for this. Is it possible to change the reply-to in mailman to be the list, and not the person who replied? Also the [Dovecot] on the subject would be handy. Just two things to get better. Thanks,

10 15

permissions when moving mails
by Alexander 'Leo' Bergolth 15 Oct '14

15 Oct '14

Hi! I am using Unix groups to control access to shared mailboxes. This works fine as long as files are not moved between mailboxes that are owned by different groups. The move operation doesn't change group ownership of the mail. Is there a way to force dovecot to e.g. inherit the ownership from the parent folder, not only for new messages bot also for messages moved from one folder to another? The wiki says: http://wiki2.dovecot.org/SharedMailboxes/Permissions -------------------- 8< -------------------- When mails are copied between Maildirs, it's usually done by hard linking. If the source and destination directory permissions are different, Dovecot create a new file and copies data the slow way so that it can assign the wanted destination permissions. The source and destination permission lookups are done only by looking at the mailbox root directories' permissions, not individual mail files. This may become a problem if the mail files' permissions aren't as Dovecot expects. -------------------- 8< -------------------- Does dovecot only look at the permissions or does it also take user/group ownership into account? The behavior I experience is that even though the mailbox directories are owned by different groups, dovecot doesn't copy the mails. (See my example below.) Setting maildir_copy_with_hardlinks = no seems to fix the problem but maybe this behavior should be considered as a bug? I am using dovecot 2.2.14 and maildir mailboxes. Cheers, --leo P.S.: Here is an example of the current behavior: before the move from PAS to GSD: -------------------- 8< -------------------- GSD/.Eingang/cur/: total 0 drwxrws--- 2 nobody gsd 6 Oct 13 13:46 . drwxrws--- 5 nobody gsd 81 Oct 15 09:41 .. PAS/.Eingang/cur/: total 4 drwxrws--- 2 nobody pas 89 Oct 15 09:39 . drwxrws--- 5 nobody pas 79 Oct 15 09:39 .. -rw-rw-r-- 1 nobody pas 4073 Jul 8 08:14 1404967500.M656935P20690.samba.intern.rk-klosterneuburg.at,S=4073,W=4213:2,S -------------------- 8< -------------------- after the move: -------------------- 8< -------------------- GSD/.Eingang/cur/: total 4 drwxrws--- 2 nobody gsd 87 Oct 15 09:42 . drwxrws--- 5 nobody gsd 81 Oct 15 09:42 .. -rw-rw-r-- 1 nobody pas 4073 Jul 8 08:14 1413358925.M22574P3008.samba.intern.rk-klosterneuburg.at,S=4073,W=4213:2,S PAS/.Eingang/cur/: total 0 drwxrws--- 2 nobody pas 6 Oct 15 09:42 . drwxrws--- 5 nobody pas 79 Oct 15 09:42 .. -------------------- 8< -------------------- Mailbox directories are: -------------------- 8< -------------------- drwxrws--- 4 nobody pas 143 Nov 25 2013 PAS drwxrws--- 5 nobody pas 79 Oct 15 09:57 PAS/.Eingang drwxrws--- 10 nobody gsd 4096 Dec 11 2013 GSD drwxrws--- 5 nobody gsd 81 Oct 15 09:42 GSD/.Eingang -------------------- 8< -------------------- (In my own litte world, the mail should now be owned by the gsd group.. :-)) -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria

1 0

Dbox and Exim
by C Peters 15 Oct '14

15 Oct '14

The Dovecot wiki [1] doesn't list either Exim, Postfix or anything else as supporting the dbox. I did some searching in the Exim archives and have found only one message mentioning dbox, sdbox or mdbox. [2] I think an RFC would go a long way towards getting the format supported. Has Timo, or anyone else, considered submitting an RFC to the IETF? My question was how are we going to use it if Exim doesn't support it. It looks like we can filter it and convert it with dovecot-lda. [3] I have generally avoided using Procmail for a few years because of various problems and it's a dead software project that hasn't been developed in what 15 to 20 years. I am thinking about trying it with Exim's built in Maildrop filtering support. Has anybody tried that? What else are people using to enable dbox? The more specific question on the problem I am trying to solve. I have been using mbox for years, and I can't even get Dovecot, IMAP and Thunderbird working on the LAN because it can't handle the GB's of old mail. Mutt works on the local machine, but I want something that works while I am using something other than ssh and mutt. For example I want to try Mailpile and K9. I would like to try dbox, but given it's lack of support in Exim or Postfix, I'll probably convert to maildir. Thanks, Chuck 1. http://wiki2.dovecot.org/MailboxFormat 1. https://lists.exim.org/lurker/message/20110220.014144.87010abd.en.html Thread view: https://lists.exim.org/lurker/thread/20110216.223226.015539ba.en.html#i2011… 2. http://ramblings.narrabilis.com/mdboxprocmail

7 9

Proxy problem: "[COMPRESSIONACTIVE] TLS compression already enabled"
by Ralf Hildebrandt 15 Oct '14

15 Oct '14

We're proxying using 2.2.14~rc1 (on our IMAP Proxy) to two dovecot backend servers running dovecot-2.2.13-r1 When we're using Thundebird to connect to the dovecot proxy, we're getting the message "The mail server for account ACCOUNTNAME responded: [COMPRESSIONACTIVE] TLS compression already enabled" But why? Capabilities reported by the proxy: . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY COMPRESS=DEFLATE QUOTA] Logged in Capabilities of the backend servers when connecting directly: . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY COMPRESS=DEFLATE QUOTA] Logged in -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

3 5

bandwidth shaping
by Alan Bailward 14 Oct '14

14 Oct '14

Hey all, sorry if this has been asked before, but I haven’t really found a good solution. I have a mac at home and randomly, maybe once a week or once a month, Mail.app will go crazy and use as much bandwidth as I have to pull down mail. I’m not home so I can’t tell what it’s done or why, and sometimes it stops after an hour, sometimes it goes all night, leaving me with a potentially big bandwidth bill for my colo’d server which hosts my mail (dovecot 1.2.9, postfix 2.7.0 on Ubuntu LTS 10.04). I’m wondering if there’s a way to restrict bandwidth that’s the “accepted” and proper way with dovecot? There isn’t anything (AFAICT) built into dovecot, and the alternative seems to be setting up bandwidth shaping with ‘ip’, but that is a pretty horrible nest of ugliness. Is there a fairly simple setup to set it so imap/pop traffic can only ever use 1mb/s (or some arbitrary number)? I’m not looking for a one liner, but each time this happens I say “I’m going to figure this out and fix this”, then get lost in ip and shaping queues and give up in frustration until the next time it happens, when I say “I’m *really* going to figure this out this time…”. Anyone have any pointers? TIA Alan

2 1

auth-worker goes wild
by Hans Morten Kind 14 Oct '14

14 Oct '14

dovecot-2.2.13 running on Solaris-10, app 5000 concurrent connections, and a pretty standard config where passdb is set up with driver = pam. We have had some situations where an auth-worker gets into a loop, the load on the server rises and the auth-master seems blocked untill finally no new imap nor pop session are accepted. A 'doveadm reload' restores normal operations. With debug logging enabled we are observing this: auth: Debug: pam(,10.10.30.33,<7WMu3UkEGQCBsR4h>): cache miss auth-worker(24556): Debug: pam(,10.10.30.33): lookup service=dovecot auth-worker(24556): Debug: pam(,10.10.30.33): #1/1 style=2 msg=Please enter user name: ... auth-worker(24556): Debug: pam(,10.10.30.33): #1/1 style=2 msg=Please enter user name: etc etc After some time dovecot only logs Warning: Auth process not responding, delayed sending initial response (greeting) Disconnected: Auth process broken (disconnected before auth was ready, waited 26 secs) 'truss -wall -aep PID' on the looping auth-worker shows the loop: 0.0757 write(2, 0x080AE0C0, 71) = 71 0101 2 4 5 5 6 p a m ( , 1 0 . 1 0 . 3 0 . 3 3 ) : # 1 / 1 s t y l e = 2 m s g = P l e a s e e n t e r u s e r n a m e : \n 0.0826 write(2, 0x080AE0C0, 71) = 71 0101 2 4 5 5 6 p a m ( , 1 0 . 1 0 . 3 0 . 3 3 ) : # 1 / 1 s t y l e = 2 m s g = P l e a s e e n t e r u s e r n a m e : \n The host at 10.10.30.33 is running RoundCube for app 1000 concurrent users. This problem started after we provided our users RoundCube, and every time we end up in this stuck situation, it is the RoundCube-server making the connections where the username has zero length. I am unable to reproduce how RoundCube puts dovecot into this mess. With the enclosed patch installed, the loop terminates and normal operation are restored immediately: auth: Debug: pam(,10.10.30.33,<Q8kV6AEFYwCBsR4h>): cache miss auth-worker(18658): Debug: pam(,10.10.30.33): lookup service=dovecot auth-worker(18658): Debug: pam(,10.10.30.33): #1/1 style=2 msg=Please enter user name: auth-worker(18658): Fatal: NO USER? auth-worker(18658): Error: Raw backtrace: 0xfef0798c -> 0xfef06d3b -> 0x807ca6e -> 0xfecc460e -> 0xfecc1d1e -> 0xfe3c0af4 -> 0xfecc2608 -> 0xfecc27f1 -> 0x807cc5e -> 0x807d162 -> 0x807d310 -> 0x8069ca1 -> 0x806ab72 -> 0x806aeec -> 0xfef1e866 -> 0xfef2001c -> 0xfef1e9ff -> 0xfef1e956 -> 0xfeeb0172 -> 0x8071bf2 -> 0x805a650 auth: Error: auth worker: Aborted PASSV request for : Worker process died unexpectedly I think however, that the pam driver should not have been initiated when the username has zero length, definitely there is nobody around to answer the prompt for a username ... hmk

2 2

Doveadm quota recalc sets quota for wrong quota root
by Rick van den Hof 14 Oct '14

14 Oct '14

Hi, In our setup, we use two quota roots. One for user quota and one for domain quota. If a user has no quota, then the domain quota is applied. For a user with user quota this is how it looks: # doveadm quota get -u test(a)shellz.nl Quota name Type Value Limit % Domain quota STORAGE 1439155 2560000 56 Domain quota MESSAGE 21257 - 0 User quota STORAGE 0 102400 0 User quota MESSAGE 0 - 0 In this case, the whole domain contains 21257 messages but the account itself contains 0. For a user with only domain quota, this is how it looks: # doveadm quota get -u rick(a)shellz.nl Quota name Type Value Limit % Domain quota STORAGE 1439155 2560000 56 Domain quota MESSAGE 21257 - 0 User quota STORAGE 693299 - 0 User quota MESSAGE 12876 - 0 So far so good. My account contains 12876 messages. Now I've sent a message containing a 1mb.bin as attachment to test(a)shellz.nl: # doveadm quota get -u test(a)shellz.nl Quota name Type Value Limit % Domain quota STORAGE 1440540 2560000 56 Domain quota MESSAGE 21258 - 0 User quota STORAGE 1384 102400 1 User quota MESSAGE 1 - 0 Seems to be fine, the message on disk is indeed 1.4MB and this has been added to both domain and user quota. Now see what happens when I run the following command: # doveadm quota recalc -u test(a)shellz.nl # doveadm quota get -u test(a)shellz.nl Quota name Type Value Limit % Domain quota STORAGE 1384 2560000 0 Domain quota MESSAGE 1 - 0 User quota STORAGE 1384 102400 1 User quota MESSAGE 1 - 0 The recalc action has updated the domain usage to reflect the specific user's usage (my 12876 messages are no longer counted). We log domain usage (usage_domain) in a seperate table from mailbox usage (usage_mailbox). These are the queries that set the usage: 256 Query BEGIN 256 Query DELETE FROM usage_domain WHERE domain = 'shellz.nl' 256 Query DELETE FROM usage_domain WHERE domain = 'shellz.nl' 256 Query INSERT INTO usage_domain (bytes,domain) VALUES ('1418118','shellz.nl') ON DUPLICATE KEY UPDATE bytes='1418118' 256 Query INSERT INTO usage_domain (messages,domain) VALUES ('1','shellz.nl') ON DUPLICATE KEY UPDATE messages='1' 256 Query COMMIT 257 Query BEGIN 257 Query DELETE FROM usage_mailbox WHERE userdomain = 'test(a)shellz.nl' 257 Query DELETE FROM usage_mailbox WHERE userdomain = 'test(a)shellz.nl' 257 Query INSERT INTO usage_mailbox (bytes,userdomain) VALUES ('1418118','test(a)shellz.nl') ON DUPLICATE KEY UPDATE bytes='1418118' 257 Query INSERT INTO usage_mailbox (messages,userdomain) VALUES ('1','test(a)shellz.nl') ON DUPLICATE KEY UPDATE messages='1' 257 Query COMMIT How do I prevent this from happening? It should only update the usage_mailbox table when I run the recalc command. Could this be because we use seperate tables for domain and user usage? dovecot -n: # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-37-generic x86_64 Ubuntu 14.04.1 LTS auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890+=_.-@& dict { expire = mysql:/etc/dovecot/dovecot-sql-expire.conf quotadomaindict = mysql:/etc/dovecot/dovecot-sql-quota-domain.conf quotauserdict = mysql:/etc/dovecot/dovecot-sql-quota-user.conf } disable_plaintext_auth = no listen = *,[::] mail_fsync = never mail_location = maildir:~/Maildir mail_plugins = quota expire managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-sql-user.conf driver = sql } plugin { expire = Trash expire2 = Spam expire_dict = proxy::expire mail_log_events = delete copy save expunge mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:Domain quota:%d:proxy::quotadomaindict quota_rule2 = Trash:ignore quota_warning = storage=99%% doquotawarning 99 %u quota_warning2 = storage=95%% doquotawarning 95 %u quota_warning3 = storage=75%% doquotawarning 75 %u quota2 = dict:User quota::proxy::quotauserdict quota2_rule2 = Trash:ignore quota2_warning = storage=99%% doquotawarning 99 %u quota2_warning2 = storage=95%% doquotawarning 95 %u quota2_warning3 = storage=75%% doquotawarning 75 %u } protocols = imap pop3 service auth-worker { unix_listener auth-worker { mode = 0600 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service doquotawarning { executable = script /usr/bin/doquotawarning.py unix_listener doquotawarning { user = vmail } user = vmail } service imap-login { chroot = process_limit = 200 process_min_avail = 2 } service imap { executable = imap postlogin } service managesieve-login { chroot = } service pop3-login { chroot = process_limit = 50 process_min_avail = 2 } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login /usr/local/postfixint/dolastlogin.py user = $default_internal_user } ssl_cert = </etc/ssl/certs/wild.totaal.net/wild.totaal.net.pem ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_key = </etc/ssl/certs/wild.totaal.net/wild.totaal.net.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql-user.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep imap_logout_format = bytes=%i/%o mail_max_userip_connections = 50 mail_plugins = quota imap_quota mail_log notify expire } protocol pop3 { mail_max_userip_connections = 50 mail_plugins = quota expire pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o pop3_uidl_format = %f } protocol lda { deliver_log_format = msgid=%m: %$ mail_fsync = optimized mail_plugins = quota sieve expire postmaster_address = hostmaster(a)vps04.totaal.net rejection_reason = Your message to <%t>' was automatically rejected:%n%r } dovecot-sql-quota-domain.conf: connect = xxx map { pattern = priv/quota/storage table = usage_domain username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = usage_domain username_field = domain value_field = messages } dovecot-sql-quota-user.conf: connect = xxx map { pattern = priv/quota/storage table = usage_mailbox username_field = userdomain value_field = bytes } map { pattern = priv/quota/messages table = usage_mailbox username_field = userdomain value_field = messages } Thanks in advance to anyone who might be able to shed some light in this situation :). Kind regards, Rick van den Hof -- Manager Engineering Totaalnet Internet Works B.V. IJsselburcht 4e 6825 BP Arnhem +31(0)26-3844944 | r.vandenhof(a)tiw.eu (PGP Key: 0x5A66E935)

2 1

Re: delete/archive old mail
by F. 14 Oct '14

14 Oct '14

>>>> Hi everybody, >>>> I store user's mail in the old mbox format. >>>> I have many scripts to manage users that works fine with mbox. >>>> I use the very old Expire_mail.pl script to delete mail older than NN >>>> days for selected users (nightly cron job). Still works fine with my >>>> CentOS dovecot-2.0.16. >>>> Now I want move the mail to a sort of archive folder instead of simply >>>> deleting it from the inbox for some users. >>>> Is there a similar script or some simple triks? >>>> >>>> Thanks, F. >>> >>> perhaps you will find some ideas here >>> >>> https://sys4.de/de/blog/2013/02/07/mailarchiv-mit-dovecot-und-postfix-sorti… >>> >>> >>> ... >>> #/bin/bash >>> YEAR=`/bin/date +%Y` >>> /usr/bin/doveadm expunge -d -u *(a)backup.server.org mailbox >>> user-backup\*$YEAR*\* savedbefore 10d >>> ... >> >> Thanks for replay. >> My German is really poor, but from a quick look of the example above and >> man doveadm[-expunge], seems that it can be used to replace the old >> Expire_mail.pl (I need to study the right syntax ...) >> >> But still there is the move/archive old mail activity not solved. > > i use the bc feature of postfix to move a copy of all mail per > maildomain into a backup maildomain archive folder, as this is done via lmtp > i am able to sort mails via sieve per user and date folder, depending > on how you like to setup you may use imap acl to give seen rights to the > orig user, so the may restore their own mail whenever they want and/or > their domain postmaster can do it for them. > > But thats only one chance you can do it, you have to decide what fits > best to your needs. > > >> >> doveadm-altmove seems interesting, but in the man page it is reported >> only work with dbox (dbox-only), and my inboxes are in mbox format. >> >> Any other advice? Sorry for the replay to sender instead of the list. As I originally wrote, my need is script or a command like the old Expire_mail.pl that parse a local mbox folder and move/copy mail older than NN days to another local mbox folder. Probably for a Perl programmer is only matter of modify this script or something similar. I searched in docs, wiki, ... but not noticied something that can simply be inserted in a cron job to do the work. Only migration tools or complex (for me), library that can be used to manipulate folders :-) Thanks, F.

1 0

Re: Mailbox format
by Robert Schetterer 14 Oct '14

14 Oct '14

Am 13.10.2014 um 23:29 schrieb Jorge Bastos: > Guys, > > Whats the best mailbox format: > - dbox > - mbox > - maildir > - ?!? there is no universal answer to this a minimum answer might be, maildir is ok in a small non cluster setup ever, please study wiki and archive for more info Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

2 1