January 2009 - dovecot

[Dovecot] Broken dovecot-uidlist files
by Ulrich Zehl 24 Feb '09

24 Feb '09

I have just upgraded to 1.1.8, hoping to fix these types of errors: Jan 20 10:39:27 laura deliver(xxxxxx(a)example.net): Broken file /srv/storage/mail/store/net/example/xx/xxxxxx/Maildir/dovecot-uidlist line 156: Invalid data: Jan 20 10:41:19 laura dovecot: POP3(yy.yy(a)example.at): Broken file /srv/storage/mail/store/at/example/yy.yy/Maildir/dovecot-uidlist line 2: Invalid data: Unfortunately, they still pop up. Since I suspect these errors to be responsible for customers complaining about downloading mails twice, I'd like to fix them. Our setup has multiple servers sharing the same NFS-mounted directory for maildirs and indexes; we have set mmap_disable = yes dotlock_use_excl = yes fsync_disable = no mail_nfs_index = yes lock_method = fcntl As far as I can tell, for at least one of these errors, there was no concurrent access to the mailbox in question. What can I do to solve this problem, or to analyze it further? Ulrich

3 11

[Dovecot] IMAP ACLs and global ACLs in v1.2
by Timo Sirainen 23 Feb '09

23 Feb '09

I just committed code for IMAP ACL support based on the code from Kolab people. I did quite large changes though. I also changed how global ACLs are handled. Previously local ACLs could override global ACLs, but now that users are able to modify the ACLs I think it should be the other way around. From what I've heard global ACLs are typically added for things like 1) disallowing user from doing something in specific mailboxes such as expunge messages, 2) giving a specific master user access to some mailboxes ONLY, such as voicemail. For 1) case a user really shouldn't be able to grant himself privileges back but still should be able to share the mailbox to others (so the global ACL should have 'a' privilege). For 2) case a user shouldn't be able to remove privileges from the voicemail. So now the user can freely update the local ACLs in whatever way he wants. But in the end if there are any global ACLs, all the local ACLs for those identifiers are ignored. Also local negative identifiers are ignored, so that e.g. global "user=foo lr" can't be disabled by a local "-user=foo lr" ACL. I guess I should update the Wiki ACL page about how exactly the ACLs are applied now that they're well in my memory. :) One thing I'm not really sure about is the "owner" handling. IMAP ACL specifications have no such concept. I think many/most other servers simply add a default ACL for the user name directly. It's a useful concept though, especially with the global ACLs. So currently Dovecot replies: x getacl inbox * ACL "inbox" "owner" lrwstiekxacd x OK Getacl completed. But should it just internally convert "owner" to "username" when replying? But then again if there's a separate rule directly for the "username" it breaks. Do any of the existing ACL-capable IMAP clients want to see the username in the ACL reply or can they handle "owner" just fine? Also what about global ACLs? The current code just lists them twice with local ACLs, which is definitely wrong: x getacl foo * ACL "foo" "xx" w "xx" lr "owner" lrwstiekxacd x OK Getacl completed. Perhaps the global ACLs could have a prefix character? Maybe '#'? Which reminds me, the current dovecot-acl file is mapped to IMAP ACLs like this: dovecot-acl : IMAP ACL - anyone : anyone - authenticated : authenticated - owner : owner - user=name : name - group=name : $name - group-override=name : !$name So all global ACLs would get the # prefix, like #anyone, #name, #!$name. Any thoughts? BTW. Listing shared mailboxes still doesn't work. I guess we'll see tomorrow if I still have energy to get that done.

6 14

[Dovecot] Problem with v1.1.10
by Cyrus Rahman 22 Feb '09

22 Feb '09

I've been using 1.1.8 successfully but tried out 1.1.10 this morning. I have a combination of system and virtual users. The system users were unaffected by the change, but the virtual users all lost access to their mail because their home directories, as interpreted by dovecot, moved from user@domain to user@user@domain. The location of the home directory is given to dovecot for virtual users using the static db as uid=vmail gid=vmail home=/var/vmail/%u mail=maildir:~/Inboxdir Were there any changes in this area that might have caused the problem? Cyrus dovecot -n output: # 1.1.10: /usr/local/etc/dovecot.conf # OS: FreeBSD 7.1-RELEASE-p2 amd64 protocols: imaps managesieve ssl_cert_file: /etc/ssl/dovecot/certs/dovecot.pem ssl_key_file: /etc/ssl/dovecot/private/dovecot.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(managesieve): /usr/local/libexec/dovecot/managesieve-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(managesieve): no verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 1000 mail_privileged_group: mail mail_location: mbox:~/.dovecot:INBOX=/var/mail/%u mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(managesieve): /usr/local/libexec/dovecot/managesieve mail_plugins(default): expire quota imap_quota trash fts fts_squat antispam autocreate mail_plugins(imap): expire quota imap_quota trash fts fts_squat antispam autocreate mail_plugins(managesieve): mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(managesieve): /usr/local/lib/dovecot/managesieve imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(managesieve): dict_db_config: /usr/local/etc/dovecot-db.conf sieve_storage(default): sieve_storage(imap): sieve_storage(managesieve): ~/.sieve sieve(default): sieve(imap): sieve(managesieve): ~/.dovecot.sieve namespace: type: private separator: / prefix: #mbox/ inbox: yes hidden: yes subscriptions: yes namespace: type: private separator: / location: maildir:~/Maildir list: yes subscriptions: yes auth default: mechanisms: plain login username_format: %Lu passdb: driver: pam args: session=yes dovecot passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: passwd args: blocking=yes userdb: driver: static args: uid=vmail gid=vmail home=/var/vmail/%u mail=maildir:~/Inboxdir socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail

4 6

[Dovecot] dovecot 1.2 virtual folders question
by Robert Schetterer 22 Feb '09

22 Feb '09

Hi Timo is this possible a possible setup ? for downloading imap folders ( including inbox ) with pop3 controlled by a a dovecot-virtual file in usr/local/virtual/%d/%u/ with content something like that * -Trash -Trash/* -Sent -Sent/* -Drafts -Drafts/* all namespace private { separator = / prefix = "" location=maildir:/usr/local/virtual/%d/%u/:CONTROL=/usr/local/virtual/%d/%u/:INDEX=/usr/local/virtual/%d/%u/ inbox = yes subscriptions = yes list = yes } namespace private { prefix = virtual/ separator = / location = virtual:/usr/local/virtual/%d/%u:LAYOUT=maildir++ inbox = no } it doesnt work yet, but i am nearly sure that i havent understand http://wiki.dovecot.org/Plugins/Virtual perhaps you might bring me on the way background : i want to include a global sieve rule which sorts marked spam mails in a subfolder of inbox called Junk, so that users never can overide it and set perhaps an autoreply by an Sieve rule by incomming mail to inbox to make it possible that pop3 users may fetch sorted Spam mails outta the Junk folder too, i wanna use virtual setup folders Trash , Sent , Drafts are standart folders are standart and should be ignored with that -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

2 11

[Dovecot] DIGEST-MD5 user/realm mismatch with Postfix
by Darren Pilgrim 21 Feb '09

21 Feb '09

This is a follow-on to the "SQL field format for digest-md5?" thread. After some additional debugging, I've found the problem is a mismatch in what Dovecot expects and Postfix uses for the user and realm. For a username "user(a)example.com" and password "sekret", Dovecot expects: username = "user" realm = "example.com" but Postfix sends the checksum of: username = "user(a)example.com" realm = "" How do I fix this?

2 1

[Dovecot] Replication status
by Asheesh Laroia 16 Feb '09

16 Feb '09

I remember last year we discussed Dovecot replication a bit on this list, e.g. http://www.dovecot.org/list/dovecot/2008-May/030446.html . Has there been any development related to this, and can I help at all? The more I think about the hacks I'm employing to synchronize mail, the more I realize I want this. So hopefully I can put either some money or some time where my mouth is. -- Asheesh. -- Its name is Public Opinion. It is held in reverence. It settles everything. Some think it is the voice of God. -- Mark Twain

7 10

[Dovecot] unkillable imap process(es) with high CPU-usage
by Arno Wald 15 Feb '09

15 Feb '09

Hello, I am having a problem with my dovecot-daemon. It is forking one or more (I saw up to perhaps 8 of them) imap processes under my user name. These processes are consuming a lot of CPU time and are not killable: > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 8616 arno 20 0 2900 1600 1204 R 98 0.2 1196:38 imap Stopping dovecot does not quit these processes. Killing them (even "kill -9" as root) is not possible. The only solution to get rid of them is to reboot. I have found a mailing list post of one who seemed to have the same problem. He solved it by upgrading to version 1.1. But in my case this did not help. Is this a known problem? What could I check or do? My system: very up to date debian/sid (sidux) kernel 2.6.27-8.slh.1-sidux-686 CPU: Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz Filesystem: ext3fs tried dovecot from sid: 1:1.0.15-2.3 tried dovecot from experimental: 1:1.1.2-3 My dovecot.conf is the original debian configuration with only one line changed into: protocols = imaps # dovecot -n # 1.1.2: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_privileged_group: mail auth default: passdb: driver: pam userdb: driver: passwd I am using dovecot locally on my system with me as the only user. As client I am using thunderbird alias icedove 2.0.0.17-1. Icedove retrieves the Mails from another imap server and sorts them into Maildir-folders in dovecot. I do not know when the imap-process are going mad. It happens after the system (and the mail-client) is up for a while. Do you need more information? Thanks, Arno

10 35

[Dovecot] Public namespace permissions documentation/questions
by Thomas Hummel 06 Feb '09

06 Feb '09

Hello Timo, In my trials to setup a shared namespace with dovecot-1.1.8/LDAP passdb/userdb (prefetch)/Maildir, I found out that : 1) ACL are mandatory (at least if the acl plugin is triggered in dovecot.conf) Am I correct ? I'm still not sure if we can do without ACL at all (only with unix permissions and system_user userdb extra field). 2) the system_user userdb extra field is supposed to be ...the logname of the user the secondary groups of whom we want to check ! i.e. if user foobar belongs to secondary groups foogid, zgid, wgid and doveshared uid=xxx(foobar) gid=yyy(foogid) groups=zzz(zgid),www(wgid),vvv(doveshared) and we dovecot to take them into account, we have to make the usedb return the system_user extra field with the value foobar. Seems obvious now and said this way, but looking at the wiki : "system_user: If this is given, the user's groups are read from /etc/group (or wherever NSS is configured to taken them from)." I thought 'system_user' was a flag (a boolean) which, when triggered made dovecot look for the secondaries group of the user (user whose name is already known). a) am I correct ? b) why isn't system_user such a boolean ? Is there a case where we'd want system_user to be different than the user dovecot runs as at the moment the check takes place ? 3) same idea with acl_groups : since this extra_field holds a list of groups for the ACL plugin, why not rely on the native unix groups of the system the user belong to ? Thanks (and sorry for the 2 previous threads where I was blindly confused by the system_user thing). -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Pôle informatique - systèmes et réseau

2 1

[Dovecot] dovecot-shared file
by Thomas Hummel 06 Feb '09

06 Feb '09

Hello Timo, About the dovecot-shared file, the wiki says : "Unless you're using a single UID for all the users, you'll need to have a dovecot-shared file in each shared maildir. The group and the file permissions for new files inside the maildir are taken from the dovecot-shared file." but says the following too : "With Maildir dovecot-shared file controls if the \Seen flags are shared or private. If you don't have the file, they're shared." The question is "does it make sense (or is it even possible) not to have the file ?". Why would one not have the file and how would it work then ? By relying on the set-gid bit and umask ? Finally, the wiki says : "If you do have the file, the \Seen flag state is stored only in the user's index files. By making each user have their own private index files, you can make the \Seen flag private for the users." So, my undersanding is that : 1) you have the file 1.a) if INDEX points to the same place, flags are shared but are lost if indexes are lost 1.b) if INDEX points to some user dependant place, flags are private but are lost if indexes are lost 2) you don't have the file flags are shared, whatever INDEX points to and, since they're stored in filename, survive indexes lost. but does shared-mailboxes even work if we don't have that file ? maybe only if the set-gid bit and umask does what the file would have done ? So maybe, if you want to share flags, 2) is safer dans if you want them private, 1.b) is the only choice ? thanks -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Pôle informatique - systèmes et réseau

2 1

[Dovecot] Shared subscription, acl-list and uidvalidity(s)
by Thomas Hummel 06 Feb '09

06 Feb '09

Hello, I'm running dovecot-1.1.8/Maildir/ACL plugin. I sucessfully set up a Maildir shared between users of the unix group 'doveshared' via a public namespace, unix permissions and ACL files. The location of my public namespace is /path/to/public. I tried 2 sub-setups : First setup ---------- drwxrws--- 4 root doveshared 4096 Jan 30 13:39 public -rw-r----- 1 root doveshared 18 Jan 30 13:38 public/subscriptions I first connect (in Thunderbird) to dovecot with user 'hummel', then with user 'doveimap'. The following files are created in /path/to/public/ : -rw------- 1 doveimap doveshared 40 Jan 30 14:03 dovecot-acl-list -rw------- 1 hummel doveshared 8 Jan 30 14:03 dovecot-uidvalidity -rw------- 1 hummel doveshared 0 Jan 30 14:02 dovecot-uidvalidity.4982fa7a I've got my own idea about what they're used for, but can you explain what they are supposed to be ? I guess the uidvalidity files are UIDVALIDITY, not for the mailboxes the namespace holds (since my understanding is that the uidvalidities of mailboxes are stored in the first line of dovecot-uidlist files), but for the namespace itself (in case it is destroyed and then recreated for instance). Am I right ? Why 2 files ? Second setup ------------- I follow the 'Shared subscriptions' section of the wiki : drw-r-s--- 4 root doveshared 4096 Jan 30 14:19 public -rw-r----- 1 root doveshared 18 Jan 30 13:38 public/subscriptions As expected, dovecot (and it shows in the logs), has not sufficient permissions to create the files above (dovecot-acl-list, dovecot-uidvalidity, dovecot-uidvalidity.4982fa7a). What are the consequences ? Thanks. -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Pôle informatique - systèmes et réseau

2 3