- dovecot - dovecot.org

[Dovecot] Outlook causes BIG dovecot index problem (beta 8, outlook 2003)
by Casey Allen Shobe 01 Jun '06

01 Jun '06

Hey all, Well, I typed up all the below, then managed to fix the problem before sending this to the list. I'll send it anyways, in case anybody else is affected by it (and because it does indicate a definite bug in Dovecot's experimental Kerberos support). The fix is to recompile without kerberos enabled, and to take gssapi out of the auth methods in the config (the latter alone may or may not be enough of a fix on it's own, I didn't try it without recompiling). I'm guessing that Outlook attempts to use Kerberos authentication by default or something and silently fails and uses other auth methods, and that any attempt to use Kerberos is in fact what hoses the server. However I have not done any further testing against beta8 with a Kerberos-enabled client such as KMail, so I leave this as an exercise for the next adventurer. I'm also uncertain as to why this problem does not manifest on our main servers, which also have Kerberos enabled (and Outlook 2003 clients). Or /had/ anyways, as I'm disabling Kerberos support as we speak as a preventative measure. Here's the problem description: I've been running dovecot happily for many months on our production mail servers hosting about 600 accounts, with no problems. People use every mail client under the sun - Apple Mail, Outlook 97|2000|XP|2003, Outlook Express, Mutt, Thunderbird, Mozilla Mail, Eudora, and other esoteric things with no problem. So today I installed a new mail server on a client site dedicated to their domain. The MTA is exim, running with almost the same configuration we use on our main servers. The dovecot.conf is 100% identical. No mail was moved by hand to the new mail server - to migrate we copied the mail setting up the old and new accounts in Thunderbird and dragging things over (just to rule out that possibility, we have also tested using brand new empty accounts to reproduce the issue). Everything is golden using Thunderbird - email is send, and shows up in the inbox immediately. Enter Outlook 2003 (SP1): It shows all folders and mail fine. It sends mail fine. Nothing shows up in the inbox. At this moment, Thunderbird also stops receiving new mail in the inbox. I check the server, and there's a mail sitting in the /new directory in the maildir. Not only that, but suddenly NONE of the accounts on the box show any new mail, exhibiting the same signs. I tried a variety of things, including 'sync', removing all the dovecot* index files in the maildir, and restarting dovecot (being sure to killall imap before bringing it back up). Sometimes, these steps would magically cause everything to work again, until I did a mail poll in Outlook which causes the problem to re-occur. Other times, these steps didn't seem to do anything. A couple times, for no apparent reason, without doing anything, mail would show up. Too hopeful, we sat around for 30 minutes sending more mail to an account and then just idling...nothing showed up. The hardware is good, we've compiled a full glibc and gcc just to ensure that there are no CPU/memory issues, and this box has a solid good history behind it in it's prior uses. Until Outlook is used, everything works fine. We do have the outlook-idle workaround enabled, and tried both with and without delay-newmail, which didn't make any difference. Cheers, -- Casey Allen Shobe | cshobe(a)seattleserver.com | 206-381-2800 SeattleServer.com, Inc. | http://www.seattleserver.com

3 3

[Dovecot] mbx to mbox conversion
by Marc Perkel 01 Jun '06

01 Jun '06

Here's a feature I'd like. If Dovecot opened an MBX format file that it would convert it to an MBOX format file. I have several files that I'm having a hard time converting. The one I got from Washington University WUIMAP is extremely flawed.

3 2

[Dovecot] Cannot remove Maildir folder
by Marcin Deranek 01 Jun '06

01 Jun '06

Hi, I have been using dovecot 1.0 beta 8 on Debian Sarge for a couple of days and I experienced some problems when removing folders.. As an IMAP client I use Thunderbird 1.5 and IMAP folders are on NFS. Here are the steps to re-produce the problem: 1. Run Thunderbird as usual 2. Create folder 3. Remove folder - In my case folder is not removed but moved to Trash (Thunderbird setting) 4. I go to Trash and remove it from there 5. Folder is removed but I get additional ..DOVECOT-TRASHED directory - next time it won't be possible to remove it Some more info: * Just before removal I run lsof to see which files are in use: bash# lsof /home/marcin ... imap 21360 marcin cwd DIR 0,15 344064 3307008 /home/marcin (filer:/vol/vol0/home/marcin) imap 21360 marcin 8u DIR 0,15 4096 4223651 /home/marcin/Maildir/.Trash.test/new (filer:/vol/vol0/home/marcin) imap 21360 marcin 9u DIR 0,15 4096 4223650 /home/marcin/Maildir/.Trash.test/cur (filer:/vol/vol0/home/marcin) imap 21360 marcin 11r REG 0,15 144 4223663 /home/marcin/Maildir/.Trash.test/dovecot.index (filer:/vol/vol0/home/marcin) imap 21360 marcin 12r REG 0,15 104 4223647 /home/marcin/Maildir/.Trash.test/dovecot.index.log (filer:/vol/vol0/home/marcin) imap 21360 marcin 13u REG 0,15 10272 4223657 /home/marcin/Maildir/.Trash.test/dovecot.index.cache (filer:/vol/vol0/home/marcin) ... There are more files in use but in this case I'm interested with PID 21360 which has opened Trash/test (the one which I'm going to remove) * Just after removal I see error message in Syslog: dovecot: IMAP(marcin): unlink_directory(/home/marcin/Maildir/..DOVECOT-TRASHED) failed: Device or resource busy Running lsof again gives the following: imap 21360 marcin cwd DIR 0,15 344064 3307008 /home/marcin (filer:/vol/vol0/home/marcin) imap 21360 marcin 11r REG 0,15 144 4223663 /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs004072af0002cb1b (filer:/vol/vol0/home/marcin) imap 21360 marcin 12r REG 0,15 104 4223647 /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs0040729f0002cb1a (filer:/vol/vol0/home/marcin) imap 21360 marcin 13u REG 0,15 10272 4223657 /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs004072a90002cb1c (filer:/vol/vol0/home/marcin) Looking into some NFS docs we can read: "A client has a file opened. Another process on that client deletes the file. So the client NFS stack rather than deleting the file and potentially hurting the other process, it renames the file to .nfs<file_handle>." To me it looks like one imap process tries to delete .Trash.test directory (first rename and then unlink) whereas other imap process has still opened dovecot.* files. Any workaround/fix for that ? Regards, Marcin Deranek

2 5

[Dovecot] beta8 starts but no POP3 access
by Obantec Support 31 May '06

31 May '06

Hi Timo i see you just answered a posting less than 1 hour old! any chance you can comment on mine (29/05/06) repeated below and very urgent i get dovecot beta8 up. I just compiled stock beta8 on FC3 with ./configure make make install all looked ok and i changed my dovecot.conf to suit. start dovecot and i see it running May 29 08:18:00 Info: Dovecot v1.0.beta8 starting up POP3 to 1 account and May 29 08:18:06 Info: auth(default): client in: AUTH 1 PLAIN service=POP3 lip=xx.xx.xx.xx rip=yy.yy.yy.yy resp=<hidden> May 29 08:18:06 Info: auth(default): client out: OK 1 user=obantec_com May 29 08:18:06 Info: auth(default): master in: REQUEST 1 19528 1 May 29 08:18:06 Info: auth(default): master out: USER 1 obantec_com system_user=obantec_com uid=501 gid=10001 home=/home/domain/domain17 May 29 08:18:06 Info: pop3-login: Login: user=<obantec_com>, method=PLAIN, rip=yy.yy.yy.yy, lip=xx.xx.xx.xx May 29 08:18:06 proteus2 dovecot: POP3(obantec_com): Corrupted index cache file /home/domain/domain17/mail/.imap/INBOX/dovecot.index.cache: indexid changed May 29 08:18:06 proteus2 dovecot: child 19536 (pop3) killed with signal 11 yy.yy.yy.yy & xx.xx.xx.xx are the ip's just masked out. OE just immediately say error , what is also worrying is the Corrupted index message even though i am using POP3. any ideas what to look for. I might add a test box running FC3 and the same .conf works! Mark -- Obantec Support www.obantec.net 0845 458 3121 WebHosting and Domains Nominet UK Member & IPStag Holder CentralNic Accredited Reseller

2 2

[Dovecot] dovecot.index.cache
by Fintec 31 May '06

31 May '06

Hopefully someone can explain/help me with the dovecot.index.cache file. I am wanting to move a dovecot server (v1.0b7) onto a new box running the same version (1.0b7) of dovecot. To do this I rsync'd the maildir files to the new server but when I logged into the new machine through IMAP dovecot reported: IMAP(username): file index-mail.c: line 105 (index_mail_get_fixed_field): assertion failed: (buffer_get_used_size(buf) == data_size) As soon as I delete the dovecot.index.cache file within the users mailbox this error goes away. Great, but that now I have some questions: 1) Does the dovecot.index.cache file store/cache server specific settings? If not why did it cause this error on the new box with exactly the same dovecot version and dovecot.conf? 2) What will happen if I delete the dovecot.index.cache file for all users? Will POP3 users have to re-download all their mail or will dovecot simply reference the dovecot.index file in the absence of the cache file? If I've missed something on the wiki explaining all this please point me in the right direction. Any help would be greatly appreciated. Gavin

3 4

[Dovecot] Apple Mail and too many open files?
by Alan Schmitt 31 May '06

31 May '06

Hello, For two days in a row, I've noticed that my dovecot server (hosted on a PowerMac running OS X) has stopped working, with a bunch of error messages in /var/log/mail.log like: May 24 07:31:10 top dovecot: pipe() failed: Too many open files To solve this, I have to kill -9 dovecot and the (many) imap processes running. The only client accessing the server is a machine running Apple Mail. I tried to increase the limit in the number of files (I did a "launchctl limit maxfiles 1024 unlimited", but the problem still happened. Is this a known problem? Should I allow still more files? Thanks, Alan -- Alan Schmitt <http://alan.petitepomme.net/> The hacker: someone who figured things out and made something cool happen. .O. ..O OOO

4 5

[Dovecot] problems with Debian Sarge
by Andrea Riela 31 May '06

31 May '06

Hi folks, I've updated my dovecot 0.99.14 with the last package (0.99.14-1sarge0), but it doesn't work. I've found that problem: http://comments.gmane.org/gmane.linux.debian.devel.bugs.general/80872 then I've removed that, and reinstalled first package (from /var/cache/apt/archives), and NOW it doesnt' work too :( When I start the daemon, I see in my logs: dovecot: May 31 12:40:28 Info: Dovecot starting up dovecot-auth: May 31 12:40:29 Fatal: Unknown userdb type 'vpopmail' dovecot: May 31 12:40:29 Error: Auth process died too early - shutting down dovecot: May 31 12:40:29 Error: child 15798 (auth) returned error 89 imap-login: May 31 12:40:29 Fatal: fd_send(-1) failed: No such file or directory imap-login: May 31 12:40:29 Fatal: fd_send(-1) failed: No such file or directory I don't understand. Any advice will be appreciated Best Regards Andrea my dovecot.conf is like that: ## Dovecot 1.0 configuration file <...> # Protocols we want to be serving: # imap imaps pop3 pop3s protocols = imaps <...> # IP or host address where to listen in for SSL connections. Defaults # to above non-SSL equilevants if not specified. imaps_listen = 192.168.17.10:993 #pop3s_listen = # Disable SSL/TLS support. ssl_disable = no # PEM encoded X.509 SSL/TLS certificate and private key. They're opened before # dropping root privileges, so keep the key file unreadable by anyone but # root. #ssl_cert_file = /etc/ssl/certs/dovecot.pem #ssl_key_file = /etc/ssl/private/dovecot.pem ssl_cert_file = /var/qmail/control/servercert.pem ssl_key_file = /var/qmail/control/servercert.pem <...> # Use this logfile instead of syslog(). /dev/stderr can be used if you want to # use stderr for logging (ONLY /dev/stderr - otherwise it is closed). log_path = /var/log/dovecot.log # For informational messages, use this logfile instead of the default info_log_path = /var/log/dovecot.log <...> ## ## IMAP login process ## login = imap # Executable location. #login_executable = /usr/lib/dovecot/imap-login # User to use for the login process. Create a completely new user for this, # and don't use it anywhere else. The user must also belong to a group where # only it has access, it's used to control access for authentication process. # Note that this user is NOT used to access mails. # http://wiki.dovecot.org/UserIds login_user = dovecot <...> ## ## Mail processes ## # Maximum number of running mail processes. When this limit is reached, # new users aren't allowed to log in. #max_mail_processes = 1024 # Show more verbose process titles (in ps). Currently shows user name and # IP address. Useful for seeing who are actually using the IMAP processes # (eg. shared mailboxes or if same uid is used for multiple accounts). verbose_proctitle = no # Show protocol level SSL errors. #verbose_ssl = no # Valid UID range for users, defaults to 500 and above. This is mostly # to make sure that users can't log in as daemons or other system users. # Note that denying root logins is hardcoded to dovecot binary and can't # be done even if first_valid_uid is set to 0. first_valid_uid = 89 last_valid_uid = 89 # Valid GID range for users, defaults to non-root/wheel. Users having # non-valid GID as primary group ID aren't allowed to log in. If user # belongs to supplementary groups with non-valid GIDs, those groups are # not set. first_valid_gid = 0 #last_valid_gid = 0 <...> # Default MAIL environment to use when it's not set. By leaving this empty # dovecot tries to do some automatic detection as described in # /usr/share/doc/dovecot-common/mail-storages.txt. There's a few special # variables you can use: # # %u - username # %n - user part in user@domain, same as %u if there's no domain # %d - domain part in user@domain, empty if user there's no domain # %h - home directory # # You can also limit a width of string by giving the number of max. characters # after the '%' character. For example %1u gives the first character of # username. Some examples: # # default_mail_env = maildir:/var/mail/%1u/%u/Maildir # default_mail_env = mbox:~/mail/:INBOX=/var/mail/%u # default_mail_env = mbox:/var/mail/%d/%n/:INDEX=/var/indexes/%d/%n # default_mail_env = maildir:~/Maildir <...> # Workarounds for various client bugs: # oe6-fetch-no-newmail: # Never send EXISTS/RECENT when replying to FETCH command. Outlook Express # seems to think they are FETCH replies and gives user "Message no longer # in server" error. Note that OE6 still breaks even with this workaround # if synchronization is set to "Headers Only". # outlook-idle: # Outlook and Outlook Express never abort IDLE command, so if no mail # arrives in half a hour, Dovecot closes the connection. This is still # fine, except Outlook doesn't connect back so you don't see if new mail # arrives. # outlook-pop3-no-nuls: # Outlook and Outlook Express hang if mails contain NUL characters. # This setting replaces them with 0x80 character. client_workarounds = oe6-fetch-no-newmail outlook-idle outlook-pop3-no-nuls <...> # Copy mail to another folders using hard links. This is much faster than # actually copying the file. This is problematic only if something modifies # the mail in one folder but doesn't want it modified in the others. I don't # know any MUA which would modify mail files directly. IMAP protocol also # requires that the mails don't change, so it would be problematic in any case. # If you care about performance, enable it. maildir_copy_with_hardlinks = yes <...> ## ## Authentication processes ## # An Authentication process is a child process used by Dovecot that # handles the authentication steps. The steps cover an authentication # mechanism (auth_mechanisms, how the client authenticates in the IMAP or # POP3 protocol), which password database should be queried (auth_passdb), # and which user database should be queried (auth_userdb, to obtain # UID, GID, and location of the user's mailbox/home directory). # # You can have multiple processes, though a typical configuration will # have only one. Each time "auth = xx" is seen, a new process # definition is started. The point of multiple processes is to be able # to set stricter permissions. (See auth_user below.) # # Just remember that only one Authentication process is asked for the # password, so you can't have different passwords accessible through # different process definitions (unless they have different # auth_mechanisms, and you're ok with having different password for # each mechanisms). # Authentication process name. auth = default # Specifies how the client authenticates in the IMAP protocol. # Space separated list of permitted authentication mechanisms: # anonymous plain digest-md5 cram-md5 # # anonymous - No authentication required. # plain - The password is sent as plain text. All IMAP/POP3 clients # support this, and the password can be encrypted by Dovecot to match # any of the encryption schemes used in password databases. # digest-md5 and cram-md5 - both encrypt the password so it is more # secure in transit, but are not well supported by clients, and # require that the password database use a matching encryption # scheme (or be in plaintext). # # See auth.txt for more details. # # If you are using SSL there is less benefit to digest-md5 and # cram-md5 as the communication is already encrypted. auth_mechanisms = plain # Space separated list of realms for SASL authentication mechanisms that need # them. You can leave it empty if you don't want to support multiple realms. # Many clients simply use the first one listed here, so keep the default realm # first. #auth_realms = # Default realm/domain to use if none was specified. This is used for both # SASL realms and appending @domain to username in plaintext logins. #auth_default_realm = # User database specifies where mails are located and what user/group IDs # own them. For single-UID configuration use "static". # http://wiki.dovecot.org/Authentication # http://wiki.dovecot.org/VirtualUsers # passwd: /etc/passwd or similiar, using getpwnam() # passwd-file <path>: passwd-like file with specified location # static uid=<uid> gid=<gid> home=<dir template>: static settings # vpopmail: vpopmail library # ldap <config path>: LDAP, see /etc/dovecot/dovecot-ldap.conf # mysql <config path>: a MySQL database, see /etc/dovecot/dovecot-mysql.conf # pgsql <config path>: a PostgreSQL database, see # /etc/dovecot/dovecot-pgsql.conf auth_userdb = vpopmail # Password database specifies only the passwords for users. # http://wiki.dovecot.org/Authentication # passwd: /etc/passwd or similiar, using getpwnam() # shadow: /etc/shadow or similiar, using getspnam() # pam [<service> | *]: PAM authentication # passwd-file <path>: passwd-like file with specified location # vpopmail: vpopmail authentication # ldap <config path>: LDAP, see /etc/dovecot/dovecot-ldap.conf # mysql <config path>: a MySQL database, see /etc/dovecot/dovecot-mysql.conf # pgsql <config path>: a PostgreSQL database, see # /etc/dovecot/dovecot-pgsql.conf auth_passdb = vpopmail #auth_executable = /usr/lib/dovecot/dovecot-auth # Set max. process size in megabytes. #auth_process_size = 256 # User to use for the process. This user needs access to only user and # password databases, nothing else. Only shadow and pam authentication # requires roots, so use something else if possible. Note that passwd # authentication with BSDs internally accesses shadow files, which also # requires roots. Note that this user is NOT used to access mails. # That user is specified by auth_userdb above. auth_user = root <...>

3 2

[Dovecot] about init.d script
by Andrea Riela 31 May '06

31 May '06

Hi folks, someone using Debian? I need the init script, if you please :) Thanks for your support Regards Andrea

3 2

[Dovecot] How to move a Maildir directory to a new server?
by Darrin Smart 31 May '06

31 May '06

Hi All, I'd like to ask about the best practice for moving a user's Maildir from one host to another. I moved a few user's home directories, and therefore their ~/Maildir folders. They have complained that since the move, they don't see all the messages in their INBOX for a while, but eventually they all appear again. They are using Thunderbird. I tried deleting the index files but it didn't help. Can anyone give advice? I have a soft link pointing from the old home directory to the new one, and both paths were on NFS. I can't see how dovecot could give different results, could it be related to message UID numbers? I did notice that when I moved my own home directory, Mail.app re-downloaded every mail when I opened each folder. Thunderbird doesn't seem to do this, perhaps its related. Thanks. Darrin

1 0

[Dovecot] public folders aren't public
by Mark Nienberg 30 May '06

30 May '06

I'm testing dovecot as a replacement for an aging UW-IMAP server. I have the basic IMAP setup working but I'm stuck on public folders (which my users are addicted to). I set up public folders generally a described in the wiki. The problem is that when a user creates a new public folder, the folder is owned by the user with permissions drwx------ which prevents other users from viewing the contents. Here is part of dovecot.conf: namespace public { separator = / prefix = Public/ location = maildir:/home/public/Maildir } and here is the Maildir directory [root@tesla Maildir]# ls -la total 70k drwxrwxrwx 7 dovecot Everyone 4.1k May 12 14:33 ./ drwxrwxrwx 3 root root 4.1k May 12 13:43 ../ drwxrwS--- 2 dovecot Everyone 4.1k May 12 13:48 cur/ drwx------ 5 david Everyone 4.1k May 12 14:33 .davi/ -rw-rw-r-- 1 dovecot Everyone 0 May 12 13:46 dovecot-shared drwxrwS--- 2 dovecot Everyone 4.1k May 12 13:48 new/ -rw------- 1 mark Everyone 6 May 12 14:07 subscriptions drwxrwS--- 2 dovecot Everyone 4.1k May 12 13:48 tmp/ drwx------ 5 mark Everyone 4.1k May 12 14:30 .tryit/ The directories .davi/ and .tryit/ were created by users david and mark respectively. These are real system users. They are both members of the Everyone group. Any suggestions appreciated. Thanks, Mark

2 3