- dovecot - dovecot.org

[Dovecot] No auto-conversion of .customflags?
by Christian Balzer 13 Jun '07

13 Jun '07

Hello, I'm in the process of migrating 0.99 dovecot maildir boxes to a pair of new machine. These servers were running the stable (etch) 1.0rc15 package of dovecot until yesterday and existing .customflags files were correctly converted to dovecot-keywords. Yesterday I upgraded to the (finally, thanks to who did this) 1.0.0 etch backports package and since then freshly migrated boxes get this error: --- Jun 12 09:48:06 mb11 dovecot: IMAP(username): stat(/longandboringpath/username/.customflags/cur) failed: Not a directory --- Firstly, I thought Linux was one of the OS' that had a free dir stat, but I guess I could have just hallucinated that. But more importantly, obviously .customflags does NOT get converted anymore and thus shows up in IMAP LIST commands. Since I doubt that the backport people did change anything in the code I suppose that this is a genuine 1.0.0 problem which probably should be addressed for the upcoming 1.0.1 release. Regards, Christian -- Christian Balzer Network/Systems Engineer NOC chibi(a)gol.com Global OnLine Japan/Fusion Network Services http://www.gol.com/

2 3

[Dovecot] 1.0.1 release candidate 2
by Timo Sirainen 13 Jun '07

13 Jun '07

I'll release v1.0.1 in a day or two, but it would be nice to get some testing before the release to see if I managed to break something after the Mercurial conversion. And did I forget something that should be fixed for v1.0.1? http://dovecot.org/tmp/dovecot-1.0.1rc2.tar.gz Changes since v1.0.0: * deliver: If Return-Path doesn't contain user and domain, don't try to bounce the mail (this is how it was supposed to work earlier too) * deliver: %variables in mail setting coming from userdb aren't expanded anymore (again how it should have worked). The expansion could have caused problems if paths contained any '%' characters. + Print Dovecot version number with dovecot -n and -a - Trying to start Dovecot while it's already running doesn't anymore wipe out login_dir and break the running Dovecot. - maildir: Fixed "UID larger than next_uid" errors which happened sometimes when dovecot-uidlist file didn't exist but index files did (usually because mailbox didn't have any messages when it was selected for the first time) - maildir: We violated maildir spec a bit by not having keyword characters sorted in the filename. - maildir: If we don't have write access to cur/ directory, treat the mailbox as read-only. This fixes some internal error problems with trying to use read-only maildirs. - maildir: Deleting a symlinked maildir failed with internal error. - mbox: pop3_uidl_format=%m wasn't working right - mbox: If non-filesystem quota was enabled, we could have failed with "Unexpectedly lost From-line" errors while saving new messages - mysql auth: %c didn't work. Patch by Andrey Panin - COPY: We sent "Hang in there.." too early sometimes and checked it too often (didn't break anything, but was slower than needed). - deliver: Postfix's sendmail binary wasn't working with mail_debug=yes - Don't corrupt ssl-parameters.dat files when running multiple Dovecot instances. - Cache compression caused dovecot.index.cache to be completely deleted with big endian CPUs if 64bit file offsets were used (default) - Fixed "(index_mail_parse_header): assertion failed" crash

5 12

[Dovecot] Help with new syslog errmsgs
by Stewart Dean 12 Jun '07

12 Jun '07

I am bringing more people onto Dovecot...and am beginning to see syslog error messages I've never seen before...what are they/are they critical? 1 =========================== > Jun 12 09:53:21 mercury mail:err|error dovecot: IMAP(gregory): Corrupted index c > ache file /var/dcndx/gregory/.imap/INBOX/dovecot.index.cache: indexid changed This has the same date stamp as the user's change date on his .subscriptions...which he created by copying his UWIMAP .mailboxlist from a login prompt. Is this what happens when the user with active sessions overwrites/creates the .subscriptions file? 2 =========================== > Jun 12 11:32:36 mercury mail:err|error dovecot: IMAP(gregory): UIDVALIDITY chang > ed (1100532544 -> 1178155834) in mbox file /home/hcrc/gregory/mail/Sent Have no idea what this is about...... -- ==== Stewart Dean, Unix System Admin, Henderson Computer Resources Center of Bard College, Annandale-on-Hudson, New York 12504 sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035

2 2

[Dovecot] Phantom message using mixed namespaces
by Ian R. Justman 12 Jun '07

12 Jun '07

Hi, all. At home, I have a mailbox where mbox is the default (for inbox and some smaller mailboxes) and all my lists are in a Maildir format. I'm building a mailstore server at work where I want to use Maildir as the default mailstore, but I want to keep mbox on the side to aid in archiving messages. My configs are as follows for mail location and namespaces: Home: mail_location = mbox:~/mail:INBOX=/var/mail/%u namespace private { separator = / inbox = yes hidden = no prefix = "mbox/" location = mbox:~/mail:INBOX=/var/mail/%u } namespace private { separator = / prefix = "Maildir/" location = maildir:~/Maildir } Work: mail_location = maildir:~/Maildir namespace private { separator = / inbox = yes prefix = "Maildir/" location = maildir:~/Maildir } namespace private { separator = / hidden = no prefix = "mbox/" location = mbox:~/mail } Here's the thing: My home mailbox server looks the way it should, offering me "mbox" and "Maildir" names to expand for more mailboxes. Work on the other hand, by and large, works as well (in fact, I based its config on the home machine's config with some slight changes). However, strange things started happening the moment I did a "mkdir ~/mail" while I was at the machine's prompt to make ready for an mbox repository. My mail program nearly immediately shows a "phantom message" in the main inbox. If I try to select this "phantom message", the mailer goes into a very tight loop (I've tried this with Seamonkey and T'bird with the same results. I get cycles of these in the log: Jun 12 11:29:14 (censored) dovecot: imap-login: Login: user=<xxxx>, method=PLAIN, rip=x.x.x.x, lip=y.y.y.y Jun 12 11:29:14 (censored) dovecot: IMAP(xxx): FETCH for mailbox INBOX UID 3 got too little data: 0 vs 512 Jun 12 11:29:14 (censored) dovecot: IMAP(xxxx): Disconnected: Disconnected I'm running the Dovecot 1.0.0 under FreeBSD 4.11 (work) (I can't run anything later on this machine, strangely enough; I haven't tried any of the later releases since updating the BIOS recently, however) and 1.0.rc17 under Ubuntu Feisty (home). If I need to provide more info, please let me know. I'd also be interested to know if it's something dumb I'm doing or if it perhaps may be a bug. Thanks! --Ian.

2 2

[Dovecot] New experimental XEXEC plugin status
by Marc Perkel 12 Jun '07

12 Jun '07

No to long ago I proposed an experimental extension to the IMAP protocol to allow IMAP to execute extername server side code and talk to email plugins to create custom client/server applications over an existing authenticated IMAP connection. I am happy to say they we have working code. It's not quite ready to download yet but we do have some life happening. I have started to document it on the Dovecot wiki here. http://wiki.dovecot.org/Plugins/xexec I'm looking to see if I can generate some interest in this. XEXEC Plugin XEXEC is an experimental non-standard extension to the IMAP protocol that allows you to run and server side application and have it communicate with email plugins (Thunderbird) allowing custom client/server applications to communicate over an existing authenticated IMAP channel. The email client sends the XEXEC command with the name of the server side module that it wishes to run along with parameters to pass to the server. Here are some examples: XEXEC blacklist add joe(a)example.com XEXEC whitelist del joe(a)example.com XEXEC changepassword MyNewPassword XEXEC forwardmail myaccount(a)yahoo.com Uses for XEXEC The XEXEC can be used to run anything but in theory it should be used for client server applications that relate to email or groupware applications like calendars and contact lists. Its main purpose is to allow email clients to set server side settings over the existing authenticated channel without having the user have to log in separately using a different application to control user settings. Examples of possible usage includes: * Setting server side white lists and black lists * Changing the password for the email account * Allowing users to set email forwarding preferences * Allowing users to set vacation messages * Allowing users to exclude email from specific countries or language character sets * Allowing users to control server side spam filter settings * Server Side Calendar and Group Contact Applications like MS Exchange has * SMTP over IMAP for sending outgoing email without separate SMTP settings * Setting rules for server side folder delivery. * Setting text message alerts to cell phones for important email notification Wouldn't it be nice if you could change your email password i your email client and have it set the new password on the server at the same time? This would eliminate the need to change the server and client passwords separately. Or imagine sending outgoing email without having to configure SMTP. You just set up the incoming email and the outgoing email works as well. Or wouldn't you like to do some of the fancy tricks that Microsoft Exchange Servers do without having to use Exchange? It is our hope that this experimental protocol will lead to significant expansion of IMAP functionality. About this Plugin This plugin is the idea of Marc Perkel of Junk Email Filter <http://www.junkemailfilter.com> who hired Nicolas Boullis ( nicolas.boullis(a)free.fr <mailto:nicolas.boullis@free.fr>) to write the code. I (Marc Perkel marc(a)perkel.com <mailto:marc@perkel.com>) am trying to get the concept out there in some sort of working form with the idea that it will catch on. Hopefully other people who find this concept useful will build on the idea, do it right, and perhaps some variation of this concept will eventually become a standard. Status This plugin is a nonstandard experimental extension to the IMAP protocol. It is offered as a proof of concept and it is my hope that it will catch on in some form and make it into the IMAP officially. At this point there is nothing on the client side to talk to anything on the server side. So this code is presently useless until more development is done. Configuration To activate and configure this plugin you have to first add it to your dovecot.conf file. You can do so as follows: protocol imap { mail_plugins = xexec } Then to add applications you specify the module name and map it to a server side executable program. xexec = module:executable Here are some examples: plugin { xexec = blacklist:/usr/local/dovecot/blacklist.sh %u xexec2 = whitelist:/usr/local/dovecot/whitelist.sh %u xexec3 = average:/usr/local/dovecot/average xexec4 = smtp:/usr/local/dovecot/smtp } The idea is that the module name will be a reference to an executable and will hide what is happening on the server. When the command XEXEC module <parameters> is passed then the server will run the executable and pass the parameters to the executable. Interacting with XEXEC A typical session with XEXEC might look like this. * OK Dovecot ready. 001 login marc(a)perkel.com password 001 OK Logged in. 002 xexec blacklist add joe(a)example.com * OK Adding joe(a)example.com to server side blacklist for marc(a)perkel.com 002 OK command exited successfully 003 xexec changepassword MyNewPassword * OK New Password set Sucessfully for marc(a)perkel.com 003 OK command exited successfully Many of the details of how everything will work under different conditions have yet to be developed. Download and Installation Coming Soon! License This software and the concepts behind is are free. No patent or copyrights are asserted on the concept. This is a gift to the world community. Code is released under GPL2. Help Needed We could use some help with this project. We haven't even started writing any client side code. We are looking for sharp people with good ideas to help develop this concept in a way that will eventually lead to becoming part of the standard. We envision that certian commands to do ordinary universal stuff (password, smpt, whit list, black list, forwarding) will become standardized so that features to set server side settings can become part of standard email applications.

1 0

[Dovecot] Warning about NFS but using local fs for INDEX
by Herbert Gasiorowski 12 Jun '07

12 Jun '07

I am playing around with dovecot and want to use local filesystem for indexing. But whenever I open a new imap connection the following warning is shown: Apr 30 10:14:42 beira dovecot: Mailbox indexes in /app/home/gasi are in NFS mount. You must set mmap_disable=yes to avoid index corruptions. If you're sure this check was wrong, set nfs_check=no. /app/home is a NFS mounted filesystem, but I want dovecot to use /export/dovecot/gasi for indexing (See namespaces below). I shurely can follow the hints(nfs_check=no), but I am not shure if I did something wrong or maybe someday will change the configuration and forget to enable the nfs check. dovecot version: dovecot-1.0-1.2.rc15.el5 on CentOS 5.0 (with all updates). [root@beira etc]# dovecot -n # /etc/dovecot.conf syslog_facility: local6 login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 namespace: type: private separator: / location: mbox:~/Mail:INBOX=/var/spool/mail/%u:INDEX=/export/dovecot/%u inbox: yes namespace: type: private separator: / prefix: #home-tmp/ location: mbox:~:INDEX=/export/dovecot/%u auth default: passdb: driver: pam args: imap userdb: driver: passwd

2 2

[Dovecot] error in logfile
by Luigi Rosa 12 Jun '07

12 Jun '07

With Dovecot 1.0.0: May 11 12:16:35 mail dovecot: IMAP(xxx): stat(/var/spool/mail2/xxx/.Trash.Squid-it/new) failed: No such file or directory With Thunderbird 2.0 IMAP client: * user deleted the mailbox * TB moved the mailbox to Trash * user asked TB to empty Trash * the above message appeared No warning/error was reported bt TB to user. I don't think it's an error and should not be logged. Ciao, luigi -- / +--[Luigi Rosa]-- \ You can tune a radio, but you can't tuna fish.

2 1

[Dovecot] Getting rid of the common newbie problems
by Timo Sirainen 12 Jun '07

12 Jun '07

I think the most common Dovecot newbie problems are: - Not looking at the (correct) logs for errors - Authentication problems - Mail location problems - Mail permission problems Below are some of my ideas how I could stop people from asking these questions with future Dovecot (v1.1+) releases. Suggestions welcome. Distribution people especially should say if they're against some change. * Logging The log file problem is the most annoying one, because a lot of the other problems can be solved once the admin figures out that Dovecot is actually giving useful error messages. Often the admin is only looking at the log file where "info" messages go (eg mail.log) because Dovecot logs its startup message and login messages there, but not where the errors go (eg mail.err). This has happened even with people who in general are experienced sysadmins. Logging to eg. /var/log/dovecot.log by default would be helpful here, but it's probably better to log via syslog by default. Cyrus logs to "local6" by default. Perhaps for Dovecot v1.1 I should make that the default too? Looks like in my Debian system the info messages then go to both /var/log/messages and /var/log/syslog. Error messages only go to /var/log/syslog. So there still isn't by default a single log file where the errors are logged, but it might help a bit. * Authentication Authentication problems can usually be solved by telling the user to set auth_debug_passwords=yes and looking at the logs. I'm not sure if there's anything that can be helped in here. Except the logging message could be updated a bit: "Aborted login: user=<asdga>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications" So the last "n failed authentications" could be added, where n could also be 0. Another possibility would be to make Dovecot remember if there have been any successful logins (/var/lib/dovecot/success file) and if not, give a bit more helpful error messages: - Client gets: "NO Authentication failed. Refer to server log for more information." instead of the normal "NO Authentication failed." - Log contains: "Aborted login: user=<asdga>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications (set auth_debug_passwords=yes to debug the problem)" I'm not sure if this is a good idea. * Mail location It seems to be difficult for some people to set mail_location. I don't know if anything can be done before Dovecot v2.0 where I'll split it to multiple settings, such as: driver = maildir root_dir = ~/Maildir index_dir = /var/indexes/%u Another problem that seems to be difficult to understand is why the mail directory contains "HOME_DIRECTORY_USED_BUT_NOT_GIVEN_BY_USERDB". The whole userdb concept appears to be weird. This could anyway be fixed by giving an error message earlier and failing the login with internal error. Probably the best place to give the error message would be already in the userdb lookup in dovecot-auth, but that would require that doveoct-auth knows if the home directory is really needed, and to give a useful error message it would also need to tell where it's tried to be used (mail_location, or some namespace's location, or ..). Probably too much trouble to be worth it. So the next best thing is to give the error when it's used: "Home directory is used in mail_location, but userdb didn't return a home directory" It would be nice if it didn't say userdb, but rather the userdb's name. I guess that would be possible if dovecot-auth told master (or deliver) which userdb was used, but that would normally be just extra overhead. * Mail permissions If mail location is difficult for some, then the concept of UIDs are how they're used in Dovecot is pretty much impossible for some to understand. One of the problems is that there exists "dovecot" user. So people think that their mails should be owned by the dovecot user. Although I've mentioned in everywhere I can think of that this should not be done, it won't help because either people won't read the pages or even if they do, they somehow still fail to ignore it even though it's written in bold. So unless people (and most importantly, distributions) are against it, I think the "dovecot" user should be renamed. "dovelogin" perhaps. In future I might split dovecot-auth even more, and then I would like to create a "doveauth" user as well. Another possibility would be to drop the dovecot user completely and instead use "nobody". That would mean that other nobody processes could kill Dovecot's login processes, but that's pretty much it. Once people understand that "dovecot" isn't the right user, they hit the next problem: "How do I tell Dovecot to run as vmail user?" I can paste links to wiki pages or tell them to "make userdb return uid=vmail", but that just doesn't seem to get through. There needs to be an easier way, and I think I figured out what it is: Add new "mail_uid" and "mail_gid" settings to dovecot.conf. Deprecate user_global_uid/gid in dovecot-ldap.conf and make all the userdbs mention that the uid/gid returned by userdb can be used to override the global mail_uid/gid. Perhaps also add "mail_home" template. This change makes it unnecessary to have a userdb configured at all. - "How do I tell Dovecot to run as vmail user?" - "Set mail_uid = vmail" in dovecot.conf - "Thanks"

10 36

[Dovecot] Dovecot LDA and address extensions
by Tom Metro 12 Jun '07

12 Jun '07

I'm converting over a virtual user installation that used Postfix and maildrop with a MySQL back-end for mailbox lookups to use Dovecot LDA. The maildrop setup used a database table like: +-------------------------+--------+-------------+--------------------+ | address | owner | domain | mailbox | +-------------------------+--------+-------------+--------------------+ | postmaster(a)example.com | tmetro | example.com | .admin.postmaster/ | | root(a)example.com | tmetro | example.com | .admin.root.lex/ | | tmetro(a)example.com | tmetro | example.com | .INBOX/ | | tmetro+spam(a)example.com | tmetro | example.com | .spam.tmetro/ | [...] where maildrop was handed ${recipient} from postfix, and it queries the above table matching ${recipient} to the address column, getting back the expression CONCAT_WS('/','/var/mail',domain,owner) as the home directory, and CONCAT_WS('/','/var/mail',domain,owner,mailbox) as the mailbox. The UID/GID are fixed, as this is a virtual user setup. So looking up the mailbox for the user postmaster(a)example.com returns: /var/mail/example.com/tmetro/.admin.postmaster/ Currently I'm using a static user database with Dovecot. This SQL appears to replicate the maildrop setup: user_query = SELECT CONCAT_WS('/','/var/mail',domain,owner) as home, CONCAT_WS('/','maildir:/var/mail',domain,owner,mailbox) as mail, 5000 as uid, 5000 as gid FROM vmbx2 WHERE userid = '%u' And this works for mail deliveries. The problem with this is that not only is a database lookup overkill for the IMAP server, as the root mail location for IMAP users follows a simple pattern adequately represented by the static userdb, it also breaks IMAP, because the returned 'mail' overrides mail_location, (currently set to maildir:/var/mail/%d/%n/), so for user tmetro(a)example.com it returns: maildir:/var/mail/example.com/tmetro/.INBOX/ instead of: maildir:/var/mail/example.com/tmetro/ (which is the same as the home directory). Can the "userdb sql" directive be added to the "protocol lda" section? Or perhaps instead create a new "auth lda" section, and move the "userdb sql" and "socket listen" directives to that section? The second issue is how to best deal with address extensions, something that wasn't fully implemented in the maildrop setup. Ideally I'd like a setup where a lookup is first performed for user+ext@domain, and if it fails to return a mailbox, then lookup user@domain. Assuming Dovecot can handle a multi-row query result (grabbing the first record and ignoring the rest), this can probably be hacked using an appropriate WHERE expression. But the difficulty is getting the address components. Postfix will provide them, but Dovecot's deliver doesn't seem to have a way of receiving them and passing them on to the user query. The example LDA setup shown here: http://wiki.dovecot.org/LDA/Postfix suggests passing the extension as an argument to the mailbox (-m) switch. But this assumes that one wants a one-to-one direct mapping between extensions and mailbox names. I suppose I could switch from: -d ${user}@${nexthop} to: -d ${recipient} and add even more complexity to the SQL to parse out the extension. But I'd rather avoid embedding the extension separator character in the SQL, and would rather eliminate it from the database as well. Maybe Sieve is the answer. I presume the argument to -m could be inspected in a Sieve script, which could then map to the desired target mailbox. The non-IMAP users (like postmaster(a)example.com) could be reimplemented as aliases that redirect to a user+ext(a)example.com address for the user responsible for that domain. But using Sieve scripts for extension to mailbox mappings seems far less elegant than using a database. Maybe I need to wedge a wrapper program between Postfix and deliver that will accept the address components from postfix, perform the lookup, and convert the results into a -d argument pointing to the "owner" (see table above) and a -m argument pointing to the mailbox. On a side note... When I used this command in postfix's master.cf: /usr/lib/dovecot/deliver -d ${user}@${nexthop} -n -m ${extension} as recommended in the wiki, the following error gets logged: ... status=bounced (command line usage error. Command output: Fatal: Unknown argument: -d Usage: deliver [-c <config file>] [-d <destination user>] [-m <mailbox>] [-f <envelope sender>] ) which doesn't make a lot of sense. I suspect what it is trying to say is that it didn't like something about the argument to the -d switch. Maybe that should be ${domain} instead of ${nexthop}. (What's up with all the whitespace between "[-m <mailbox>]" and "[-f <envelope sender>]"? Also, might be a good idea to have deliver always write errors - even startup errors - to the specified logging facility, given that not all programs that launch it will report what it writes to STDERR. I notice most postfix commands - even the ones people interact with - log errors.) I ended up switching to -d ${recipient} to emulate the old maildrop setup for now, and that worked fine. The sample dovecot.conf has: ## dovecot-lda specific settings ## # socket listen { # master { # path = /var/run/dovecot/auth-master I uncommented the first two lines, but left the 'path' directive commented, under the assumption that the comment was showing the default. But without it uncommented, the socket doesn't get created. This is an inconsistency compared to how defaults are shown elsewhere in dovecot.conf. -Tom

2 1

[Dovecot] vpopmail quota
by Onno Molenkamp 12 Jun '07

12 Jun '07

Hi, I'd like the vpopmail user database to return the quota settings for users, instead of having to rely on the validity of the maildirsize files. It doesn't look like this is currently possible. Would simply adding a auth_stream_reply_add call for "quota" with a suitable value in vpopmail_lookup be sufficient to make this work? (the quota can be read from the pw_shell field of the vqpasswd struct) Onno

2 3