- dovecot - dovecot.org

[Dovecot] Need suggestion regarding index file and Quota settings.
by Tim Chen 20 Jul '08

20 Jul '08

Dear All, My system has about 30,000 accounts and these accounts are created as unix system users. Each account has its own entry in /etc/passwd with different UID and home directory. We plan to deploy an Layer 4 switch as a load balancer. Proposed architecure is as the following: ---- internet ---- Layer 4 switch ---- mail server1 ---- NFS storage | | |--- mail server2 -----| Both mail servers will share the same storage which contains users' home with Maildir inside. Dovecot will be installed on the mail servers serving pop3 and imap4 protocols. Since the layer 4 switch will direct connection to backend mail server in random(or some of its algorithm), users can not guarantee which mail server to use. My first question is which kind of INDEX settings should I apply? 1) INDEX=MEMORY 2) INDEX=/nfs/%u 3) INDEX=/var/spool/%u Will INDEX stored on disk outperform INDEX stored in memory? If INDEX stored on NFS storage, we can be sure the consistency of INDEX file, but what about the performance and locking problem? If INDEX stored on local disk, since users may connect to different mail servers every time. Will that impose the problem of inconsistency? What kind of syndrome will happen if the INDEX became inconsistent to real maildir content? My second question is about quota. We use maildir as mail storage format. Our users were divided into 3-4 groups, each group of users with different quota size ranging from 100MB - 1G. To my best knowledge, dovecot's quota setting in config file supports only "one global configuration". (We are not using db, we use only /etc/passwd). dovecot.conf quota_rule = *:storage=10MB Since dovecot supports both fs and maildir++ quota, I would like to ask for your suggestion: 1. Should I use system quota instead? How about the comparision of performance? Is there any drawback/limitation using system quota? 2. Is there any settings of dovecot that could fullfil my situation? 3. My thought is to modify dovecot.conf, comment out quota_rule = *:storage=10MB and leave only the quota = maildir in the plugin { block. According to my experiment, dovecot will read the maildirsize file under user's home. Then I have to set the maildirsize file for every user individually! It seems to be a little crappy but works. Looking for your valuable suggestions! Thanks very much. Tim Chen

3 2

[Dovecot] Is it ok to upgrade to 1.1?
by Bjørn T Johansen 19 Jul '08

19 Jul '08

Or is it some issues left that need to be sorted out? Regards, BTJ -- ----------------------------------------------------------------------------------------------- Bjørn T Johansen btj(a)havleik.no ----------------------------------------------------------------------------------------------- Someone wrote: "I understand that if you play a Windows CD backwards you hear strange Satanic messages" To which someone replied: "It's even worse than that; play it forwards and it installs Windows" -----------------------------------------------------------------------------------------------

3 3

[Dovecot] OT: Deleting email
by Mauricio Tavares 19 Jul '08

19 Jul '08

I have a mail server running postfix and then using dovecot so others can reach it using imap. I go check mail using thunderbird and delete a few emails, emptying the trash afterwards. Then I quit it and access it using pine. The deleted emails are still there, marked as marked for deletion. I delete a few other emails within pine and quit. It asks to delete the marked emails, I tell go for it, and it does. When I go back to THunderbird all those emails are gone. Next I tell thunderbird to mark certain messages as spam and delete email I marked as spam. I quit it and go back to pine. The spam marked messages which thunderbird swore were deleted are still there. THis really does not seem to be a dovecot issue, but has someone seen that happen? If so, what is going on?

2 1

Re: [Dovecot] Problems with dovecot/sieve and vacation
by iaslanidis＠flumotion.com 19 Jul '08

19 Jul '08

Hello, Thank you for the reply. I will plan to do the upgrade some time soon, that is not questionable. Now back to this, I checked the following: # strings /usr/libexec/dovecot/deliver | grep sendmail /usr/lib/sendmail So, if I understand correctly, this is fine. What I have in maillog is the following: 614u19e82aed7fe5a435(a)mail.gmail.com> for <iaslanidis(a)flumotion.com> ORCPT=rfc822;iaslanidis(a)flumotion.com Jul 19 01:58:56 se01 spampd[9143]: clean message <ea440b1d0807181658s458ba614u19e82aed7fe5a435(a)mail.gmail.com> (0.28/3.50) from <aslanidis(a)gmail.com> for <iaslanidis(a)flumotion.com> ORCPT=rfc822;iaslanidis(a)flumotion.com in 0.04s, 1926 bytes. Jul 19 01:58:56 se01 clamsmtpd: 10C4A0: from=aslanidis(a)gmail.com, to=iaslanidis(a)flumotion.com, status=CLEAN Jul 19 01:58:56 se01 postfix/smtp[15494]: AC414250084: to=<iaslanidis(a)flumotion.com>, relay=127.0.0.1[127.0.0.1]:10025, delay=0.86, delays=0.55/0.01/0.06/0.24, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 253AE250133) Jul 19 01:58:56 se01 deliver(iaslanidis(a)flumotion.com): execv(/usr/lib/sendmail) failed: Permission denied Jul 19 01:58:56 se01 deliver(iaslanidis(a)flumotion.com): Sendmail process terminated abnormally, exit status 89 Jul 19 01:58:56 se01 deliver(iaslanidis(a)flumotion.com): sieve runtime error: Vacation: Error sending mail Jul 19 01:58:57 se01 deliver(iaslanidis(a)flumotion.com): msgid=<ea440b1d0807181658s458ba614u19e82aed7fe5a435(a)mail.gmail.com>: saved mail to INBOX Jul 19 01:58:57 se01 deliver(iaslanidis(a)flumotion.com): msgid=<ea440b1d0807181658s458ba614u19e82aed7fe5a435(a)mail.gmail.com>: saved mail to INBOX Jul 19 01:58:57 se01 postfix/pipe[15529]: 253AE250133: to=<iaslanidis(a)flumotion.com>, relay=dovecot, delay=1.5, delays=0.24/0.01/0/1.2, dsn=2.0.0, status=sent (delivered via dovecot service) For what I see here, this could be a SELinux permission problem, am I right? Regards, Ioannis On Sat, 19 Jul 2008 02:35:11 +0300, Uldis Pakuls <neko(a)it4u.lv> wrote: > Vacation code calls 2 functions: first marks dupes (it seems working) , > then, if it success second function is called which actualy sends > vacation message. This action (success or failure) is loged in log file. > 1. wthat is in your log files? > 2. the only reason I can guess about failure in second function is > sendmail binary. default value of "sendmail_path" (hardcoded) is > /usr/lib/sendmail. You can verify this: "strings > /path/to/dovecot/deliver | grep sendmail" If default value is incorrect > try add sendmail_path variable in dovecot.conf

2 1

[Dovecot] Problems with dovecot/sieve and vacation
by Ioannis Aslanidis 18 Jul '08

18 Jul '08

Hello, First of all, initial data: # dovecot --version 1.0.rc15 # dovecot -n # /etc/dovecot.conf protocols: imaps pop3s disable_plaintext_auth: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_max_processes_count: 256 login_max_connections: 512 verbose_proctitle: yes first_valid_uid: 200 last_valid_uid: 200 first_valid_gid: 200 last_valid_gid: 200 mail_location: maildir:%h/Maildir maildir_copy_with_hardlinks: yes umask: 7 mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %u auth default: mechanisms: login plain user: nobody passdb: driver: ldap args: /etc/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail I am trying to get a vacation autoresponder to work, but I was not able to. Everything else is working fine. Spam scripts work, forward scripts work, ... but vacation does not work. I have tried different configurations, simple: require "vacation"; vacation "I am currently testing the vacation autoresponder."; And a little more complex: require ["fileinto", "vacation"]; ## Move spam to spam folder if exists "X-Spam-Flag" { fileinto "spam.incoming"; # Stop here so that we do not reply on spams stop; } vacation # Reply at most once a day to a same sender :days 1 :subject "Out of office reply test" # List of recipient addresses which are included in the auto replying. # If a mail's recipient is not on this list, no vacation reply is sent for it. :addresses ["iaslanidis(a)flumotion.com"] "I'm testing the vacation autoresponder."; The script gets compiled as I have checked the dates and the .dovecot.lda-dupes gets created; however, there is no autoreply message. Additionally, I noticed that my recipient e-mail address gets added to .dovecot.lda-dupes: # cat .dovecot.lda-dupes � �H+�� ,tԾD+��iaslanidis(a)flumotion.com And, I also noticed that, if I delete this file and I send an e-mail to that address, I receive the e-mail duplicated, but only the first time, before .dovecot.lda-dupes gets created. I checked the mail headers of the e-mail upon reception and they seem fine: Return-Path: <aslanidis(a)gmail.com> X-Original-To: iaslanidis(a)flumotion.com Delivered-To: iaslanidis(a)flumotion.com Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.232]) by mail.fluendo.com (Postfix) with ESMTP id E5777250133 for <iaslanidis(a)flumotion.com>; Fri, 18 Jul 2008 17:48:30 +0200 (CEST) Received: by wr-out-0506.google.com with SMTP id c49so243003wra.19 for <iaslanidis(a)flumotion.com>; Fri, 18 Jul 2008 08:48:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=iMPULXt7n6+6MVu47vQWBga4lMRSIYkfGnO59XcugXQ=; b=bLLBZZxblyF2LFTPMXPgVesAM/Ep/P5GwgN7aaGjd7JFVpXBBNU7Mq4OaeNbFZRzIN jk+wgDl3uvAg3GkhOso7+kxAI+SOe/u8nDBzzCp5muKieMC5xe84gJOh0+bviaaoahDf vxKWX9e/BTvBkq1AOTOfTnlHRONQ00aoNza44= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=PgczmkJcZnKNF2sIhkrPgtbt9OJaNjYVXeJ5498WjvodyiY97ZQEpTFjXoFrM3RWGI EsRDyre253LVRbBJIZses3nbJLWGCQ9aebPnJQMQr6I1dnNL8AIqB6oMjIDp43Yjlj30 muP/LU0PigmTrnsy9+k9kWEy0MRG+ehBBUgn0= Received: by 10.90.101.17 with SMTP id y17mr289500agb.72.1216396109274; Fri, 18 Jul 2008 08:48:29 -0700 (PDT) Received: by 10.90.99.17 with HTTP; Fri, 18 Jul 2008 08:48:29 -0700 (PDT) Message-ID: <ea440b1d0807180848l106e0f08pace1ab165d143ee2(a)mail.gmail.com> Date: Fri, 18 Jul 2008 17:48:29 +0200 From: "Ioannis Aslanidis" <aslanidis(a)gmail.com> To: iaslanidis(a)flumotion.com Subject: Test 12 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: ClamAV using ClamSMTP -- Ioannis Aslanidis I would really like to have this solved as soon as possible, so any help is greatly welcome. If you require any further information, please, do not hesitate to contact me. Thanks in advance for your prompt reply. Regards, Ioannis

4 4

Re: [Dovecot] changing INTERNALDATE or similar
by Karl Rudnick 18 Jul '08

18 Jul '08

Looking at the IMAP4rev1 protocol (RFC 2060), we have the following definition: INTERNALDATE The internal date of the message. 2.3.3. Internal Date Message Attribute The internal date and time of the message on the server. This is not the date and time in the [RFC-822] header, but rather a date and time which reflects when the message was received. In the case of messages delivered via [SMTP], this SHOULD be the date and time of final delivery of the message as defined by [SMTP]. In the case of messages delivered by the IMAP4rev1 COPY command, this SHOULD be the internal date and time of the source message. In the case of messages delivered by the IMAP4rev1 APPEND command, this SHOULD be the date and time as specified in the APPEND command description. All other cases are implementation defined. How could any implementation of this protocol possibly use a file system time stamp to represent that important piece of meta-data, no matter where the file lives? It seems totally reasonable that this is what Outlook uses for the Received date (and I rarely defend Microsoft). This seems like a real design flaw in the dovecot implementation. I am fairly new to dovecot (and like many aspects of it over uw-imap), but having to really be careful with my mail store's mtimes borders on the absurd. I realize it is "implementation defined", but the intent of the definition surely does not refer to file system time stamps. Any chance this can be reconsidered? Is this an actual dovecot issue or a more general Maildir issue? On Fri, 2008-07-18 at 02:00 -0700, dovecot-request(a)dovecot.org wrote: > ------------------------------ > > Message: 4 > Date: Thu, 17 Jul 2008 23:12:55 +0300 > From: Timo Sirainen <tss(a)iki.fi> > Subject: Re: [Dovecot] changing INTERNALDATE or similar > To: richs(a)whidbey.net > Cc: Dovecot Mailing List <dovecot(a)dovecot.org> > Message-ID: <1216325575.31765.349.camel@hurina> > Content-Type: text/plain; charset="utf-8" > > On Thu, 2008-07-17 at 13:04 -0700, richs(a)whidbey.net wrote: > > I believe it's still true that any mail files that are IMAP STORE'd > > (uploaded) > > You mean IMAP APPENDed. > > > to Dovecot by a client will have a current mtime, rather than > > the "Date" in the message. That means might need to compare mtime's > to > > "Date" headers for new files that appear too. > > This is only because the client doesn't specify the date to APPEND > command. > > > Here we just patched Dovecot to treat "INTERNALDATE" the same as the > > "Date" header. Constantly verifying mtime's became too much of a > chore > > for those clients that rely on it (although this technically is > against > > the RFC). > > Would probably have been better to set the default INTERNALDATE based > on > the Date: header in APPEND command. Should be pretty easy to do with > Maildir I think.

3 5

[Dovecot] v1.1.0 released
by Timo Sirainen 18 Jul '08

18 Jul '08

http://dovecot.org/releases/1.1/dovecot-1.1.0.tar.gz http://dovecot.org/releases/1.1/dovecot-1.1.0.tar.gz.sig Two hours later than promised, I slept longer than intended. :) No changes since v1.1.rc13. Below are the largest changes since v1.0: * After Dovecot v1.1 has modified index or dovecot-uidlist files, they can't be opened anymore with Dovecot versions earlier than v1.0.2. * See doc/wiki/Upgrading.1.1.txt (or for latest changes, http://wiki.dovecot.org/Upgrading/1.1) for list of changes since v1.0 that you should be aware of when upgrading. + IMAP: Added support for UIDPLUS and LIST-EXTENDED extensions. + IMAP SORT: Sort keys are indexed, which makes SORT commands faster. + When saving messages, update cache file immediately with the data that we expect client to fetch later. + NFS caches are are flushed whenever needed. See mail_nfs_storage and mail_nfs_index settings. + Out of order command execution (SEARCH, FETCH, LIST), nonstandard command cancellation (X-CANCEL <tag>) + IMAP: STATUS-IN-LIST draft implementation + Expire plugin can be used to keep track of oldest messages in specific mailboxes. A nightly run can then quickly expunge old messages from the mailboxes that have them. The tracking is done using lib-dict, so you can use either Berkeley DB or SQL database. + Namespaces are supported everywhere now. + Namespaces have new list and subscriptions settings. + Full text search indexing support with Lucene and Squat backends. + OTP and S/KEY authentication mechanisms (by Andrey Panin). + mbox and Maildir works with both Maildir++ and FS layouts. You can change these by appending :LAYOUT=maildir++ or :LAYOUT=fs to mail_location. + LDAP: Support templates in pass_attrs and user_attrs + Support for listening in multiple IPs/ports. + Quota plugin rewrite: Support for multiple quota roots, warnings, allow giving storage size in bytes or kilo/mega/giga/terabytes, per-mailbox quota rules. + Filesystem quota backend supports inode limits, group quota and RPC quota for NFS. + SEARCH and SORT finally compare non-ASCII characters case-insensitively. We use i;unicode-casemap algorithm. + Config files support splitting values to multiple lines with \

12 26

[Dovecot] Kerberos check logic incorrect
by Bryan Jacobs 18 Jul '08

18 Jul '08

In configure.in, there is a check that does `krb5-config --version | grep -v 1\.2`, making sure there is no "1.2" in the version of Kerberos in use. This is to prevent compiling against MIT Kerberos version 1.2, which is too old for Dovecot. Unfortunately for this idea, Heimdal 1.2.1 is out. And Dovecot builds fine against it. Whoever committed this check also appears to have removed the comment warning that it was broken, to add insult to injury. Please fix your checks to be for FEATURES, not version numbers. You never know when something else that correctly implements the features you want will come along. Bryan Jacobs

2 1

Re: [Dovecot] changing INTERNALDATE or similar
by Rudnick, Karl 17 Jul '08

17 Jul '08

I successfully executed the script which "fixed" my mail store. I also see now why there isn't a buttoned down solution out there. The basic steps are simple as outlined below, but when dealing with ~35,000 emails going back before 1993, there were just about a dozen that didn't adhere to the "Date:" screening and subsequent capture for use by "touch --date". Adding these exceptions to my script completely automated the INTERNALDATE resetting, but I would not publish this publicly on the internet. Nevertheless, I would share it with anyone who requests through this mail group given the caveat that it is only guaranteed to work for MY personal data store. Thanks - Karl -----Original Message----- From: Rudnick, Karl Sent: Thursday, July 17, 2008 9:06 AM To: dovecot(a)dovecot.org Cc: Timo Sirainen Subject: RE: [Dovecot] changing INTERNALDATE or similar Terrific - I'll just write a script to: 1. create a list of all Maildir folders ending in "/cur" 2. remove dovecot.index* in each folder 3. For each mail file in each Mailder /cur folder: a. grep/sed "Date:" to grab first date/time stamp in email b. pipe the output of a. into "touch --date" for that file On my windows side, all the imap info is saved in one pst file in "C:\Documents and Settings\<user>\Local Settings\Application Data\Microsoft\Outlook\Outlook*imap*.pst" and if I blow it away it gets correctly rebuilt - I just tested it and the blown away index files get rebuilt so that Outlook then has the correct Received date displayed. Pretty easy to do, but surprised a script wasn't openly available - or at least my googling skills weren't good enough. Since I have ~35,000 emails in about 450 folders, this needs to be scripted. I'll make it available when I get it smoothed out. Thanks again for the clarification. - Karl -----Original Message----- From: Timo Sirainen [mailto:tss@iki.fi] Sent: Thursday, July 17, 2008 4:58 AM To: Rudnick, Karl; Dovecot Mailing List Subject: Re: [Dovecot] changing INTERNALDATE or similar On Wed, 2008-07-16 at 16:56 -0700, Karl Rudnick wrote: > I have a question which others must have encountered. I switched from > uw-imap to dovecot-imap a few months ago and moved my fairly large mail > archive (~15 yrs) by simply mounting both accounts in a mail app and > simply copying my folders over. This worked well, and all my mail is in > order when I view it in evolution or thunderbird on any number of > machines. However, since our company uses MS Exchange Server, when I add > my dovecot imap account to outlook, everything is great except the > received dates - it looks like the date I made the transition, but in > all instances it's not simply the time stamp in the Maildir folder. I > googled around on this and have gathered that this date is coming > somewhere from within the dovecot index file. INTERNALDATE comes from the maildir files' mtimes. So you can use whatever utility/script you want to modify the mtime (I don't have any suggestions). Then delete dovecot.index.cache file to make sure the old value wasn't cached. You might also have to recreate the account in your clients to make sure they haven't cached the old values.

1 0

[Dovecot] changing INTERNALDATE or similar
by Karl Rudnick 17 Jul '08

17 Jul '08

I have a question which others must have encountered. I switched from uw-imap to dovecot-imap a few months ago and moved my fairly large mail archive (~15 yrs) by simply mounting both accounts in a mail app and simply copying my folders over. This worked well, and all my mail is in order when I view it in evolution or thunderbird on any number of machines. However, since our company uses MS Exchange Server, when I add my dovecot imap account to outlook, everything is great except the received dates - it looks like the date I made the transition, but in all instances it's not simply the time stamp in the Maildir folder. I googled around on this and have gathered that this date is coming somewhere from within the dovecot index file. Is there a utility for scanning that so I can ascertain where the date shown in Outlook is coming from? Is there also a utility which would just reset the INTERNALDATE (or whatever it is that Outlook is keying on) to the date in the message on the "Date: ..." line? This issue is not life threatening as I don't use Outlook much, but when I do, it is confusing to not have the "Received" column be meaningful, especially for sorting. Thanks for any help - Karl

4 4