- dovecot - dovecot.org

[Dovecot] Corrupted Index Cache
by Jeff Grossman 23 Apr '09

23 Apr '09

I am starting to use Squirrelmail a little bit more to read my mail during the day and I seem to be getting this error more frequently while using Squirrelmail. Just a coincidence or do you think using Squirrelmail might have something to do with it? Corrupted index cache file /home/vmail/jeff/dovecot.index.cache: record points outside file I am using Squirrelmail, IMAPProxy, and Dovecot 1.2RC3. Here is a copy of my dovecot -n. # 1.2.rc3: /usr/local/etc/dovecot.conf # OS: Linux 2.6.29-1-amd64 x86_64 Debian squeeze/sid ext3 base_dir: /var/run/dovecot/ protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/stikman-godaddy.crt ssl_key_file: /etc/ssl/private/stikman-godaddy.key ssl_cipher_list: ALL:!LOW:!SSLv2 disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login mail_location: maildir:/home/vmail/%u mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 auth default: mechanisms: plain login username_format: %Lu passdb: driver: pam userdb: driver: passwd args: uid=vmail gid=vmail home=/home/vmail/%u socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: sieve: sieve

1 0

[Dovecot] possible bug in lib-mail source
by Johann Klasek 23 Apr '09

23 Apr '09

Browsing through the source code to find some properties of deliver I stuck here: --- dovecot-1.2.rc3/src/lib-mail/message-header-parser.c.orig 2009-01-07 19:33:26.000000000 +0100 +++ dovecot-1.2.rc3/src/lib-mail/message-header-parser.c 2009-04-23 09:53:08.000000000 +0200 @@ -284,7 +284,7 @@ Exception to this is if the value consists only of LWSP, then skip only the one LWSP after ':'. */ for (pos = 0; pos < line->value_len; pos++) { - if (!IS_LWSP(line->value[0])) + if (!IS_LWSP(line->value[pos])) break; } I'm not clear if this was really indented but the for-loop does not use the variable pos in the loop body. The expression !IS_LWSP(line->value[0]) is invariant and always false if the first character after ':' is a white space, forcing the loop to go through the whole value. This happens only if flag MESSAGE_HEADER_PARSER_FLAG_SKIP_INITIAL_LWSP is active. I saw this in 1.1.4 too. Johann Klasek -- Johann E. Klasek Zentraler Informatikdienst - Kommunikation Technische Universität Wien Tel: +43 1 58801-42049 A-1040 Wien, Wiedner Hauptstr. 8-10/020C Fax: +43 1 58801-42099 http://pgpkeys.tuwien.ac.at/ PGP Key jklasek

2 1

Re: [Dovecot] FTS Plugin design
by rui.carneiro＠portugalmail.net 23 Apr '09

23 Apr '09

On Thu, Apr 23, 2009 at 5:47 AM, <tomas(a)tuxteam.de> wrote: Note that some formats might require to seek to some point in the file [1] (typically the end), so reading from stdin is awkward (it would require stdin to be seekable, so either the app or the caller would have to put the whole file somewhere anyway). [1] Notably PDF has some index tables at EOF - 1k if I remember correctly. I hadn't thought on that before but I think you are right. The only question here is writing data to memory or hd. Thank you all, Rui Carneiro -- Portugalmail, Comunicações S.A. www.portugalmail.net

2 1

[Dovecot] changing architecture
by Giuliano Gavazzi 23 Apr '09

23 Apr '09

Dear all, moving Maildirs from a PPC cpu to an Intel one (MacOS X to FreeBSD) not surprisingly I got: Apr 23 12:08:50 drum dovecot: IMAP(x): Rebuilding index file /usr/home/ x/Maildir/dovecot.index: CPU architecture changed Apr 23 12:08:52 drum dovecot: IMAP(x): Corrupted index cache file /usr/ home/x/Maildir/dovecot.index.cache: field header points outside file Apr 23 12:09:01 drum dovecot: IMAP(x): fscking index file /usr/home/x/ Maildir/.Drafts/dovecot.index Apr 23 12:09:01 drum dovecot: IMAP(x): Corrupted transaction log file / usr/home/x/Maildir/.Drafts/dovecot.index.log: file size shrank Apr 23 12:09:01 drum dovecot: IMAP(x): fscking index file /usr/home/x/ Maildir/.Posta indesiderata/dovecot.index Apr 23 12:09:01 drum dovecot: IMAP(x): Corrupted transaction log file / usr/home/x/Maildir/.Posta indesiderata/dovecot.index.log: file size shrank I suppose owed to the endianness change. Can I thus just remove those dovecot.index files, including dovecot.index.cache and dovecot.index.cache.log? I know I could let dovecot do it, but as the cache files in particular are quite large, it would save me band simply not transferring them with rsync from the old server. thanks Giuliano

1 1

[Dovecot] bad maildir location
by denis 23 Apr '09

23 Apr '09

Hi, I use dovecot 1.0.15 on Debian Lenny. In my /etc/dovecot/dovecot-sql.conf, i use user_query = SELECT path, 33 AS uid, uid AS gid FROM mail_users WHERE alias = '%u' The result is something like this /var/alternc/mail/d/denis_collectifs.net and is the correct location (the Maildir is created by an script) But the problem is that dovecot creates (when you log in for the first time) a directory in the form of the exact address. Here for example, /var/alternc/mail/d/denis(a)collectifs.net And so all connections will be on an empty location. How is it possible to resolve this? As mail_location I have this "maildir:/var/alternc/mail/%1u/%u/Maildir" which is not really correct. Thanks Denis PS: For now, I temporarily symlinks /var/alternc/mail/d/denis(a)collectifs.net /var/alternc/mail/d/denis_collectifs.net

2 4

[Dovecot] server is not imap4 compatible
by marco ghidinelli 23 Apr '09

23 Apr '09

hello, i have a problem with my dovecot installation. sometimes, the thunderbird clients on windows machines complains about: "server is not imap4 compatible". on the server, at the same time, i found this error: Apr 20 16:21:17 harlock dovecot: imap-login: Disconnected: Shutting down: rip=192.168.0.194, lip=10.70.0.1, TLS handshake the imap server is behind a dmz which sometimes drops connections. the server ran without problem until i upgraded from 1.0.rc15 to 1.0.15. after the upgrade i was forced to enhance the proc.sys.fs.inotify.max_user_instances = 1024 because i had error messages about "Inotify instance limit for user exceeded" my question is: does the upgrade triggered a new way of communication between clients and server that exacerbates the problem of dropped connections from the DMZ server? p.s. the dovecot configuration is: $ sudo dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/harlock_digicert_2009.pem ssl_key_file: /etc/ssl/private/harlock_digicert_2009.key login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_max_processes_count: 256 mail_privileged_group: mail mail_location: maildir:/var/mail/vhosts/turboden.net/%n/ mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login default_realm: DOMAIN.LOCAL username_format: %Ln@%Ud passdb: driver: pam userdb: driver: static args: uid=5000 gid=5000 socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix

3 5

[Dovecot] Mailbox is locked?
by Kyle Wheeler 23 Apr '09

23 Apr '09

Hello, I recently upgraded from Dovecot 1.1.8 to 1.1.14, and I've started to get errors I never got before. Specifically, I'm getting "Mailbox is locked, will abort in xx seconds" errors. My mail client (mutt) will just sit there in the background (presumably in IDLE) and randomly will show this error (I THINK it has something to do with delivering new messages to the folder; I use the dovecot LDA). Nothing else about my configuration has changed, just the version of dovecot. Does anyone know what could be causing these? I assume it has something to do with the index files, but... Nothing interesting shows up in the log files. How do I get rid of these error messages? Or even begin tracking them down? For what it's worth, mail is stored on the server in Maildirs, and the lock_method is fcntl, the mbox_write_locks is fcntl (I don't set mbox_read_locks). ~Kyle -- For to accuse, requires less eloquence, such is man's nature, than to excuse; and condemnation, than absolution more resembles justice. -- Hobbes, Leviathan

2 2

[Dovecot] overnight clients causing lockfiles to be converted to root-owned messages on debian lenny
by Brett Dikeman 23 Apr '09

23 Apr '09

Greetings all, After running for more than a year without issue on debian lenny while it was in the testing- we're now experiencing problems under Lenny/Stable. Dovecot now converts some sort of lockfile into messages, and they're owned by root. The result is a mess; clients can't read their mailboxes with the root-owned files, and when the file is changed to be owned by them, many mail clients don't like the zero-length message (Outlook being one major client with this "problem".) This happens to users who leave their email clients open overnight. I haven't found a cronjob that could be responsible for this. Further details- 64 bit machine, we use maildir, /var/spool/mail is on an xfs volume, and we run postfix, procmail, and spamassassin. Dovecot version is 1.0.15; the machine is fully up-to-date with Debian Stable (Lenny.) Sample log entry of the aftermath; please pardon the sanitizing (host/username): Apr 22 00:08:34 host dovecot: IMAP(user): Fixed a duplicate: 1240289413.9181_0.host:2,S.lock -> 1240373314.P6717Q0M159857.host Apr 22 00:08:34 host dovecot: IMAP(user): Fixed a duplicate: 1240356531.15928_0.host:2,S.lock -> 1240373314.P9995Q3.host Apr 22 00:08:34 host dovecot: IMAP(user): open(/var/spool/mail/user/Maildir/cur/1240373314.P6717Q0M159857.host:2,) failed: Permission denied Brett

5 8

[Dovecot] expire-tool
by Jeff Rice 23 Apr '09

23 Apr '09

Hi, I'm having some trouble getting the expire plugin to work, and I'm not sure where to turn for troubleshooting. This is on 1.2rc3. To my dovecot.conf, I've added: mail_plugins = expire under the protocol imap and lda sections. I also added: dict { expire = mysql:/etc/dovecot/dovecot-dict-expire.conf } plugin { expire = Trash 30 Trash/* 30 Spam 30 expire_dict = proxy::expire auth_socket_path = /var/run/dovecot/auth-master } /etc/dovecot/dovecot-dict-expire.conf contains: -- connect = host=/var/run/mysqld/mysqld.sock dbname=email user=XXX password=XXX map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } -- I created the table and granted all permissions to the user. The socket is the same one used by the auth_socket process, so that should work. Dovecot restarts fine, but the table is empty. A couple questions: First, the mailboxes to expire are stored as user(a)domain/.INBOX.Trash. How should I list them in the conf file? Also, is there any way to have the system come up to date since older emails already exist in those boxes and should be deleted. Will it detect those, or only start to delete old emails that were delivered after the plugin was enabled? Second, should the table be empty if mail has been delivered & deleted since I enabled the plugin? I assumed it would start to fill. Jeff

2 1

[Dovecot] Quota for Shared Folders
by Anton Dollmaier 22 Apr '09

22 Apr '09

Good morning list, first of all: dovecot works really great, the performance is overwhelming (especially compared to courier), the configuration flexible as hell, it is good documented - I love this software. But as things get complicated, I think I need some additional help. I'm using dovecot to replace the currently used courier-mailserver in a shared hosting environment based on the control panel Confixx. Confixx uses per default filesystem-users, giving each mailuser a filesystem-quota, mails are stored in ~/Maildir/. The imap-logins are all in the form of "webxpy", being "webx" the customer (prefix 'web', and a increasing number), and "py" the number of the pop account. Because of the filesystem-quota we moved the indexes to another partition. This basic setup with dovecot works great, with auth directly against the confixx-database etc, imap_quota-plugin, quota-plugin working as expected. We now want to implement shared folders for each customer: all mailaccounts of a customer should be able to subscribe to per-customer shared folders in a specified maildir, thus I re-mapped the usernames with a new user_query (and password_query) to the format of "webxpy@webx" - being "webx" the virtual domain and "webxpy" the mailaccount, but still using "webxpy" as the imap-login (no need for customers to change the login). This also works, we even can control with ACL-vfiles the permissions of specified users, to enable or disable specific folders. The shared folders are specified as a new public namespace and the "domain" is part of the storage-path (see configuration at the end of this mail). To allow easier quota-management (no need to create a new system-user), the quota of the shared folders is maildir-based, with a mysql-quota-dictionary (later, all accounts will be migrated to a virtual-user-mailstorage, with only maildir-quota and no fs-quotas). But when things come to the shared quota, I currently don't know how to solve the wishes of the customer: The current solution for shared folders (at the customer's local site) is based on mdaemon, every shared folder (of a customer, not a single mail-account) has a quota assigned. This quota cannot be exceeded by the users, no more mails can be saved into the shared folders if used space exceeds the hard quota-value. As my tests have shown, dovecot uses a different approach to the quota-issue: users can always move their mails from local folders to the shared namespace, regardless of the shared quota. The quota itself is only checked when receiving new mails, where the combined quota (local + shared) may not be exceeded. If it is, the mail is not delivered to the mailbox. Our quota-config is this: > quota_rule: *:storage=50M:messages=1000 > quota_rule2: Trash:storage=50M:messages=100 > quota_rule3: shared*:storage=100M:messages=1000 (also being altered by user_query, but the scheme is the same) As of dovecot-wiki, this means: Users can save "50M + 50M + 100M" of mails in all specified Folders, so the storage-value for the "shared*"-folders works additionally to the "*"-storage-value. Different from this, we want the following to happen: If the customer exceeds the shared folders-quota, his mail-users may not move mails from INBOX to shared, this move-operation should fail with a reasonable error-message. If the local quota of a mail user is exceeded, new mails should be rejected/delayed, regardless of the shared quota-value (e.g., shared quota exceeded, but no local mails [new mail-account], mails should be delivered to INBOX). Sieve-filters are not a problem, we do not allow user-specific sieve-rules (managesieve is disabled). Hopefully, I could explain my issues to you. Our current dovecot-configuration is attached to this mail, any help would be gladly appreciated. If additional configuration-values/information is needed, I'll happily provide them, if possible. best regards, Anton Dollmaier > # 1.1.13: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.26-1-686 i686 Debian 5.0.1 > log_timestamp: %Y-%m-%d %H:%M:%S > protocols: imap imaps pop3s pop3 > listen: *, [::] > ssl_cert_file: /etc/dovecot/dovecot.pem > ssl_key_file: /etc/dovecot/dovecot.pem > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > mail_max_userip_connections(default): 25 > mail_max_userip_connections(imap): 25 > mail_max_userip_connections(pop3): 10 > first_valid_uid: 249 > mail_access_groups: poponly > mail_privileged_group: poponly > mail_location: maildir:~/Maildir:INDEX=~/index:CONTROL=~/control > mail_debug: yes > mail_executable(default): /usr/lib/dovecot/imap > mail_executable(imap): /usr/lib/dovecot/imap > mail_executable(pop3): /usr/lib/dovecot/pop3 > mail_plugins(default): quota imap_quota acl > mail_plugins(imap): quota imap_quota acl > mail_plugins(pop3): quota acl > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > imap_client_workarounds(default): netscape-eoh > imap_client_workarounds(imap): netscape-eoh > imap_client_workarounds(pop3): > pop3_client_workarounds(default): > pop3_client_workarounds(imap): > pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh > namespace: > type: private > separator: . > inbox: yes > list: yes > subscriptions: yes > namespace: > type: private > prefix: INBOX. > hidden: yes > subscriptions: yes > namespace: > type: public > separator: . > prefix: shared. > location: maildir:/var/mail/shared/%d/:INDEX=~/shared > list: yes > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: postfix > master: > path: /var/run/dovecot/auth-master > mode: 432 > user: vmail > group: vmail > plugin: > quota: dict:::proxy::quotadict > quota2: dict::%d:proxy::quota2dict > quota_rule: *:storage=50M:messages=1000 > quota_rule2: Trash:storage=50M:messages=100 > quota_rule3: shared*:storage=100M:messages=1000 > acl: vfile > expire: Trash 7 Spam 30 > expire_dict: proxy::expire > dict: > quotadict: mysql:/etc/dovecot/dovecot-dict-quota.conf > quota2dict: mysql:/etc/dovecot/dovecot-dict-quota2.conf (the two quota-dicts could probably be merged into one) /etc/dovecot/dovecot-sql.conf: > driver = mysql > connect = host=localhost dbname=confixx user=confixx password=p4ssw0rd > default_pass_scheme = CRYPT > password_query = SELECT CONCAT(account, '@', kunde) as user, longpw as password FROM pop3 WHERE (account='%u' and gesperrt='0' > user_query = SELECT CONCAT('/var/mail/vmail/', p.kunde, '/', p.account,'/') as home, 249 as uid, 249 as gid, CONCAT('*:storage=', p.maxkbhard) AS quota_rule, CONCAT('shared.*:storage=', k.shared_maxkb) as quota2_rule FROM pop3 AS p, kunden AS k WHERE k.kunde = p.kunde AND CONCAT(p.account, '@', p.kunde) = '%u' (quota-values in kilobytes) /etc/dovecot/dovecot-dict-quota.conf: > connect = host=localhost dbname=confixx user=confixx password=p4ssw0rd > table = dovecot_quota > select_field = current > where_field = path > username_field = username /etc/dovecot/dovecot-dict-quota2.conf: > connect = host=localhost dbname=confixx user=confixx password=p4ssw0rd > table = dovecot_quota_shared > select_field = current > where_field = path > username_field = username (tables are exactly as specified in dovecot-wiki)

2 3