October 2006 - dovecot

[Dovecot] quota dict (mysql)
by Peter Veerman 16 Oct '06

16 Oct '06

Hi, I have dovecot configured to make use of the dict (mysql) plugin in order to keep track of the used quota. First, as you can see in the mysql table output below it seems that values aren't stored correctly. Secondly I get all kinds of database related errors in my mail logs. The (relevant) sections of the configuration files are also included below. My question is, what am I doing wrong? Peter mysql> select * from quota; +------------------+-------+---------+ | username | path | current | +------------------+-------+---------+ | userc@domain1 | 0 | 0 | | usera@domain1 | 0 | 0 | | userb@domain1 | 0 | 0 | | userb@domain1 | 5417 | 0 | | userb@domain1 | 937 | 0 | | userb@domain1 | 3724 | 0 | | userb@domain1 | -9141 | 0 | | userb@domain1 | 4887 | 0 | | userb@domain1 | -4887 | 0 | | usera@domain2 | 0 | 0 | | usera@domain1 | 3134 | 0 | | usera@domain1 | 2177 | 0 | | usera@domain1 | 5137 | 0 | | usera@domain1 | 4953 | 0 | +------------------+-------+---------+ 14 rows in set (0.03 sec) dovecot.conf: dict { quotadict = mysql:/usr/local/etc/dovecot/dovecot-dict-quota.conf } plugin { quota = dict:storage=10240 proxy::quotadict } dovecot-dict-quota.conf: connect = host=localhost dbname=postfix user=postfix password=******* table = quota select_field = current where_field = path username_field = username dovecot.log gives: Oct 02 23:25:49 Info: imap-login: Login: user=<user(a)domain.net>, method=plain, rip=88.211.149.123, lip=10.0.0.4, TLS Oct 02 23:25:49 Info: IMAP(user(a)domain.net): Effective uid=125, gid=125 Oct 02 23:25:49 Info: IMAP(user(a)domain.net): Loading modules from directory: /usr/local/lib/dovecot/imap Oct 02 23:25:49 Info: IMAP(user(a)domain.net): Module loaded: /usr/local/lib/dovecot/imap/lib01_quota_plugin.so Oct 02 23:25:49 Info: IMAP(user(a)domain.net): Module loaded: /usr/local/lib/dovecot/imap/lib02_imap_quota_plugin.so Oct 02 23:25:49 Info: IMAP(user(a)domain.net): Namespace: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, subscriptions=no Oct 02 23:25:49 Info: IMAP(user(a)domain.net): maildir: data=/data/postfix/virtual/domain.net/user Oct 02 23:25:49 Info: IMAP(user(a)domain.net): maildir: root=/data/postfix/virtual/domain.net/user, index=/data/postfix/virtual/domain.net/user, control=, inbox= Oct 02 23:25:49 Info: IMAP(user(a)domain.net): dict quota: uri = proxy::quotadict Oct 02 23:25:49 Info: IMAP(user(a)domain.net): dict quota: byte limit = 524288000 Oct 02 23:25:49 Info: IMAP(user(a)domain.net): dict quota: count limit = 0 Oct 02 23:25:49 Info: dict: mysql: Connected to localhost (postfix) Oct 02 23:25:55 Info: dict: mysql: Connected to localhost (postfix) Oct 02 23:25:56 Info: dict: mysql: Connected to localhost (postfix) Oct 02 23:26:37 Info: dict: mysql: Connected to localhost (postfix) Oct 02 23:27:07 Info: imap-login: Aborted login: rip=10.0.0.25, lip=10.0.0.22, TLS Oct 02 23:27:11 Info: dict: mysql: Connected to localhost (postfix) mail.debug gives: Oct 2 23:26:39 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:39 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:39 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:39 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:40 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:41 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:42 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:42 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:42 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:48 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:48 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota Oct 2 23:26:48 bubbles dovecot: IMAP(user(a)domain.net): dict_quota: Couldn't update quota

3 6

[Dovecot] Conf parsing error
by Adnae Mac Uthidir 16 Oct '06

16 Oct '06

Dear Timo, I have found a conf-parse bug in rc10. If the last line in the configuration-file a }, the parser ignore this. Example: ----------------------------- brugh ~ # cat /etc/dovecot/dovecot.conf [...] plugin { trash = /etc/dovecot/trash.conf } brugh ~ # dovecot -c /etc/dovecot/dovecot.conf Error: Missing '}' ----------------------------- Add a empty line at the end of file and you can start without errors ;) -- Mit freundlichen Grüßen Adnae Mac Uthidir mailto:adnaeuthidir@gmail.com

2 1

[Dovecot] Segfault in in rc7 when index does not exists
by Marco Nenciarini 16 Oct '06

16 Oct '06

This is the trace (gdb) run Starting program: /root/tmp/dovecot-1.0.rc7/src/imap/imap x select inbox Program received signal SIGSEGV, Segmentation fault. mail_index_write_base_header (index=0x80e2a28, hdr=0xaff66028) at mail-index.c:1313 1313 memcpy(index->map->mmap_base, hdr, hdr_size); (gdb) bt #0 mail_index_write_base_header (index=0x80e2a28, hdr=0xaff66028) at mail-index.c:1313 #1 0x080902eb in mail_index_mark_corrupted (index=0x80e2a28) at mail-index.c:1839 #2 0x0809ca30 in mail_transaction_log_file_create (log=0x80e3928, path=0x80d2130 "/dovecot.index.log", dev=0, ino=0, file_size=0) at mail-transaction-log.c:575 #3 0x0809e138 in mail_transaction_log_file_open_or_create (log=0x80e3928, path=0x80d2130 "/dovecot.index.log") at mail-transaction-log.c:803 #4 0x0809e65c in mail_transaction_log_open_or_create (index=0x80e2a28) at mail-transaction-log.c:240 #5 0x08092f21 in mail_index_open (index=0x80e2a28, flags=MAIL_INDEX_OPEN_FLAG_CREATE, lock_method=MAIL_INDEX_LOCK_FCNTL) at mail-index.c:1489 #6 0x0808aa52 in index_storage_mailbox_init (ibox=0x80e3548, index=0x80e2a28, name=0x80c32e4 "INBOX", flags=0, move_to_memory=false) at index-storage.c:362 #7 0x08064d68 in maildir_open (storage=0x80dd420, name=0x80c32e4 "INBOX", flags=<value optimized out>) at maildir-storage.c:487 #8 0x08065628 in maildir_mailbox_open (_storage=0x80dd420, name=0x80c32e4 "INBOX", input=0x0, flags=0) at maildir-storage.c:560 #9 0x08059917 in _cmd_select_full (cmd=0x80dd62c, readonly=false) at cmd-select.c:32 #10 0x08059af7 in cmd_select (cmd=0x80dd62c) at cmd-select.c:92 #11 0x0805b3d4 in _client_input (context=0x80dd5e8) at client.c:377 #12 0x080b7f96 in io_loop_handler_run (ioloop=0x80db9c0) at ioloop-poll.c:199 #13 0x080b7348 in io_loop_run (ioloop=0x80db9c0) at ioloop.c:281 #14 0x0806316e in main (argc=Cannot access memory at address 0x0 ) at main.c:280 (gdb) p index->map $1 = (struct mail_index_map *) 0x0 (gdb) the envisonment is: DBOX_ROTATE_DAYS="1" DBOX_ROTATE_MIN_SIZE="16" DBOX_ROTATE_SIZE="2048" DOVECOT_VERSION="1.0.rc7" HOME="/srv/vmbox/spool/test/" IMAP_CLIENT_WORKAROUNDS="outlook-idle" IMAP_MAX_LINE_LENGTH="65536" IP="::ffff:127.0.0.1" LOCK_METHOD="fcntl" LOGGED_IN="1" LOG_TO_MASTER="1" MAIL="maildir:/" MAILBOX_IDLE_CHECK_INTERVAL="30" MAIL_CACHE_FIELDS="flags" MAIL_CACHE_MIN_MAIL_COUNT="0" MAIL_MAX_KEYWORD_LENGTH="50" MAIL_NEVER_CACHE_FIELDS="imap.envelope" MBOX_DIRTY_SYNCS="1" MBOX_DOTLOCK_CHANGE_TIMEOUT="120" MBOX_LAZY_WRITES="1" MBOX_LOCK_TIMEOUT="300" MBOX_MIN_INDEX_SIZE="0" MBOX_READ_LOCKS="fcntl" MBOX_WRITE_LOCKS="dotlock fcntl" POP3_CLIENT_WORKAROUNDS="" POP3_LOGOUT_FORMAT="top=%t/%p, retr=%r/%b, del=%d/%m, size=%s" POP3_UIDL_FORMAT="" RESTRICT_CHROOT="/srv/vmbox/spool/test" RESTRICT_GID_FIRST="1" RESTRICT_SETEXTRAGROUPS="mail" RESTRICT_SETGID="200" RESTRICT_SETUID="200" STDERR_CLOSE_SHUTDOWN="1" SYSLOG_FACILITY="16" USER="test(a)prato.linux.it" The directory /srv/vmbox/spool/test contains: # find /srv/vmbox/spool/test /srv/vmbox/spool/test /srv/vmbox/spool/test/cur /srv/vmbox/spool/test/new /srv/vmbox/spool/test/tmp I hope this is enougth Please keep me on CC because i'm not subscribed. Ciao P.S: the rc8 do exactly the same. -- --------------------------------------------------------------------- | Marco Nenciarini | Debian/GNU Linux Developer - Plug Member | | mnencia(a)prato.linux.it | http://www.prato.linux.it/~mnencia | --------------------------------------------------------------------- Key fingerprint = FED9 69C7 9E67 21F5 7D95 5270 6864 730D F095 E5E4

2 5

[Dovecot] File isn't in mbox format: /home/foo/mail/Trash
by James Lupolt 16 Oct '06

16 Oct '06

Hello, I'm seeing the following error on a server: File isn't in mbox format: /home/foo/mail/Trash The Trash file is full of the following at the beginning: ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^ I've repaired the file several times, but this keeps happening. Would any of you mind offering me some insight into what could be causing this and how to fix it? Below is some information about the system: [root@server1 mail]# uname -a Linux server1.example.com 2.6.9-42.0.2.EL #1 Thu Aug 17 17:36:53 EDT 2006 i686 athlon i386 GNU/Linux [root@server1 mail]# cat /etc/redhat-release Red Hat Enterprise Linux ES release 4 (Nahant Update 4) [root@server1 mail]# rpm -qa|grep -i dovecot dovecot-0.99.11-4.EL4 [root@sever1 mail]# rpm -qa|grep -i sendmail sendmail-cf-8.13.1-3.RHEL4.5 sendmail-8.13.1-3.RHEL4.5 Here's everything that isn't a comment or blank line in dovecot.conf: [root@server1 mail]# grep -v ^# /etc/dovecot.conf|grep -v ^$ protocols = imap imaps pop3 pop3s imap_listen = [::] pop3_listen = [::] login_dir = /var/run/dovecot-login login = imap login = pop3 mbox_locks = flock auth = default auth_mechanisms = plain auth_userdb = passwd auth_passdb = pam auth_user = root Any advice would be appreciated. Best Regards, James Lupolt

2 1

[Dovecot] System-wide sieve script?
by Ty! Boyack 16 Oct '06

16 Oct '06

In the dovecot LDA documentation, it mentions having sieve scripts located in ~/.dovecot.sieve, which works great for per-user sieve code. I was trying to find out if there was a system-wide sieve script (maybe /etc/dovecot.sieve?) that would hopefully be run first, followed by any per-user sieve code. Anyone know if this is already possible, or if not, if there are any plans to implement this? I'm assuming one could make a kludgey fix by having every .dovecot.sieve include a system-wide file, but that would allow a user to remove that "include" if they choose. I'd like to have a centrally controlled, immutable set of rules that all mail goes through. If I missed something in the wiki or archives, I offer my apologies in advance. Thanks! -Ty!

5 7

[Dovecot] LDAP Memory Leak
by Daniel 16 Oct '06

16 Oct '06

Hi Timo I use LDAP auth on a system with around 2000 email account. I use dual OpenLDAP servers. I haven't noticed a leak of memory. Is there any more information and I will have a look to see if I can see anything that might help. Is it only affecting certain machines or all? Regards Daniel Rowe Date: Mon, 16 Oct 2006 02:39:26 +0300 From: Timo Sirainen <tss(a)iki.fi> Subject: [Dovecot] 1.0.rc10 released To: dovecot-news(a)dovecot.org, dovecot(a)dovecot.org Message-ID: <1160955566.5448.126.camel@hurina> Content-Type: text/plain; charset="us-ascii" http://dovecot.org/releases/dovecot-1.0.rc10.tar.gz http://dovecot.org/releases/dovecot-1.0.rc10.tar.gz.sig I've finally read all the mails in the mailing list and in my INBOX. If I haven't replied to some of your mail, please resend it. Remember that since 1.0.rc9 release dovecot.index.cache files will get rebuilt in 64bit systems, and it's probably better to delete them manually so you don't get errors in log files. There are only a couple of issues left in my v1.0-TODO list: - Master process appears to be leaking log fds with kqueue. Could someone again give me access to a system where this happens? - Login process problems. How well does it work now? Hopefully well enough that v1.0 could be released. - LDAP authentication is leaking memory? Can anyone confirm this? Even better, can someone figure out what exactly is leaking? :) Not a v1.0 blocker though. I think v1.0 will be released once no-one has reported any major problems for a Dovecot release in 2-4 weeks. I think login process handling is the only potentially major problem left. There are a few patches from people that I haven't forgotten, but I've decided not to put them into v1.0 anymore: - Filesystem quota group. I don't think it's that important feature, and it might break something. - HFS+ hardlink avoiding - Managesieve - vmailmgr support And finally the changes in this release: * When matching allowed_nets IPs, convert IPv6-mapped-IPv4 addresses to actual IPv4 addresses first. + IMAP: Try to avoid sending duplicate/useless message flag updates + Added support for non-plaintext authentication for vpopmail if it returns plaintext passwords. Based on patch by Remi Gacogne. + Added %D modified to return "sub.domain.org" as "sub,dc=domain,dc=org" (for LDAP queries). Patch by Andrey Panin. - rc9 broke cache files in 64bit systems - deliver works now with mail_chroot - auth cache didn't work properly with multiple passdbs - Fixes to handling CRLF linefeeds in mboxes.

2 1

[Dovecot] Dovecot and CRAM-MD5 authentication (brief) HOWTO
by James Turnbull 16 Oct '06

16 Oct '06

Hi all, I just finished configuring CRAM-MD5 for Dovecot and noted there were not a lot of resources out there that described the process. I've written up a rough and brief HOWTO at: http://www.kartar.net/weblog/cram-md5-authentication-with-dovecot/ Comments welcome and hope it helps someone. Regards James Turnbull -- James Turnbull <james(a)lovedthanlost.net> --- Author of Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) Hardening Linux (http://www.amazon.com/gp/product/1590594444/) --- PGP Key (http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x0C42DF40)

1 0

[Dovecot] LDAP Memory Leak
by Daniel 16 Oct '06

16 Oct '06

Hi Timo I use LDAP auth on a system with around 2000 email account. I use dual OpenLDAP servers. I haven't noticed a leak of memory. Is there any more information and I will have a look to see if I can see anything that might help. Is it only affecting certain machines or all? Regards Daniel Rowe Date: Mon, 16 Oct 2006 02:39:26 +0300 From: Timo Sirainen <tss(a)iki.fi> Subject: [Dovecot] 1.0.rc10 released To: dovecot-news(a)dovecot.org, dovecot(a)dovecot.org Message-ID: <1160955566.5448.126.camel@hurina> Content-Type: text/plain; charset="us-ascii" http://dovecot.org/releases/dovecot-1.0.rc10.tar.gz http://dovecot.org/releases/dovecot-1.0.rc10.tar.gz.sig I've finally read all the mails in the mailing list and in my INBOX. If I haven't replied to some of your mail, please resend it. Remember that since 1.0.rc9 release dovecot.index.cache files will get rebuilt in 64bit systems, and it's probably better to delete them manually so you don't get errors in log files. There are only a couple of issues left in my v1.0-TODO list: - Master process appears to be leaking log fds with kqueue. Could someone again give me access to a system where this happens? - Login process problems. How well does it work now? Hopefully well enough that v1.0 could be released. - LDAP authentication is leaking memory? Can anyone confirm this? Even better, can someone figure out what exactly is leaking? :) Not a v1.0 blocker though. I think v1.0 will be released once no-one has reported any major problems for a Dovecot release in 2-4 weeks. I think login process handling is the only potentially major problem left. There are a few patches from people that I haven't forgotten, but I've decided not to put them into v1.0 anymore: - Filesystem quota group. I don't think it's that important feature, and it might break something. - HFS+ hardlink avoiding - Managesieve - vmailmgr support And finally the changes in this release: * When matching allowed_nets IPs, convert IPv6-mapped-IPv4 addresses to actual IPv4 addresses first. + IMAP: Try to avoid sending duplicate/useless message flag updates + Added support for non-plaintext authentication for vpopmail if it returns plaintext passwords. Based on patch by Remi Gacogne. + Added %D modified to return "sub.domain.org" as "sub,dc=domain,dc=org" (for LDAP queries). Patch by Andrey Panin. - rc9 broke cache files in 64bit systems - deliver works now with mail_chroot - auth cache didn't work properly with multiple passdbs - Fixes to handling CRLF linefeeds in mboxes.

1 0

[Dovecot] Apple Mail quota display
by Timothy Martin 16 Oct '06

16 Oct '06

I know this isn't necessarily a dovecot issue, but I was wondering if anyone has the quota displaying correctly in Apple Mail? I have it set up and working and displaying properly in thunderbird, roundcube (webmail), and squirrelmail. But for some reason Apple's Mail doesn't show the values or its nice thermometer bar. Are there additional configuration options for quota not documented on the wiki? .tim

2 3

[Dovecot] Duplicate mails
by Vegar Nilsen 16 Oct '06

16 Oct '06

Hi, one of our customers reported a problem after we moved her maildir to another disk. (Jail in FreeBSD 6, moved to another disk with rsync.) For some reason she's seeing duplicates of almost every message from November, 2005 until June, 2006 (the move happened in September, 2006.) It seems like these mails have been respooled somehow and been inserted into the maildir with the same date and time as the original: root#loc [</var/gopostal/maildirs/wow/lina/cur>] diff 1158863410.P19488Q34M511758.locusts.copyleft.no:2,STbf 1132869216.H332479P92237.mail9.copyleft.no,S=7755:2,STac root#loc [</var/gopostal/maildirs/wow/lina/cur>] ls -l 1158863410.P19488Q34M511758.locusts.copyleft.no:2,STbf -rw------- 1 mailnull mail 7755 Nov 24 2005 1158863410.P19488Q34M511758.locusts.copyleft.no:2,STbf root#loc [</var/gopostal/maildirs/wow/lina/cur>] ls -l 1132869216.H332479P92237.mail9.copyleft.no,S=7755:2,STac -rw------- 1 mailnull mail 7755 Nov 24 2005 1132869216.H332479P92237.mail9.copyleft.no,S=7755:2,STac As the diff command shows these mails are identical, except that one was created about 10 months after the other. 1132869216.H332479P92237.mail9.copyleft.no,S=7755:2,STac was written by Exim in November (we've configured Exim to add the size (",S=7755") to the file name), but I have no idea where the other file comes from. Could this be Dovecot that has recreated these mails for some reason? (Since the file names are different I doubt that this happened during the rsync, instead it must come from some program that conforms to the maildir standard of naming files. The mailserver had a different name back in November, mail9 was the old name, locusts is the new one.) Originally this file 1158863410.P19488Q34M511758.locusts.copyleft.no:2,STbf was named 1158863410.P19488Q34M511758.locusts.copyleft.no:2,bf which, if I understand it correctly, means it's unread. (The customer has deleted it since reporting this problem.) root#loc [</var/gopostal/maildirs/wow/lina/cur>] uname -r ; dovecot --version ; exim -bV 6.1-RELEASE 1.0.beta3 Exim version 4.60 #0 (FreeBSD 6.0) built 22-Feb-2006 13:01:52 If anybody has an idea as to what caused this I would really appreciate any pointers or info. Thanks in advance, Vegar Nilsen -- Copyleft Software AS < http://www.copyleft.no/ >

2 1