May 2008 - dovecot - dovecot.org

[Dovecot] UID inserted in the middle of mailbox errors
by Dean Brooks 04 May '08

04 May '08

Hi, We are still getting, on a daily basis, users who cannot move messages to Trash or expunge Trash due to these errors: Mar 28 10:43:57 tm2 dovecot: [ID 107833 mail.error] IMAP(kss021): mbox sync: UID inserted in the middle of mailbox /home/kss021/Mail/Trash (3137 > 536, seq=2, idx_msgs=76) Our users have no other access to their mailbox other than Dovecot, and we are using mbox format and 1.1rc3. Deleting the indexes fixes the problem, so it seems to be a problem with index corruption and not something inherently wrong with the mailbox. That is, once I delete the index files, the problem does not come back immediately. I've seen others on the list with the same problem, but I wasn't able to find a workaround. It's happening more and more frequently. I guess what I don't understand is why Dovecot cant just mark the index invalid and purge it instead of failing over and over. I thought the whole point behind Dovecot's indexing is that it was able to detect index inconsistencies and mark the index invalid. Any help is much appreciated. Even a quick stopgap fix would be a lifesaver, short of a cronjob that just purges everyone's index files every 2 hours (which sort of defeats the point of using Dovecot). -- Dean Brooks dean(a)iglou.com

3 4

[Dovecot] Embedded From_ lines breaking Content-Length (and Dovecot)
by Lennart Lövstrand 04 May '08

04 May '08

I feel like I'm going totally crazy. Is it just me, or have embedded From_ lines really been breaking mbox messages since (at least) dovecot 1.0? I found a whole lot of broken messages in an old mailbox of mine and when I looked closer at it, it seemed like Dovecot was ignoring the Content-Length header and truncating the messages at the first embedded From_ line instead. Figuring that this probably just was a fluke / old bug, I erased the index and upgraded Dovecot, first to 1.0.10, then to 1.1.beta14, but I'm still seeing the bug -- and it's even gotten worse! Earlier versions would just quietly split the message at the From_ line, but 1.1.beta14 is more diligent and will complain with: > dovecot[44795]: IMAP(lennart): FETCH for mailbox Drafts UID 196 got > too little data: 590 vs 719 before dropping the connection and exiting... which cause Thunderbird to reopen the connection and reissue the same request... which will cause Dovecot to complain and exit again... etc ad nauseam... It's trivial to reproduce too -- just mail yourself a message with a valid From_ line in it (assuming that your delivery system isn't doing >From-escaping), or put it in a draft plain text message and save it. Then go and look in your Drafts folder... (Like this one: "From someone Tue Jan 29 03:15:52 2008") Just be sure that you can stop your spinning mail client and/or server if you're using 1.1.beta14... --- OK, I am definitely not very familiar with the Dovecot source, but I spent some time trying to track this down today and it looks to me like i_stream_raw_mbox_read is being asked to find the end of the message (at the From_ line) before mbox_sync_parse_next_mail has had a chance to parse the Content-Length header and find out how big it is. Maybe someone who knows the code better can take a look at it and tell me if I'm barking up the totally wrong tree... Thanks, --Lennart

3 5

[Dovecot] Mailbox locking issue?
by Mark Sapiro 04 May '08

04 May '08

Here's my configuration. dovecot -n # 1.0.7: /usr/local/etc/dovecot.conf protocols: pop3 ssl_disable: yes disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/pop3-login mail_extra_groups: mail mail_location: /var/spool/mail/%u mbox_write_locks: fcntl dotlock The above setting was just added in an attempt to resolve the issue. It was default before. mail_executable: /usr/local/libexec/dovecot/pop3 mail_plugin_dir: /usr/local/lib/dovecot/pop3 pop3_uidl_format: %08Xu%08Xv auth default: mechanisms: plain apop login worker_max_count: 5 passdb: driver: passwd-file args: /usr/local/etc/dovecot.passwd passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix Postfix 2.3.3 mailbox_delivery_lock = fcntl, dotlock The problem is the following. A user uses fetchmail via an ssh tunnel to download mail via pop3 every 5 minutes. This normally works fine. perhaps once a week or so, maillog shows that Postfix delivers a message while fetchmail is retrieving. Here is one such. Apr 8 19:07:20 sbh16 dovecot: pop3-login: Login: user=<xxx>, method=PLAIN, rip=72.52.113.36, lip=72.52.113.36, secured Apr 8 19:07:22 sbh16 postfix/local[13745]: D959169046B: to=xxx(a)example.com>, orig_to=<xxx(a)example.org>, relay=local, delay=16, delays=15/0/0/1.1, dsn=2.0.0, status=sent (delivered to mailbox) Apr 8 19:07:22 sbh16 dovecot: POP3(xxx): Disconnected: Logged out top=2/15995, retr=0/0, del=2/2, size=15985 Five minutes later when fetchmail comes back, we see Apr 8 19:12:42 sbh16 dovecot: POP3(xxx): mbox sync: Expunged message reappeared in mailbox /var/spool/mail/xxx (UID 4683 < 4684, seq=2, idx_msgs=0) Apr 8 19:12:42 sbh16 dovecot: pop3-login: Login: user=<xxx>, method=PLAIN, rip=72.52.113.36, lip=72.52.113.36, secured Apr 8 19:12:46 sbh16 dovecot: POP3(xxx): mbox sync: UID inserted in the middle of mailbox /var/spool/mail/xxx (4684 > 4683, seq=2, idx_msgs=1) Apr 8 19:12:46 sbh16 dovecot: POP3(xxx): Disconnected: Logged out top=1/46375, retr=0/0, del=1/1, size=46365 >From now on, as messages are added to the mailbox by Postfix, fetchmail keeps retrieving both old and new messages. i.e. even though dovecot's log says del 1/1 (or del n/n) no messages are actually deleted from the mailbox. This situation continues until someone accesses the mailbox locally (e.g. via mutt through a local shell) and deletes the old messages. As noted above, this has been observed a few times with dovecot's default mbox_write_locks. I have since changed that to mbox_write_locks: fcntl dotlock to be consistent with Postfix's mailbox_delivery_lock = fcntl, dotlock I don't know if that will fix the problem or not, but it seems the issue with default dovecot mbox_write_locks should have been deadlock and time out rather than what we see. Also it seems that dovecot should recover from the 'corrupt' mailbox and fix it rather than just continuing to encounter the same problem. Any ideas or advice? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 4

[Dovecot] Sieve 1.1.2 -> special chars
by mikkel＠euro123.dk 04 May '08

04 May '08

I'm unable to use local chars in vacation/auto replies with sieve. .dovecot.sieve.err gives the same error as when there's a parse error. Example of error: line 25: string 'æ ø å ö ä ü é á. ': My dovecot is v. 1.1b5 and sieve is v. 1.1.2. This issue was also present with dovecot 1.0.x Is there a way around this? - Mikkel

3 2

[Dovecot] Quota not showing correctly after conversion (mb2md.pl)
by k bah 04 May '08

04 May '08

-- SCENARIO -- I'm using dovecot-1.0.5-6 on opensuse 10.3. I use the maildir quota plugin, and I'm converting mailboxes from UW IMAP (mbox) to Maildir format using that mb2md.pl script (3.10 I guess). My migration happens like this: - if a user receives mail (before ever using the new mail system), it goes to his/her inbox, dovecot proceeds normally, like when a user exists, but never received mails. Quota is shown ok. - if a user logs in before receiving any new mail, then, /var/spool/data/USER/new is created, and all converted messages are copied to that dir. If that user had deleted messages (some users mark messages as deleted, others delete for real), then a /var/spool/data/USER/.Trash/new dir is created, and messages are put there too. No other files/dirs (dovecot-uidlist maildirsize cur tmp), are created by "me" or the migration scripts. Index files (/var/spool/index/USER) are not created neither. They are created by dovecot itself. I have a problem, when a user has mails that exceed the limit in mb's (I set just size limit, no messages limit), neither "getquotaroot" (via telnet to port 143), neither webmail software (horde) show the quota usage correctly (I'm not talking about new mail, talking about those messages copied, when they exceed the size limit). Just after trying to delete messages it is shown (gives me an error message, quota exceeded, since Trash is not excluded from quota limits, for now, I'll exclude it later). This server has some issues: many old users have lots of mails, in thousands (1000-15000). Some have 50mb to 150mb of mails. The migration is part of the cleanup process. --- MY QUESTIONS --- Is it possible that the migration order (creating dirs in that way) is causing the quota usage to be shown wrong? Is it possible that the amount of messages and the size can be the cause? Misconfiguration? Other causes? How can I manually tell dovecot to recalculate the quota, after copying messages (when a user logs in, and didn't receive new mails on the new mail system, but had messages big enough to exceed the quota limit) from the old system? Should I run any commands to rebuild indexes or any operation to keep information integrity about the mails/mailboxes to dovecot, after copying the converted messages? many thanks = -- Powered by Outblaze

2 1

[Dovecot] Master user and missing folders
by Anders 04 May '08

04 May '08

Hello again. I am trying to use a master user. The login works. However, a LIST command gives me only the INBOX. Is this to be expected? When logging in normally, I do get the full list of folders. (using Dovecot 1.1 rc3.) Anders.

2 7

[Dovecot] Expire plugin isn't working
by Raphael Bittencourt S. Costa 04 May '08

04 May '08

Hello all, I followed the wiki to configured the expire plugin on dovecot-1.1rc4 and it isn't working. No error messages on logs or on the execution of "dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool". mailserver02:~ # dovecot -n # 1.1.rc4: /etc/dovecot//dovecot.conf syslog_facility: local1 protocols: imap pop3 ssl_disable: yes disable_plaintext_auth: no shutdown_clients: no login_dir: /usr//var/run/dovecot/login login_executable(default): /usr//libexec/dovecot/imap-login login_executable(imap): /usr//libexec/dovecot/imap-login login_executable(pop3): /usr//libexec/dovecot/pop3-login login_process_per_connection: no login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_process_size: 128 login_processes_count: 30 login_max_processes_count: 1024 login_max_connections: 512 max_mail_processes: 10240 mail_max_userip_connections(default): 20 mail_max_userip_connections(imap): 20 mail_max_userip_connections(pop3): 10 mail_uid: 1033 mail_gid: 1033 mail_location: maildir:%h/Maildir mail_executable(default): /usr//libexec/dovecot/imap mail_executable(imap): /usr//libexec/dovecot/imap mail_executable(pop3): /usr//libexec/dovecot/pop3 mail_plugins(default): expire quota imap_quota autocreate mail_plugins(imap): expire quota imap_quota autocreate mail_plugins(pop3): quota mail_plugin_dir(default): /usr//lib/dovecot/imap mail_plugin_dir(imap): /usr//lib/dovecot/imap mail_plugin_dir(pop3): /usr//lib/dovecot/pop3 pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %f pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: mechanisms: plain login cache_size: 20480 cache_ttl: 300 cache_negative_ttl: 0 passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 438 plugin: quota: maildir quota_rule: *:bytes=10240 quota_rule2: *:messages=30000 autocreate: SPAM.E Spam autocreate2: SPAM.Marcar Spam autocreate3: SPAM autocreate4: SPAM.Desmarcar Spam expire: Trash 1 SPAM 10 SPAM/* 10 Sent 30 expire_dict: proxy::expire auth_socket_path: /var/run/dovecot/auth-master dict: expire: db:/usr/lib/dovecot/expidb/expire.db

3 13

[Dovecot] Fatal: Running as root isn't permitted
by Juan Asensio Sánchez 04 May '08

04 May '08

Hi! It compiles perfect now and works fine without returning the uid in the ldap query. What about the inclusion of the antispam and autocreate plugins in the (source) debian package? Thank you very much. 2008/5/4 Timo Sirainen <tss(a)iki.fi>: > On Sun, 2008-05-04 at 15:19 +0200, Juan Asensio Sánchez wrote: > > mech-gssapi.o: In function `gssapi_krb5_userok': > > /root/tmp/dovecot-1.1.rc4/src/auth/mech-gssapi.c:325: undefined > > reference to `krb5_get_error_message' > > This fixes it: http://hg.dovecot.org/dovecot-1.1/rev/ad0f32abda6d > >

1 0

[Dovecot] The CONTROL problem with fs quotas.
by Jorgen Lundman 04 May '08

04 May '08

dovecot-1.0.10 on Solaris 10 x86, with NFS. We currently have the problem that when customers go over their disk quota, the dovecot-uidlist file stops them from logging in. In the interest of solving it, we considered moving the CONTROL file to a different file system. But: 1: Using CONTROL= with "%u" will quite quickly run into the problem that the directory will fill up, and hit the maximum (around 32k) or so. We have 170k users so far, and more are being imported. You could cull some from crontab or similar, but is generally a poor way to solve a problem. 2: We attempted to use CONTROL= with "%h", which would give us full directory hashing, however, we found that when it creates all the directories, it does so as the uid of the first user. So all following users (that hash in at least one part) will fail. Most likely because dovecot has already setuid()ed. Pre-creating the hashes is not feasible. 3: Using 2, we could change umask to "0000" and hence it does not matter than all directory hash node are owned by a user's id, but this unfortunately includes all other directories, and having all directories at 777 is not an option. Now, naturally, I can easily patch dovecot to call mkdir_parents with 0777 only for CONTROL. But I would rather not have local patches, meaning upgrades will be difficult. It would be preferable if: => Dovecot would call "mkdir_parents" on "%h" _before_ setuid, but create the final directory after. OR, Dovecot support (at least 2 level) directory hashing when specifying CONTROL=%1/%2/%u OR, The ability to disable dovecot-uidlist (in memory is fine). (Setting INDEX=memory does not work for CONTROL=). Are there other solutions out there? -- Jorgen Lundman | <lundman(a)lundman.net> Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell) Japan | +81 (0)3 -3375-1767 (home)

2 2

[Dovecot] Fatal: Running as root isn't permitted
by Juan Asensio Sánchez 04 May '08

04 May '08

Hi I am running Dovecot 1.1RC4, built from the source packages at the debian repositories (http://xi.rename-it.nl/debian/) When a message is delivered through the LDA, i get this error: deliver(******@*****): May 02 14:43:28 Fatal: Running as root isn't permitted These are my config files: master.cf: ======================================= # Dovecot dovecot unix - n n - - pipe flags=DRhu user=dovecot:mail argv=/usr/lib/dovecot/deliver -d $recipient ======================================= dovecot.conf ======================================= protocols = imap pop3 listen = 0.0.0.0 login_greeting = Servidor de correo para preparado. disable_plaintext_auth = no log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot-info.log mail_uid = mailuser mail_gid = mailgroup # IMAP configuration protocol imap { mail_plugins = quota imap_quota } # POP3 configuration protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } # LDA configuration protocol lda { log_path = /var/log/dovecot-lda.log postmaster_address = postmaster auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota cmusieve global_script_path = /home/vmail/globalsievesrc } # LDAP authentication auth default { mechanisms = plain login passdb ldap { args = /etc/dovecot/dovecot-ldap.conf } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } socket listen { master { path = /var/run/dovecot/auth-master mode = 0660 user = dovecot group = mail } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } ======================================= dovecot-ldap.conf ======================================= hosts = 127.0.0.1 auth_bind = yes ldap_version = 3 base = dc=*******,dc=local scope = subtree user_attrs = =home=/var/vmail/%d/%n,mailuserquota=quota=maildir:storage,=mail=maildir:/var/vmail/%d/%n/Maildir user_filter = (&(objectClass=mailAccount)(mail=%u)(mailenable=OK)) pass_attrs = mail=user,userPassword=password pass_filter = (&(objectClass=mailAccount)(mail=%u)(mailenable=OK)) default_pass_scheme = CRYPT ======================================= The folder /var/vmail has these permissions: pdc-server:~# ls -la /var/vmail/ total 16 drwxrwx--- 3 mailuser mailgroup 4096 2008-05-02 14:34 . drwxr-xr-x 16 root root 4096 2008-05-02 14:14 .. Any idea? Thanks in advance.

2 4