April 2008 - dovecot - dovecot.org

[Dovecot] Mailbox locking issue?
by Mark Sapiro 04 May '08

04 May '08

Here's my configuration. dovecot -n # 1.0.7: /usr/local/etc/dovecot.conf protocols: pop3 ssl_disable: yes disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/pop3-login mail_extra_groups: mail mail_location: /var/spool/mail/%u mbox_write_locks: fcntl dotlock The above setting was just added in an attempt to resolve the issue. It was default before. mail_executable: /usr/local/libexec/dovecot/pop3 mail_plugin_dir: /usr/local/lib/dovecot/pop3 pop3_uidl_format: %08Xu%08Xv auth default: mechanisms: plain apop login worker_max_count: 5 passdb: driver: passwd-file args: /usr/local/etc/dovecot.passwd passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix Postfix 2.3.3 mailbox_delivery_lock = fcntl, dotlock The problem is the following. A user uses fetchmail via an ssh tunnel to download mail via pop3 every 5 minutes. This normally works fine. perhaps once a week or so, maillog shows that Postfix delivers a message while fetchmail is retrieving. Here is one such. Apr 8 19:07:20 sbh16 dovecot: pop3-login: Login: user=<xxx>, method=PLAIN, rip=72.52.113.36, lip=72.52.113.36, secured Apr 8 19:07:22 sbh16 postfix/local[13745]: D959169046B: to=xxx(a)example.com>, orig_to=<xxx(a)example.org>, relay=local, delay=16, delays=15/0/0/1.1, dsn=2.0.0, status=sent (delivered to mailbox) Apr 8 19:07:22 sbh16 dovecot: POP3(xxx): Disconnected: Logged out top=2/15995, retr=0/0, del=2/2, size=15985 Five minutes later when fetchmail comes back, we see Apr 8 19:12:42 sbh16 dovecot: POP3(xxx): mbox sync: Expunged message reappeared in mailbox /var/spool/mail/xxx (UID 4683 < 4684, seq=2, idx_msgs=0) Apr 8 19:12:42 sbh16 dovecot: pop3-login: Login: user=<xxx>, method=PLAIN, rip=72.52.113.36, lip=72.52.113.36, secured Apr 8 19:12:46 sbh16 dovecot: POP3(xxx): mbox sync: UID inserted in the middle of mailbox /var/spool/mail/xxx (4684 > 4683, seq=2, idx_msgs=1) Apr 8 19:12:46 sbh16 dovecot: POP3(xxx): Disconnected: Logged out top=1/46375, retr=0/0, del=1/1, size=46365 >From now on, as messages are added to the mailbox by Postfix, fetchmail keeps retrieving both old and new messages. i.e. even though dovecot's log says del 1/1 (or del n/n) no messages are actually deleted from the mailbox. This situation continues until someone accesses the mailbox locally (e.g. via mutt through a local shell) and deletes the old messages. As noted above, this has been observed a few times with dovecot's default mbox_write_locks. I have since changed that to mbox_write_locks: fcntl dotlock to be consistent with Postfix's mailbox_delivery_lock = fcntl, dotlock I don't know if that will fix the problem or not, but it seems the issue with default dovecot mbox_write_locks should have been deadlock and time out rather than what we see. Also it seems that dovecot should recover from the 'corrupt' mailbox and fix it rather than just continuing to encounter the same problem. Any ideas or advice? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 4

[Dovecot] Sieve 1.1.2 -> special chars
by mikkel＠euro123.dk 04 May '08

04 May '08

I'm unable to use local chars in vacation/auto replies with sieve. .dovecot.sieve.err gives the same error as when there's a parse error. Example of error: line 25: string 'æ ø å ö ä ü é á. ': My dovecot is v. 1.1b5 and sieve is v. 1.1.2. This issue was also present with dovecot 1.0.x Is there a way around this? - Mikkel

3 2

[Dovecot] Quota not showing correctly after conversion (mb2md.pl)
by k bah 04 May '08

04 May '08

-- SCENARIO -- I'm using dovecot-1.0.5-6 on opensuse 10.3. I use the maildir quota plugin, and I'm converting mailboxes from UW IMAP (mbox) to Maildir format using that mb2md.pl script (3.10 I guess). My migration happens like this: - if a user receives mail (before ever using the new mail system), it goes to his/her inbox, dovecot proceeds normally, like when a user exists, but never received mails. Quota is shown ok. - if a user logs in before receiving any new mail, then, /var/spool/data/USER/new is created, and all converted messages are copied to that dir. If that user had deleted messages (some users mark messages as deleted, others delete for real), then a /var/spool/data/USER/.Trash/new dir is created, and messages are put there too. No other files/dirs (dovecot-uidlist maildirsize cur tmp), are created by "me" or the migration scripts. Index files (/var/spool/index/USER) are not created neither. They are created by dovecot itself. I have a problem, when a user has mails that exceed the limit in mb's (I set just size limit, no messages limit), neither "getquotaroot" (via telnet to port 143), neither webmail software (horde) show the quota usage correctly (I'm not talking about new mail, talking about those messages copied, when they exceed the size limit). Just after trying to delete messages it is shown (gives me an error message, quota exceeded, since Trash is not excluded from quota limits, for now, I'll exclude it later). This server has some issues: many old users have lots of mails, in thousands (1000-15000). Some have 50mb to 150mb of mails. The migration is part of the cleanup process. --- MY QUESTIONS --- Is it possible that the migration order (creating dirs in that way) is causing the quota usage to be shown wrong? Is it possible that the amount of messages and the size can be the cause? Misconfiguration? Other causes? How can I manually tell dovecot to recalculate the quota, after copying messages (when a user logs in, and didn't receive new mails on the new mail system, but had messages big enough to exceed the quota limit) from the old system? Should I run any commands to rebuild indexes or any operation to keep information integrity about the mails/mailboxes to dovecot, after copying the converted messages? many thanks = -- Powered by Outblaze

2 1

[Dovecot] Master user and missing folders
by Anders 04 May '08

04 May '08

Hello again. I am trying to use a master user. The login works. However, a LIST command gives me only the INBOX. Is this to be expected? When logging in normally, I do get the full list of folders. (using Dovecot 1.1 rc3.) Anders.

2 7

[Dovecot] Expire plugin isn't working
by Raphael Bittencourt S. Costa 04 May '08

04 May '08

Hello all, I followed the wiki to configured the expire plugin on dovecot-1.1rc4 and it isn't working. No error messages on logs or on the execution of "dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool". mailserver02:~ # dovecot -n # 1.1.rc4: /etc/dovecot//dovecot.conf syslog_facility: local1 protocols: imap pop3 ssl_disable: yes disable_plaintext_auth: no shutdown_clients: no login_dir: /usr//var/run/dovecot/login login_executable(default): /usr//libexec/dovecot/imap-login login_executable(imap): /usr//libexec/dovecot/imap-login login_executable(pop3): /usr//libexec/dovecot/pop3-login login_process_per_connection: no login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_process_size: 128 login_processes_count: 30 login_max_processes_count: 1024 login_max_connections: 512 max_mail_processes: 10240 mail_max_userip_connections(default): 20 mail_max_userip_connections(imap): 20 mail_max_userip_connections(pop3): 10 mail_uid: 1033 mail_gid: 1033 mail_location: maildir:%h/Maildir mail_executable(default): /usr//libexec/dovecot/imap mail_executable(imap): /usr//libexec/dovecot/imap mail_executable(pop3): /usr//libexec/dovecot/pop3 mail_plugins(default): expire quota imap_quota autocreate mail_plugins(imap): expire quota imap_quota autocreate mail_plugins(pop3): quota mail_plugin_dir(default): /usr//lib/dovecot/imap mail_plugin_dir(imap): /usr//lib/dovecot/imap mail_plugin_dir(pop3): /usr//lib/dovecot/pop3 pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %f pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: mechanisms: plain login cache_size: 20480 cache_ttl: 300 cache_negative_ttl: 0 passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 438 plugin: quota: maildir quota_rule: *:bytes=10240 quota_rule2: *:messages=30000 autocreate: SPAM.E Spam autocreate2: SPAM.Marcar Spam autocreate3: SPAM autocreate4: SPAM.Desmarcar Spam expire: Trash 1 SPAM 10 SPAM/* 10 Sent 30 expire_dict: proxy::expire auth_socket_path: /var/run/dovecot/auth-master dict: expire: db:/usr/lib/dovecot/expidb/expire.db

3 13

[Dovecot] The CONTROL problem with fs quotas.
by Jorgen Lundman 04 May '08

04 May '08

dovecot-1.0.10 on Solaris 10 x86, with NFS. We currently have the problem that when customers go over their disk quota, the dovecot-uidlist file stops them from logging in. In the interest of solving it, we considered moving the CONTROL file to a different file system. But: 1: Using CONTROL= with "%u" will quite quickly run into the problem that the directory will fill up, and hit the maximum (around 32k) or so. We have 170k users so far, and more are being imported. You could cull some from crontab or similar, but is generally a poor way to solve a problem. 2: We attempted to use CONTROL= with "%h", which would give us full directory hashing, however, we found that when it creates all the directories, it does so as the uid of the first user. So all following users (that hash in at least one part) will fail. Most likely because dovecot has already setuid()ed. Pre-creating the hashes is not feasible. 3: Using 2, we could change umask to "0000" and hence it does not matter than all directory hash node are owned by a user's id, but this unfortunately includes all other directories, and having all directories at 777 is not an option. Now, naturally, I can easily patch dovecot to call mkdir_parents with 0777 only for CONTROL. But I would rather not have local patches, meaning upgrades will be difficult. It would be preferable if: => Dovecot would call "mkdir_parents" on "%h" _before_ setuid, but create the final directory after. OR, Dovecot support (at least 2 level) directory hashing when specifying CONTROL=%1/%2/%u OR, The ability to disable dovecot-uidlist (in memory is fine). (Setting INDEX=memory does not work for CONTROL=). Are there other solutions out there? -- Jorgen Lundman | <lundman(a)lundman.net> Unix Administrator | +81 (0)3 -5456-2687 ext 1017 (work) Shibuya-ku, Tokyo | +81 (0)90-5578-8500 (cell) Japan | +81 (0)3 -3375-1767 (home)

2 2

[Dovecot] Dovecot 1.1: Ignoring Trash/Junk for maildirsize?
by richs＠whidbey.net 04 May '08

04 May '08

Hi all, We're testing Dovecot 1.1rc3, but have run into a snag with IMAP Quota's and the way Trash and Junk folders can be handled. Our MTA (Exim) quota configuration uses this option: maildir_quota_directory_regex = ^(?:cur|new|\.(?!Junk$)(?!Trash$).*)$ This completely excludes "Junk" and "Trash" from the "maildirsize" file and quota calculations. Dovecot 1.0 could do this using "ignore=Trash" (although only on one folder). Is there any way to duplicate this behavior in Dovecot 1.1? We see that "quota_rule" can be used to set an unlimited quota (storage=0), but for some reason Dovecot still records their tallies in "maildirsize", which breaks the MTA. Thanks! -Rich

2 1

[Dovecot] Patch for zlib and maildir for 1.0.13
by Richard Platel 04 May '08

04 May '08

When using the zlib plugin with maildir and copying with hardlinks, if a compressed message is copied, the 'Z' suffix on the file isn't copied, so the new message isn't uncompressed when it's fetched. I wasn't smart enough to figure out a clean way to carry a file suffix through a copy, so I changed how the zlib-plugin detects if a message is compressed. This patch peeks at the first two bytes of the message looking for the zlib header. -- Richard Platel Tucows

3 3

[Dovecot] 1.1rc4 problem with maildir++ quota and POP3
by Laurent Papier 04 May '08

04 May '08

Hi, it looks like there is a bug (in dovecot or in my config) with quota on maildir mailboxes when accessed via POP3. The maildirsize is not created when not already present (even after a DELE + QUIT command). The quota limit (the first line of maildirsize) is not updated if values are not in sync with those in the database. If maildirsize is present, DELE commands are correctly reported in maildirsize. maildirsize is correctly updated when the same mailbox is accessed in IMAP or mail is delivered with dovecot LDA. Here is my config. Timo 'dovecot -n' does not show the lda config part... # 1.1.rc4: /etc/dovecot.conf protocols: imap pop3 ssl_disable: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: please to meet you login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_processes_count: 100 login_max_processes_count: 512 max_mail_processes: 3072 mail_uid: 1000 mail_gid: 1000 mail_location: maildir:~/ mmap_disable: yes maildir_stat_dirs: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): imap_quota quota mail_plugins(imap): imap_quota quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle imap_client_workarounds(imap): outlook-idle imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls auth default: user: nobody passdb: driver: sql args: /etc/dovecot-mysql.conf userdb: driver: sql args: /etc/dovecot-mysql.conf userdb: driver: prefetch socket: type: listen master: path: /var/run/dovecot/auth-master mode: 432 user: mailonly group: mailonly plugin: quota: maildir quota_rule2: Trash:storage=12288:messages=100 --- SQL queries ---- # prefetch auth password_query = SELECT cpasswd as password, home as userdb_home, concat('*:bytes=',quotas,':messages=',quotai) AS userdb_quota_rule FROM `mail` WHERE email='%u' # for delivery user_query = SELECT home, concat('*:bytes=',quotas,':messages=',quotai) AS quota_rule FROM `mail` WHERE email='%u' -- Laurent Papier - 03 88 75 80 50 Admin. système - SdV Plurimedia - <http://www.sdv.fr/>

2 1

[Dovecot] Global and personal sieve scripts
by Juan Asensio Sánchez 04 May '08

04 May '08

Hi According to Dovecot documentation (): # If there is no user-specific Sieve-script, global Sieve script is # executed if set. (v1.0.1 and older used "global_script_path") #sieve_global_path = Then, if one user has its own script, the global script won't be executed? I would like to execute first the global script (for moving SPAM to the Junk folder), and then the user scripts is they exists. Any idea? Thanks in advance.

2 1