May 2009 - dovecot - dovecot.org

[Dovecot] migration and conversion from courier
by Giuliano Gavazzi 18 May '09

18 May '09

Dear all, back in January I posted about migration from courier and in particular on the script courier-dovecot-migrate.pl. That script had a bug on BSD (or should I say not-Linux?) type systems*. On FreeBSD (6.1) and MacOSX (10.5) it appended a lot of binary garbage at the end of dovecot-keywords files. On a couple of systems I had not actually run the script with the -- convert option, as I had misunderstood the procedure, so those systems, ironically, are the ones that I migrated better... Now to the issue. My final question at the time was (http://dovecot.org/pipermail/dovecot/2009-January/036292.html) can one delete the dovecot-keywords and rerun the script (leaving alone the dovecot-uidlist)? There is a problem if this cannot be done, as those files have the following structure: 0 flag0 1 flag1 ... m binary garbage [optionally followed by more lines like the last] and if a new keyword is added, the file ends up like (this is copy and paste example): 0 JunkRecorded 1 $NotJunk 2 $Junk 3 Redirected 4 ^A<F0># 5 <B0><96># 6 $Forwarded It does not look like a nice thing to me. I doubt though that the script will cope with flags correctly, as in principle will get the flags from an out of date courierimapkeywords, thus missing any new ones. Perhaps the only solution is to have a new script that deletes the garbage lines and renumbers the following ones, so that the above example would become: 0 JunkRecorded 1 $NotJunk 2 $Junk 3 Redirected 4 $Forwarded and then renames all *g files to *e. Thoughts? Giuliano * This bug was fixed back then by Timo who replaced the script on the site http://www.dovecot.org/tools/courier-dovecot-migrate.pl. But there's no note of the change on the site nor was an announcement made to the list, AFAIK.

2 4

[Dovecot] Quota over NFS
by Flavio Costa 18 May '09

18 May '09

I have accidentaly reply only to Timo, so I'm forwarding to discussion back to the list. Sorry! ---------- Forwarded message ---------- From: Timo Sirainen <tss(a)iki.fi> Date: Mon, May 11, 2009 at 3:12 PM Subject: Re: [Dovecot] Quota over NFS To: Flavio Costa <flavio.cdc(a)gmail.com> On Mon, 2009-05-11 at 15:03 -0300, Flavio Costa wrote: > I get that the server doesn't reply correctly to Dovecot, but the > thing is: why does the LDA works properly? > > May 6 13:15:35 SERVER_NAME deliver(USER_TO): > msgid=<eb4d7eac0905060915y130b9af6i93567e198659d5f7(a)mail.gmail.com>: rejected: Not enough disk space Because it's the NFS server that enforces the quota here. You'd get this error even if you didn't load quota plugin at all. Also note that it's saying "Not enough disk space", not "Quota exceeded" which is the error message from Dovecot v1.0's quota plugin. > And yet, why does 'quota <user>' replies correctly and Dovecot > doesn't? Looking at quota-tools sources, it seemed to ignore the active-check entirely. So I don't know if Dovecot should ignore it too.. > They all obtain the info from the rpc.rquotad running on the > fileserver, right? Yeah. -- Flávio Coutinho da Costa

2 8

[Dovecot] expire plugin error
by Michael Lechtermann 18 May '09

18 May '09

Hi, I am using the db backend for the expire plugin and found the following error message in the logfile: dovecot: Jan 22 10:10:55 Error: dict: secondary db: unable to allocate space from the buffer cache dovecot: Jan 22 10:10:55 Error: dict: Failed to initialize dictionary 'expire' dovecot: Jan 22 10:10:55 Error: IMAP(xxxxxxxxxx): read(/var/dovecot/dict-server) failed: Remote disconnected Anyone know how to fix this? I do not want to use SQL as backend, that would be overkill. # ls -alFh /var/dovecot/dict-server srwxrwxrwx 1 root wheel - 0B Jan 22 10:08 /var/dovecot/dict-server= # dovecot -n # 1.1.8: /etc/dovecot.conf # OS: OpenBSD 4.4 sparc64 ffs base_dir: /var/dovecot/ log_path: /var/log/dovecot.log protocols: imaps pop3s listen: xxxxxxxxxxxx ssl_ca_file: /etc/ssl/xxxxxxxxxxxx.ca ssl_cert_file: /etc/ssl/xxxxxxxxxxxx.crt ssl_key_file: /etc/ssl/private/xxxxxxxxxxxx.key login_dir: /var/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_user: _dovecot mail_max_userip_connections(default): 25 mail_max_userip_connections(imap): 25 mail_max_userip_connections(pop3): 5 first_valid_uid: 10000 last_valid_uid: 19999 first_valid_gid: 100 last_valid_gid: 100 mail_gid: 100 mail_location: mbox:/var/data/mail/%u/mbox:INBOX=/var/data/mail/%u/mbox/inbox:INDEX=/var/dovecot/index/%u mbox_write_locks: fcntl mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): expire imap_quota quota trash zlib mail_plugins(imap): expire imap_quota quota trash zlib mail_plugins(pop3): expire quota zlib mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: passdb: driver: bsdauth userdb: driver: passwd socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 plugin: quota: fs:user:noenforcing:mount=/var/data trash: /etc/dovecot-trash.conf expire: Trash 14 expire_dict: proxy:/var/dovecot/dict-server:expire dict: expire: db:/var/lib/dovecot/expire.db

3 5

[Dovecot] pine vs dovecot-zlib (mbox.gz)
by Darabos Edvárd Konrád 18 May '09

18 May '09

Hello! Always reproducible pine vs dovecot-imap-mbox.gz scene has weird errors. - Initial message is OK: [Folder "SAVE.gz" opened with 211 messages READONLY] - Some mails at the end are listed correctly, others are listed with empty sender/size/subject (just numbering). - All mails have no bodies. Full header command produces total blank screen. Even before any error message. Error messages appear during navigation: [{localhost/imap/user="nil"}SAVE.gz : Missing IMAP reply key: )] [MAIL FOLDER "SAVE.gz" CLOSED DUE TO ACCESS ERROR] Seamonkey works with both mbox-es and mbox.gz-s. Mbox-es are working with both SeaMonkey and pine. Technical environment description follows... Thanks, Nil. PS: Tried to gzip other folder, same results. -- 8< -- Dovecot version: 1.0.15 Linux version 2.6.26-1-amd64 (Debian 2.6.26-13) (waldi(a)debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-24)) #1 SMP Sat Jan 10 17:57:00 UTC 2009 Debian GNU/Linux 5.0 (lenny) Filesystem: ext3 -- 8< -- ~/.pinerc (all other lines are comments or equal to nothing) smtp-server=localhost inbox-path={localhost}INBOX default-fcc={localhost}Sent feature-list=enable-full-header-and-text, enable-full-header-cmd, enable-8bit-esmtp-negotiation, enable-8bit-nntp-posting character-set=utf-8 folder-collections=IMAP {localhost}[] last-time-prune-questioned=109.2 last-version-used=5.10 ssh-path=/usr/local/bin/imapd.sh color-style=use-termdef current-indexline-style=flip-colors titlebar-color-style=default normal-foreground-color=255,255,255 normal-background-color=000,000,000 meta-message-foreground-color=255,255,255 meta-message-background-color=000,000,255 index-opening-foreground-color=000,000,000 index-opening-background-color=000,255,255 -- 8< -- /usr/local/bin/imapd.sh (just tried to redirect server stderr, did not helped): #!/bin/bash /usr/lib/dovecot/imap 2> /dev/null -- 8< -- # 1.0.15: /etc/dovecot/dovecot.conf Warning: mail_extra_groups setting was often used insecurely so it is now deprecated, use mail_access_groups or mail_privileged_group instead log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3s listen(default): localhost listen(imap): localhost listen(pop3): * ssl_listen: * login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_extra_groups: mail mail_access_groups: mail mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins: zlib mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: passdb: driver: pam userdb: driver: passwd -- 8< --

2 2

[Dovecot] expire plugin + dovecot 1.1.7
by Georg Käfer 18 May '09

18 May '09

Hi, testing expire plugin I get an error message "User lookup failed": /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool -test Error: userdb(lukas(a)manual.at) didn't return uid Info: User lookup failed: lukas(a)manual.at Info: gkaefer2(a)manual.at/INBOX.Sent: stop, expire time in future: Wed Mar 31 14:38:33 2010 Reviewing the expires database I see simmilar entries like for gkaefer2(a)manual.at (which is working) lukas(a)manual.at/INBOX.Trash 1241181173 lukas(a)manual.at/INBOX 1270238935 lukas(a)manual.at/INBOX.Sent 1271417514 Dovecot -n # 1.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-gentoo-r3 i686 Gentoo Base System release 1.12.11.1 ext3 syslog_facility: local0 protocols: pop3 pop3s imap imaps ssl_cert_file: /etc/ssl/dovecot/xxx.pem ssl_key_file: /etc/ssl/dovecot/xxx.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_process_per_connection: no login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_process_size: 128 login_processes_count: 10 login_max_processes_count: 256 mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 30 first_valid_uid: 8 last_valid_uid: 8 first_valid_gid: 12 last_valid_gid: 12 mail_privileged_group: mail mail_uid: mail mail_gid: mail mail_location: maildir:/var/mail/%d/%n/Maildir/:INDEX=/var/mail/%d/%n/indexes mail_debug: yes mail_full_filesystem_access: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota trash expire mail_plugins(imap): quota imap_quota trash expire mail_plugins(pop3): quota expire mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: mail group: mail plugin: quota: maildir:User quota quota_rule: *:storage=500M quota_rule2: *:messages=5000 quota_rule3: Trash:storage=10%% quota_rule4: Spam:storage=20%% quota_warning: storage=90%% /usr/bin/quota-warning.sh 90 quota_warning2: storage=70%% /usr/bin/quota-warning.sh 70 quota_warning3: messages=70%% /usr/bin/quota-warning.sh 70 trash: /etc/dovecot/dovecot-trash.conf expire: Trash 30 Trash/* 30 INBOX.Trash 30 Spam 30 Junk-E-Mail 30 INBOX.Sent 365 Sent 365 INBOX 365 expire_dict: proxy::expire dict: expire: mysql:/etc/dovecot/dovecot-dict-expire.conf dovecot-dict-expire.conf: connect = host=localhost dbname=postfixadmin user=postfixadmin password=xxx # dovecot v1.1 only: table = expires select_field = expire_stamp where_field = mailbox username_field = not_used and users home (lukas@] ls -ltra shows: -rw------- 1 mail mail 0 Mar 31 10:25 dovecot-uidvalidity.49d1d387 drwx------ 4 mail mail 4096 Mar 31 10:25 .. -rw------- 1 mail mail 36 Mar 31 11:53 subscriptions -rw------- 1 mail mail 8 Mar 31 14:31 dovecot-uidvalidity drwx------ 5 mail mail 4096 Apr 1 14:33 .INBOX.Trash drwx------ 2 mail mail 4096 Apr 2 22:09 tmp -rw------- 1 mail mail 161 Apr 2 22:09 dovecot-uidlist drwx------ 2 mail mail 4096 Apr 3 13:42 new drwx------ 2 mail mail 4096 Apr 3 13:42 cur drwx------ 8 mail mail 4096 Apr 16 11:32 . drwx------ 5 mail mail 4096 Apr 16 11:35 .INBOX.Drafts -rw------- 1 mail mail 64 Apr 16 13:31 maildirsize drwx------ 5 mail mail 4096 Apr 22 12:44 .INBOX.Sent any idea what is causing the "didn't return uid" / "User lookup failed:" ? many thanks, Liebe Gruesse Georg Kaefer

2 1

[Dovecot] dovecot 1.2 plugin virtual and shared namespace with thunderbird
by Robert Schetterer 18 May '09

18 May '09

Hi Timo, using latest dovecot 1.2 and latest thunderbird i noticed if using virtual plugin like here with mysql http://wiki.dovecot.org/Plugins/Virtual everthing works like expected until i add a shared namespace Tb then looses its subscriptions under imap to the private namespace only the shared namespace gets shown up , any idea? I dont see something special in the logs why this happens -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

2 16

[Dovecot] latest update dovecot 1.2 breaks Error: imap dump-capability
by Robert Schetterer 18 May '09

18 May '09

Hi Timo, lates update 1.2 seems to break something it gives me Fio_loop_handle_add: epoll_ctl(1, 0): Operation not permitted Error: imap dump-capability: Couldn't read capability (got 0 bytes) Fatal: Invalid configuration in /etc/dovecot/dovecot.conf at restart, nothing in the conf changed Any Idea? Regards

2 4

[Dovecot] Problem with domain part in user_query in dovecot 1.1.14
by Konstantin 18 May '09

18 May '09

Hi. I'm using dovecot 1.1.14 with pgsql. I tried to configure dovecot to get the domain part in user_query by specifying different variants of auth_username_format variable such as %Lu, %Lu@%Ld, %Lu-at-%Ld and so on... So dovecot gets domain part in password_query, but not in user_query. I noticed the username is changed like this: kostas(a)mgupb.net->kostas in log. For some reason it happens before user_query so there is no domain part already. So one thing that had worked out for me was commenting the below code in src/auth/auth-request.c: if (strcmp(request->user, value) != 0) { /* remember the original username for cache */ if (request->original_username == NULL) { request->original_username = p_strdup(request->pool, request->user); } auth_request_log_debug(request, "auth", "username changed %s -> %s", request->user, value); request->user = p_strdup(request->pool, value); } I know that it is not so good, but it is working for me, now everything is fine, but I want it working without code deformations. What should I do? Thanks. P.S. Such things were in 1.1.11 from which I upgraded to 1.1.14 last night. Dovecot config: # 1.1.14: /usr/local/etc/dovecot.conf # OS: FreeBSD 7.0-RELEASE amd64 protocols: imap imaps pop3 pop3s ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_privileged_group: mail maildir_copy_preserve_filename: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login digest-md5 user: vmail username_format: %Lu verbose: yes debug: yes passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix dovecot-sql.conf: driver = pgsql connect = host=localhost dbname=hosting_db user=hosting_db password=adsfweae password_query = SELECT NULL AS password, 'Y' as nopassword, userid AS user FROM main.mail_base WHERE userid = '%n' AND password = '%w' and domain='%d' #password_query = SELECT password, userid AS user FROM main.mail_base WHERE user id = '%n' and domain='%d' #user_query = SELECT '/var/mail/vhosts/' || domain || '/' || userid as mail, '/v ar/mail/vhosts/' || domain || '/' || userid as home, uid, gid FROM main.mail_bas e WHERE userid = '%n' and domain='%d' user_query = SELECT '/var/mail/vhosts/' || domain || '/' || userid as mail, '/va r/mail/vhosts/' || domain || '/' || userid as home, uid, gid FROM main.mail_base WHERE userid = '%n' and domain='%d' default_pass_scheme = PLAIN -- Konstantin <kostas(a)mgupb.net>

2 2

[Dovecot] "Broken mailbox list" with virtual plugin
by Matthijs Kooijman 18 May '09

18 May '09

Hi, I'm using the virtual plugin for a virtual inbox (with dovecot 1.2, r8980), whose dovecot-virtual file looks as follows: INBOX INBOX/Lists INBOX/Bulk INBOX/MaybeSpam Folders/TODO all Everything seems to work fine, but I get the following error in my logfiles: IMAP(matthijs): virtual /home/matthijs/Mail/virtual/INBOX: Broken mailbox list Some investigation of the source code tells me that this error means that there was corrupt data in the index. In particular, the ext_data read from the index was not long enough to contain all mailbox names. Adding some debug code confirms this: ext_size: 187 ext_data: \b0\5\0\0\5\0\0\0E\0\0\0\17\20w\9c\1\0\0\0\5\0\0\0\e9\af\e4I\92\1\0 \0\e7\2\0\0\0\0\0\0\7\0\0\0\b\0\0\0\e8\af\e4IO\3\0\0\5c\5\0\0\0\0\0 \0\8\0\0\0\a\0\0\0\ed\af\e4I\e6\0\0\0j\1\0\0\0\0\0\0\9\0\0\0\f\0\0 \0\3a\b0\e4I1\0\0\0e\0\0\0\0\0\0\0E\0\0\0\c\0\0\0\eb\af\e4I\c8\0\0 \0e\0\0\0\0\0\0\0INBOXINBOX/ListsINBOX/BulkINBOX/MaybeSpamFolders/TO ext_hdr: change_counter=1456 mailbox_count=5 highest_mailbox_id=69 search_args_crc32=2625052695 mailbox: id=1 name_len=5 uid_validity=1239724009 next_uid=402 higest_modseq=743 mailbox name: INBOX mailbox: id=7 name_len=11 uid_validity=1239724008 next_uid=847 higest_modseq=1372 mailbox name: INBOX/Lists mailbox: id=8 name_len=10 uid_validity=1239724013 next_uid=230 higest_modseq=362 mailbox name: INBOX/Bulk mailbox: id=9 name_len=15 uid_validity=1239724090 next_uid=49 higest_modseq=101 mailbox name: INBOX/MaybeSpam mailbox: id=69 name_len=12 uid_validity=1239724011 next_uid=200 higest_modseq=101 virtual /home/matthijs/Mail/virtual/INBOX: Broken mailbox list As you can see in the ext_data dump, the last mailbox name (Folders/TODO) is cut short. I'm not so sure what caused this error. I've just tried removing the index files, which seems to fix the error message. The index file gets recreated and my debug statements show proper output. Another solution is adding an extra file to the folder list, in which case the index will get fixed as well. Perhaps it would be good to recreate the index file when this error is found, instead of complaining? Also, from the code (virtual_sync_ext_header_read in src/virtual/virtual-sync.c) it seems that the last mailbox should be skipped because of this error, but the mailbox works fine. I'm probably not looking closely enough :-) I haven't been able to reliably break the index file with some quick testing (reconnecting, restarting and modifying the mailbox list), but I did manage to break it once (by removing a mailbox from halfway the list, and then adding it again). I couldn't reproduce that, though, since removing a mailbox doesn't always seem to recreate the index for some reason... I've also just upgraded to the latest version, so if this is a bug fixed between r8980 and r9018, then you'll probably not hear from me again :-) I hope all this info helps a bit, I'm available for further debugging and testing (also on IRC, nick is blathijs). Gr. Matthijs

2 2

Re: [Dovecot] Fail2Ban and the Dovecot log
by Lou Duchez 18 May '09

18 May '09

Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line: http://www.voip-info.org/wiki/view/Fail2Ban+(with+iptables)+And+Asterisk

5 16