August 2008 - dovecot

[Dovecot] Dovecot 1.1 Quota Plugin Configuration?
by David Reid 07 Aug '08

07 Aug '08

With 1.0 I had the quota plugin set to allow some accounts to have no quota by setting their size (in bytes) to 0 (via mysql). Following the upgrade to 1.1 this no longer seems to work and has resulted in a lot of annoyed people having messages bumped :-( The config I have is as follows... quota = maildir quota_rule = *:storage=15M quota_rule2 = Trash:ignore quota_rule3 = INBOX.Trash:ignore quota_rule4 = INBOX.SpamTrain:ignore quota_rule5 = SpamTrain:ignore In the sql.conf I get the quota for each mailbox using this line user_query = SELECT '/var/vmail/%d/%n' as home, 'maildir:/var/vmail/%d/%n' as mail, xxx AS uid, xxx AS gid, concat('maildir:storage=', mb_quota) AS quota FROM mailboxes WHERE mb_mbname = '%u' AND mb_active = '1' With this setup when I have a mb_quota set to 0 it simply bounces all messages for the mailboxes that should have no quota. Any ideas how I need to fix this? david

1 0

[Dovecot] Experience moving mailboxes from Dovecot 0.99.14 to Dovecot 1.07 => Improvement possible
by Benoît DESLOGES 07 Aug '08

07 Aug '08

Hi, This small mail to share my observation about a recent move of mailboxes between two servers and ask about explanation and/or improvement about UIDL in dovecot. SV1 : Dovecot 0.99.14 / Red Hat Fedore Core 4 SV2 : Dovecot 1.07 / CentOS 5.2 Mailboxes in /var/spool/mail on the twoo servers. Test will be done with outlook express with option "leave message on server" checked. Goal is simply to move users mailboxes from SV1 to SV2 without re-downloading all messages. Try 1: - Stop dovecot on SV2 - Clear all dovecot indexes on SV2 - Rsync of my mailbox - Start dovecot on SV2 - Update pop setting in outlook and send/receive Result : => starting to download 3000 mails... After some debug, I discovered that UIDL where not of the same format => put pop3_uidl_format = %v.%u in dovecot.conf of SV2. Try 2: Same operations as Try1 Result : => starting to download 3000 mails... UIDL's where of the same format but values where not corresponding... Teleting on SV1 and asking UIDL, last value is xxxxxxxxxx.85878. On SV2, xxxxxxxxxx.85879 was the FIRST value of the list. Conclusion for the moment, Dovecot has a problem with the detection of existing UID of the moved mailbox. Comparaison of first header found from the two mailboxes show ... differences ! SV1 give the following : (...) X-UID: 70570 Content-Length: 1561 X-IMAPbase: xxxxxxxxxx 85845 $MDNSent X-Keywords: (...) SV2 give the following (after first POP attempt) : (...) Content-Length: 1561 X-IMAPbase: xxxxxxxxxx 0000089204 $MDNSent X-Keywords: X-UID: 85846 (...) Observation : X-UID: 85846 is not at the same place in the header. Try 3: - Stop dovecot on SV2 - Clear all dovecot indexes on SV2 - Rsync of my mailbox - Edit mailbox and move X-UID header after the X-Keywords header - Start dovecot on SV2 - Update pop setting in outlook and send/receive Result : => OK. Is that normal that dovecot is impacted by the position of the headers ? Maybe some improvement possible ? Maybe theyre is another method to make my migration ? Any observation or comment is welcome... -- Benoît Desloges Network Engineer

7 8

[Dovecot] Fwd: Re: sasl parameters missing (in postfix)
by Daniel Black 07 Aug '08

07 Aug '08

In response to my request for postfix to support dovecot auth arguments I got the forwarded reply. If someone gets around to this before me I won't be offended. ------------ Story is I deployed a webmail with certificate based authentication that substitutes a global master password (http://wiki.dovecot.org/Authentication/MasterUsers) when the certificate matches. The webmail accesses the inbox by imap and reuses the password for smtp through postfix. I configured dovecot sasl authentication to allow a particular global password to be allowed from one IP address of the webmail server. Unfortuanately it seems as though postfix doesn't pass rip= (remote ip) or the other AUTH parameters of the protocol (http://dovecot.org/doc/auth-protocol.txt) Is adding these parameters to postfix's sasl authentication a useful feature request? ---------- Forwarded Message ---------- Subject: Re: sasl parameters missing Date: Thu, 7 Aug 2008 From: Wietse Venema <wietse(a)porcupine.org> To: Daniel Black <daniel.subs(a)internode.on.net> Daniel Black: > Thanks Wietse, > > On Tue, 5 Aug 2008 09:30:44 am Wietse Venema wrote: > > Postfix passes the information in the SMTP client's AUTH command. > > This is how I got the Dovecot extension from Timo. If someone is > > willing to monitor his docs for changes, > > it seems fairly stable. Going off the doc/auth-protocol.txt changelog > Nov 12 2006 lport/rport was added. > Aug 07 2005 changed valid-client-cert to ssl-valid-cert > Oct 22 2004 original documentation > > Current implementation of the authentication server in dovecot seems to ignore > parameters it doesn't understand. > > > then they are welcome to do so. I won't. > > On the basis of this apparent stability and compatibility would you consider > accepting a patch? Yes. No promise, though, that it will be adopted. One consideration is that Postfix does not talk directly to Dovecot, but instead talks to an abstraction layer that is used for both Cyrus SASL and for Dovecot. Obviously, that XSASL abstraction layer must not be made specific to the underlying Cyrus SASL or Dovecot implementation. The solution therefore is not to extend XSASL functions with one extra argument for each Dovecot feature. Apart from being Dovecot-specific, functions with many parameters are difficult to update correctly; compilers can't always tell that two arguments should be swapped. I solved the problem of many-parameter functions by using macros such as TLS_SERVER_START(). This gives more assurance that data is passed correctly, and it less likely to break due to human maintainer error. Wietse ------------------------------------------------------- -- Daniel Black -- Proudly a Gentoo Linux User. Gnu-PG/PGP signed and encrypted email preferred http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x76677097 GPG Signature D934 5397 A84A 6366 9687 9EB2 861A 4ABA 7667 7097

1 0

[Dovecot] Purpose of files under .imap
by Kenneth Porter 07 Aug '08

07 Aug '08

I've got a user with an unusable account. Tbird just sits in a loop endlessly logging in over and over when a particular folder is selected. What files under .imap associated with that folder can be safely deleted? What's stored in all of them? I see all of these: .customflags dovecot.index dovecot.index.cache dovecot.index.log .imap.index .imap.index.data .imap.index.log .imap.index.tree

2 2

[Dovecot] Next message unexpectedly lost
by Kenneth Porter 07 Aug '08

07 Aug '08

Using 1.1.2. I'm getting the above message over and over in the error log. Dovecot drops the connection and the client (Tbird) just keeps retrying in an endless loop. Using Wireshark to sniff the exchange, it looks like the command that fails looks like this: 5 UID fetch 1:9749,9751:9763,9765:9930,9932,9934:9942 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)]) Dovecot seems to be shutting down after receiving this. Mulberry doesn't seem to have the problem, presumably because it's sending a different command sequence that doesn't stroke this. Here's the failing code, in i_stream_raw_mbox_read : if (stream->istream.v_offset - rstream->hdr_offset + new_pos > rstream->mail_size) { /* istream_raw_mbox_set_next_offset() used invalid cached next_offset? */ i_error("Next message unexpectedly lost from %"PRIuUOFF_T, rstream->hdr_offset + rstream->mail_size); rstream->eof = TRUE; rstream->corrupted = TRUE; return -1; } I've tried deleting the associated .imap/folder contents but it must regenerate the same bad data as it happens again.

1 0

[Dovecot] Unlimited quota in 1.1?
by David Reid 07 Aug '08

07 Aug '08

How do I set an unlimited quota for a user in 1.1? Setting it to 0 bytes worked in 1.0, but doesn't seem to work in 1.1 :-( david

2 1

Re: [Dovecot] Dovecot PAM sessions with Maildir
by Sean Smitz 06 Aug '08

06 Aug '08

I able to connect to the mail server with a MUA (Thunderbird) via SSL prior to attempting to set up the session and Maildir options /var/log/audit/audit.log type=USER_AUTH msg=audit(1217992493.198:389): user pid=8022 uid=0 auid=10243 subj=user_u:system_r:dovecot_auth_t:s0 msg='PAM: authentication acct="[me]" : exe="/usr/libexec/dovecot/dovecot-auth" (hostname=::ffff:[client], addr=::ffff:[client], terminal=dovecot res=success)' type=USER_ACCT msg=audit(1217992493.199:390): user pid=8022 uid=0 auid=10243 subj=user_u:system_r:dovecot_auth_t:s0 msg='PAM: accounting acct="[me]" : exe="/usr/libexec/dovecot/dovecot-auth" (hostname=::ffff:[client], addr=::ffff:[client], terminal=dovecot res=success)' type=USER_START msg=audit(1217992493.199:391): user pid=8022 uid=0 auid=10243 subj=user_u:system_r:dovecot_auth_t:s0 msg='PAM: session open acct="[me]" : exe="/usr/libexec/dovecot/dovecot-auth" (hostname=::ffff:[client], addr=::ffff:[client], terminal=dovecot res=failed)' /var/log/maillog Aug 5 23:14:37 mailserv1 dovecot: Dovecot v1.0.7 starting up Aug 5 23:14:38 mailserv1 dovecot: auth(default): new auth connection: pid=8014 Aug 5 23:14:38 mailserv1 dovecot: auth(default): new auth connection: pid=8013 Aug 5 23:14:38 mailserv1 dovecot: auth(default): new auth connection: pid=8015 Aug 5 23:14:46 mailserv1 dovecot: auth(default): new auth connection: pid=8016 Aug 5 23:14:52 mailserv1 dovecot: auth(default): client in: AUTH 1 PLAIN service=IMAP secured lip=::ffff:[server] rip=::ffff:[client] Aug 5 23:14:52 mailserv1 dovecot: auth(default): client out: CONT 1 Aug 5 23:14:52 mailserv1 dovecot: auth(default): client in: CONT<hidden> Aug 5 23:14:52 mailserv1 dovecot: auth(default): pam([me],::ffff:[client]): lookup service=dovecot Aug 5 23:14:52 mailserv1 dovecot: auth(default): pam([me],::ffff:[client]): pam_open_session() failed: Cannot make/remove an entry for the specified session Aug 5 23:14:53 mailserv1 dovecot: auth(default): client out: FAIL 1 user=[me] Aug 5 23:14:53 mailserv1 dovecot: auth(default): client in: AUTH 2 PLAIN service=IMAP secured lip=::ffff:[server] rip=::ffff:[client] resp=<hidden> Aug 5 23:14:53 mailserv1 dovecot: auth(default): pam([me],::ffff:[client]): lookup service=dovecot Aug 5 23:14:53 mailserv1 dovecot: auth(default): pam([me],::ffff:[client]): pam_open_session() failed: Cannot make/remove an entry for the specified session Aug 5 23:14:55 mailserv1 dovecot: auth(default): client out: FAIL 2 user=[me] Aug 5 23:14:58 mailserv1 dovecot: imap-login: Aborted login: user=<[me]>, method=PLAIN, rip=::ffff:[client], lip=::ffff:[server], TLS

2 1

[Dovecot] Maildir Index Question
by Ron Culler 06 Aug '08

06 Aug '08

We have several hundred mailboxes that have thousands of files in them (spam boxes) We currently run a find command and remove the files that are over 7 days old but still have boxes with upto 60k messages in them. I know that dovecot will update the index files for each mailbox when the boxes is accessed by an IMAP client(Webmail using Squirrel)but that can take quite a while to index. What I would like to do is via a cron job after the old file remove is to force dovecot to reindex these boxes. That way when the users login to the webmail interface it is quicker. Does anyone know if a script is available to do this? Thanks Ron Culler

2 1

[Dovecot] Dovecot PAM sessions with Maildir
by Sean Smitz 06 Aug '08

06 Aug '08

I am configuring a new mail server on RHEL 5 x64. I have configured dovecot as follows: ... protocols = imaps ... ssl_cert_file = /etc/pki/dovecot/certs/mailserver.cer ssl_key_file = /etc/pki/dovecot/private/mailserver.pem ... login_process_size = 64 ... mail_location = maildir:~/Maildir ... passdb pam { args = "session=yes cache_key=%u%s dovecot" } ... I'm getting the following error messages in /var/log/maillog: Aug 5 20:57:45 [hostname] dovecot: imap-login: Aborted login: rip=::ffff:128.8.244.15, lip=::ffff:[server], TLS Aug 5 21:01:34 [hostname] dovecot: imap-login: Aborted login: user=<[me]>, method=PLAIN, rip=::ffff:[server], lip=::ffff:[server], TLS I'm getting the following messages in /var/log/audit/audit.log: type=USER_AUTH msg=audit(1217984152.016:347): user pid=7669 uid=0 auid=10243 subj=user_u:system_r:dovecot_auth_t:s0 msg='PAM: authentication acct="[me]" : exe="/usr/libexec/dovecot/dovecot-auth" (hostname=::ffff:[client], addr=::ffff:[client], terminal=dovecot res=success)' type=USER_ACCT msg=audit(1217984152.017:348): user pid=7669 uid=0 auid=10243 subj=user_u:system_r:dovecot_auth_t:s0 msg='PAM: accounting acct="[me]" : exe="/usr/libexec/dovecot/dovecot-auth" (hostname=::ffff:[client], addr=::ffff:[client], terminal=dovecot res=success)' type=USER_START msg=audit(1217984152.017:349): user pid=7669 uid=0 auid=10243 subj=user_u:system_r:dovecot_auth_t:s0 msg='PAM: session open acct="[me]" : exe="/usr/libexec/dovecot/dovecot-auth" (hostname=::ffff:[client], addr=::ffff:[client], terminal=dovecot res=failed)' I find the message in the audit.log interesting because I configured the connection with SSL not TLS, but if I use TLS the result is the same.

2 1

[Dovecot] Dovecot proxying results in "too many open files"
by Gianpaolo Del Matto 06 Aug '08

06 Aug '08

Hi, Some words about my Dovecot setup: - 10 dovecot servers - each server has some local mailboxes - each server has proxying enabled to seamlessly connect to another host if the mailbox is not local Running Dovecot 1.1.1 on FreeBSD 6.3-RELEASE-p3. First of all, local mailbox access is fine. Proxying works also as expected. Now for the actual problem: Shortly after startup, Dovecot starts throwing errors concerning connection which should be proxied. Here are some examples from the logs: Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143) failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: accept() failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143) failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143) failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: socket() failed: Too many open files Aug 6 15:49:10 dovecot: imap-login: proxy(XYZ): connect(1.2.3.4, 143) failed: Too many open files Aug 6 15:49:11 dovecot: imap-login: socket() failed: Too many open files [..] Aug 6 16:00:05 dovecot: imap-login: accept() failed: Too many open files Aug 6 16:00:08 dovecot: imap-login: accept() failed: Too many open files Aug 6 16:00:11 dovecot: imap-login: accept() failed: Too many open files Aug 6 16:00:13 dovecot: imap-login: accept() failed: Too many open files As from the context it seems to be a file descriptor issue, as such I have done the following: - added a 'limits -e -n 32768 -U dovecot' to dovecot startup file to ensure it has enough FD's - additionally, I've checked my kernel file limits, however I always set it through loader and sysctl to be as large as 64k # sysctl -a |grep files kern.maxfiles: 65535 kern.maxfilesperproc: 32768 kern.openfiles: 1632 - also checked the limits of the dovecot user like this: # su -c dovecot root -c 'ulimit -a' core file size (blocks, -c) unlimited data seg size (kbytes, -d) 1048576 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 32768 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 131072 cpu time (seconds, -t) unlimited max user processes (-u) 5547 virtual memory (kbytes, -v) unlimited - done some monitoring of file descriptor usage for the dovecot user, though somewhat inacurate, like this: while [ : ]; do date; fstat -u dovecot |wc -l; sleep 1; done this gave me these results while tailing the logfile: Wed Aug 6 15:44:12 CEST 2008 585 Wed Aug 6 15:44:13 CEST 2008 579 Wed Aug 6 15:44:14 CEST 2008 582 Wed Aug 6 15:44:15 CEST 2008 582 Wed Aug 6 15:44:16 CEST 2008 586 Wed Aug 6 15:44:17 CEST 2008 585 Wed Aug 6 15:44:18 CEST 2008 582 So it didn't seem to be anywhere near the theoretical upper limit I've set at 32k. On the other hand, when restarting Dovecot, it works for a few minutes for at least as long as the FD usage as reported above stays at around 500. I remember from the docs that dovecot-login would require the double amount of file descriptors to run. Assuming my report is somewhat inaccurate and lags behind, I could imagine that upon a proxy requests it would effectively peak our at some 500 x 2 FDs. Thus effectively reaching the 1024 FD barrier. I conclude this from the fact that I currently have some 350 - 450 logins in parallel, most of which can and shell be proxied. From the error message in the logs I got the impression, that only proxied connections are affected. I was not able to reproduce the issue with non-proxied logins to local mailboxes. Some further steps I've done from the configuration side to no avail: - disabled imaps/pop3s/ssl - changed between login_process_per_connection yes/no - played around with login_max_connections et all (raised/lowered) limits - tried with/without plugins enabled - tried with SQL-based and passwd-file based userdb/passdb, single and mixed Dovecot config: # dovecot -n # 1.1.1: /usr/local/etc/dovecot.conf base_dir: /var/run/dovecot/ protocols: imap pop3 listen: *, [::] ssl_disable: yes ssl_cert_file: /usr/local/etc/postfix/tls/server.crt ssl_key_file: /usr/local/etc/postfix/tls/server.key disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_process_per_connection: no login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_processes_count: 16 login_max_processes_count: 64 login_max_connections: 64 max_mail_processes: 256 mail_max_userip_connections: 3 verbose_proctitle: yes mail_privileged_group: mail mail_uid: 1000 mail_gid: 1000 mail_location: maildir:~/Maildir fsync_disable: yes maildir_copy_preserve_filename: yes mail_drop_priv_before_exec: yes mail_executable(default): /mailserver/scripts/dovecot/imap mail_executable(imap): /mailserver/scripts/dovecot/imap mail_executable(pop3): /mailserver/scripts/dovecot/pop3 mail_plugins: expire mail_log mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_log_max_lines_per_sec: 0 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra- mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra- mailbox-sep imap_client_workarounds(pop3): pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes pop3_enable_last(default): no pop3_enable_last(imap): no pop3_enable_last(pop3): yes pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace: type: private inbox: yes list: yes subscriptions: yes namespace: type: private separator: . prefix: INBOX. hidden: yes auth default: mechanisms: plain login cram-md5 cache_size: 131072 username_translation: #@/@%@ username_format: %Lu verbose: yes debug: yes debug_passwords: yes worker_max_request_count: 16384 passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: prefetch userdb: driver: passwd-file args: username_format=%n /usr/local/etc/postfix/tables/ deliver_passwd userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vpostfix group: vpostfix plugin: expire: Trash 14 Trash/* 14 Spam 14 Spam/* 14 VirusAlerts 14 VirusAlerts/* 14 Quarantine 14 Quarantine/* 14 expire_dict: proxy::expire dict: expire: mysql:/usr/local/etc/dovecot-expire.conf Maybe I am missing something here. I'd really, really appreciate some help on this to get it sorted out. Thank you. Regards, Gianpaolo

2 3