- dovecot - dovecot.org

[Dovecot] extremely slow delete/move operations?
by Gregory K. Ruiz-Ade 01 Feb '06

01 Feb '06

I've been working to migrate several hundered users on to our new mail servers, which run dovecot 0.99.11 (RHEL4). The problem I'm seeing is that deleting or moving a message seems to take a very long time. It's not uncommon for Thundirbird (1.0.7) to have delays of 5-10 seconds to delete a single message from my Inbox (<500k) and place it in my Trash folder (~100MB right now). The systems are Dell PowerEdge 1850 (dual 3.6GHz Xeon) with two 300GB SCSI drives in a hardware mirror. Filesystems are ext3 on LVM. Inboxes are on /var/spool/mail (/export/mail, symlinked into /var). User mail folders are in their home directories (/export/homes1/<user>, automounted to /home/<user>; some home directories are NFS automounted from a NetApp). Both the mail spool and home directories are available over NFS for compatibility reasons with the former setup. What's really bugging me is that I've used the same software (RHEL4 and dovecot 0.9x.x) at my last workplace, and the performance was outstanding; it beat UWIMAPd hands-down, especially with large mailboxes. Here, the opposite seems to be true. The primary difference seems to be that we're using NFS here, but I wasn't using NFS at my last job. I'm getting a lot of grief over the delays when users delete mail or move mail from their inbox to one of their mail folders. Enough so that they're now resisting moving to the new server. Where do I look to find out what's causing the slowness? What can I do to make it faster? Mailbox formats are mbox. Using these options in /etc/dovecot.conf: ----- protocols = imaps pop3s imap_listen = [::] pop3_listen = [::] ssl_cert_file = /usr/share/ssl/certs/csefast.ucsd.edu_cert.pem ssl_key_file = /usr/share/ssl/private/csefast.ucsd.edu_key.pem login_dir = /var/run/dovecot-login login = imap login = pop3 verbose_proctitle = yes first_valid_uid = 100 default_mail_env = mbox:~/:INBOX=/var/mail/%u mail_full_filesystem_access = yes mbox_locks = dotlock fcntl auth = default auth_mechanisms = plain auth_userdb = passwd auth_passdb = pam auth_user = root ----- Thanks in advance, Gregory -- Gregory K. Ruiz-Ade Sr. Systems Administrator Computer Science and Engineering University of California, San Diego Office: EBU3b 1216 Phone: (858) 822-2625 E-mail: gkra(a)ucsd.edu

9 10

Re: [Dovecot] Question about Dovecot and NFS
by Timo Sirainen 01 Feb '06

01 Feb '06

On Tue, 2006-01-31 at 22:42 -0800, Tony Kay wrote: > How does dovecot handle this kind of conflict. I.e. how would one dovecot IMAP on server A detect that dovecot on server B had rewritten something like flags on the mailbox they are both accessing? I'm sure it doesn't leave the dot lock sitting around, since that would block mail delivery. It just checks if mbox's mtime has changed. If it has, it checks if there are new mails or if it needs to do some other synchronization (mbox_dirty_syncs / mbox_very_dirty_syncs causes it to delay it as long as possible). UW-IMAP also checks if mtime has changed, but instead of trying to figure out what changed it just disconnects the client with an "unexpected mbox change" error.

1 0

[Dovecot] Immutable Maildir (or: "I know drag&drop")
by Michael Renner 31 Jan '06

31 Jan '06

Hi, I'm 'constantly' (every odd month) faced with users happily dragging around or (un)subscribing folders in our shared IMAP account. Is there a "clean" way to prevent users from doing that when having a Maildir? Unix permissions probably won't cut the slack since setting any Maildir (sub)directory to readonly (immutable or chmod u-w ) will probably cause dovecot to hit a wall at high speed sooner or later (lock files, rotated cache/log files) Got any other ideas? best regards, Michael Renner

2 1

[Dovecot] Question about Dovecot and NFS
by Tony Kay 31 Jan '06

31 Jan '06

Hi, We are using mbox and UW IMAP, and are having issues because UW IMAP does not support multiple client access when you use NFS for the mail files. I've done some research (see the comments after my 2 questions below), and most of the other major IMAP servers simply don't support it (courier is about the only one that has a positive comment). How much testing have you done w/NFS-mounted mail in mbox format? Have you found a reliable way to handle NFS mbox files? The UW IMAP docs say that multiple client access to mail spools that are NFS mounted is not safe, and the source has comments like this about NFS mail spools: /* Make fcntl() locking of NFS files be a no-op the way it is with flock() * on BSD. This is because the rpc.statd/rpc.lockd daemons don't work very * well and cause cluster-wide hangs if you exercise them at all. The * result of this is that you lose the ability to detect shared mail_open() * on NFS-mounted files. If you are wise, you'll use IMAP instead of NFS * for mail files. * * Sun alleges that it doesn't matter, because they say they have fixed all * the rpc.statd/rpc.lockd bugs. This is absolutely not true; huge amounts * of user and support time have been wasted in cluster-wide hangs. */ Cyrus' FAQ says this about NFS mail spools: (http://asg.web.cmu.edu/cyrus/imapd/install-FAQ.html#nfs) NFS (don't) Q: Weird things are happening. The mailspool is mounted over NFS, and... A: Don't mount the mail spool over NFS. Due to locking problems with NFS, this doesn't work (and for various other reasons, probably isn't worth the bother; IMAP appears to be I/O bound, not CPU bound, and scaling in this way is not a good idea). More specifically, if you use NFS and it looks to work, you may have locking problems in the future which will result in silently lost mail. If you use NFS and things go berzerk, it's because mmap(2) apparently has different semantics on local disk than it has on NFS. You can get the above behavior (silently lost mail) if you work at it. Courier is the only one that seems friendly (excerpt from their web site): Courier-IMAP is popular on Qmail/Exim/Postfix sites that are configured to use maildirs. The primary advantage of maildirs is that multiple applications can access the same Maildir simultaneously without requiring any kind of locking whatsoever. Maildir is a faster and more efficient way to store mail. It works particularly well over NFS, which has a long history of locking-related woes. -- Anthony Kay University Computing Center (541) 346-1719 GPG Fingerprint: B0DB D46A 60AF FAE7 A94A 5075 0CB4 4D88 9F4F 7F09 When they discover the center of the universe, a lot of people will be disappointed to discover they are not it. Bernard Bailey

3 2

[Dovecot] worth a few Euros?
by -- -- 31 Jan '06

31 Jan '06

Yo Timo, did you say that login processes "just don't crash"? Found these while doing load/malformed data tests. Got the kernel patch allow_setid_cores working. Don't have time to check if they're exploitable. Probably time to get a gamma version out for people, eh? Cheers, Jared :) =========================================================================================== Note: Running dovecot-1.0.beta2 with simple (no funniness) config Note: Running on Intel (in VmWare) with RH9. Note: I'm using shadow authentication, but tried with PAM and it still dumped. Note: For this first core, I see the server responding with: "*OK Waiting for authentication process to respond.." Note: Not sure, but I think these problems are timing related. Be sure to check if a pointer is valid and that will get rid of most of these. Note: I have plenty of sample cores if you need one to fix. -Jared DeMott [root@server dovecot]# gdb ./dovecot-auth -c core.22753 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... Core was generated by `dovecot-auth'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libpam.so.0...done. Loaded symbols for /lib/libpam.so.0 Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 auth_request_unref (_request=0x807b0f0) at auth-request.c:96 96 i_assert(request->refcount > 0); (gdb) bt #0 auth_request_unref (_request=0x807b0f0) at auth-request.c:96 #1 0x080523ee in auth_request_handler_unref (_handler=0x807b0f0) at auth-request-handler.c:66 #2 0x08050217 in auth_client_connection_destroy (_conn=0x807b0f0) at auth-client-connection.c:327 #3 0x0804ffd8 in auth_client_input (context=0x807b0f0) at auth-client-connection.c:227 #4 0x080607d0 in io_loop_handler_run (ioloop=0x8073100) at ioloop-poll.c:189 #5 0x0805fe29 in io_loop_run (ioloop=0x8073100) at ioloop.c:235 #6 0x08055329 in main (argc=1, argv=0xbfffe454) at main.c:309 #7 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) p request $1 = (struct auth_request *) 0x1 (gdb) i r eax 0x807b0f0 134721776 ecx 0x1 1 edx 0x6c 108 ebx 0x8084e78 134762104 esp 0xbfffe2f0 0xbfffe2f0 ebp 0xbfffe2f8 0xbfffe2f8 esi 0x80a43c8 134890440 edi 0xbfffe314 -1073749228 eip 0x80512d1 0x80512d1 eflags 0x286 646 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x33 51 (gdb) ==================================================================================== [root@server 1]# gdb ./dovecot-auth -c core.30100 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... warning: exec file is newer than core file. Core was generated by `dovecot-auth'. Program terminated with signal 6, Aborted. #0 0xffffe002 in ?? () (gdb) bt #0 0xffffe002 in ?? () #1 0x42028a73 in ?? () #2 0x0805d47c in default_info_handler (format=0x8067d00 "\024P?u\024?u\020?u\f?u\b?\t", args=0xbffff264 "z|\006\b!") at failures.c:162 #3 0x0805d04b in t_buffer_alloc (size=1108544020) at data-stack.c:347 #4 0x0804fa54 in reply_line_hide_pass (line=0x1 <Address 0x1 out of bounds>) at auth-client-connection.c:35 #5 0x0804fa71 in reply_line_hide_pass (line=0x0) at auth-client-connection.c:36 #6 0x080523bb in auth_request_handler_unref (_handler=0x0) at auth-request-handler.c:74 #7 0x08052b65 in auth_request_handler_auth_continue (handler=0x0, args=0x80774e0 "HU\n\b") at auth-request-handler.c:350 #8 0x0805f916 in io_add (fd=2, condition=39, callback=0x80778e8, context=0x0) at ioloop.c:23 #9 0x080602c8 in io_loop_notify_remove (ioloop=0x0, io=0x8075b00) at ioloop-notify-dn.c:138 #10 0x0805f985 in io_add (fd=0, condition=1073792608, callback=0x80678bc <pwrite_full+64>, context=0x0) at ioloop.c:43 #11 0x08054f94 in add_extra_listeners () at main.c:148 #12 0x42015574 in ?? () (gdb) i r eax 0x0 0 ecx 0x6 6 edx 0x42130a14 1108544020 ebx 0x7594 30100 esp 0xbffff0e0 0xbffff0e0 ebp 0xbffff0e8 0xbffff0e8 esi 0x806daa8 134666920 edi 0x80e92e8 135172840 eip 0xffffe002 0xffffe002 eflags 0x246 582 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x33 51 (gdb) ======================================================================================== [root@server login]# gdb ./imap-login -c core.25116 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... Core was generated by `imap-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libssl.so.4...done. Loaded symbols for /lib/libssl.so.4 Reading symbols from /lib/libcrypto.so.4...done. Loaded symbols for /lib/libcrypto.so.4 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /usr/kerberos/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/kerberos/lib/libgssapi_krb5.so.2 Reading symbols from /usr/kerberos/lib/libkrb5.so.3...done. Loaded symbols for /usr/kerberos/lib/libkrb5.so.3 Reading symbols from /usr/kerberos/lib/libk5crypto.so.3...done. Loaded symbols for /usr/kerberos/lib/libk5crypto.so.3 Reading symbols from /usr/kerberos/lib/libcom_err.so.3...done. Loaded symbols for /usr/kerberos/lib/libcom_err.so.3 Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 i_stream_read (stream=0x0) at istream.c:48 48 if (stream->closed) (gdb) bt #0 i_stream_read (stream=0x0) at istream.c:48 #1 0x0804ade2 in client_read (client=0x8070410) at client.c:311 #2 0x0804ae3c in client_input (context=0x8070410) at client.c:333 #3 0x08055558 in io_loop_handler_run (ioloop=0x806cc50) at ioloop-poll.c:189 #4 0x08054bb1 in io_loop_run (ioloop=0x806cc50) at ioloop.c:235 #5 0x0804d3ed in main (argc=1, argv=0xbffff504, envp=0xbffff50c) at main.c:341 #6 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) p stream $1 = (struct istream *) 0x0 (gdb) ====================================================================================== [root@server core]# gdb ./imap -c core.15043 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... warning: exec file is newer than core file. Core was generated by `imap'. Program terminated with signal 6, Aborted. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 #0 0xffffe002 in ?? () (gdb) bt #0 0xffffe002 in ?? () #1 0x42028a73 in abort () from /lib/tls/libc.so.6 #2 0x0809f9fc in i_internal_panic_handler (fmt=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfffdd94 "\0251\v\b?") at failures.c:375 #3 0x0809f5cb in i_panic (format=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:173 #4 0x08082277 in mail_cache_transaction_reserve_more (ctx=0x80cb4a0, block_size=332, commit=true) at mail-cache-transaction.c:257 #5 0x0808248a in mail_cache_transaction_get_space (ctx=0x80cb4a0, min_size=332, max_size=332, offset_r=0x0, available_space_r=0x0, commit=true) at mail-cache-transaction.c:347 #6 0x08082b90 in mail_cache_header_add_field (ctx=0x80cb4a0, field=0) at mail-cache-transaction.c:646 #7 0x08082e0e in mail_cache_add (ctx=0x80cb4a0, seq=1, field=9, data=0x0, data_size=0) at mail-cache-transaction.c:693 #8 0x0807b314 in index_mail_cache_add (mail=0x80cfca0, field=9, data=0x0, data_size=0) at index-mail.c:349 #9 0x0807c799 in index_mail_parse_header_finish (mail=0x80cfca0) at index-mail-headers.c:75 #10 0x0807ce0c in index_mail_parse_header (part=0x42130a14, hdr=0x0, mail=0x80cfca0) at index-mail-headers.c:277 #11 0x0809b3d1 in message_parse_part_header (parser_ctx=0x80d7be8) at message-parser.c:253 #12 0x0809bb01 in message_parser_parse_header (ctx=0x80d7be8, hdr_size=0x80cfd74, callback=0, context=0x0) at message-parser.c:608 #13 0x0807cf18 in index_mail_parse_headers (mail=0x80cfca0, headers=0x80d4088) at index-mail-headers.c:361 #14 0x0807d42f in index_mail_get_headers (_mail=0x80cfca0, field=0x80b0ff5 "Date") at index-mail-headers.c:535 #15 0x0807d4d9 in index_mail_get_first_header (mail=0x80cfca0, field=0x80b0ff5 "Date") at index-mail-headers.c:576 #16 0x080933a4 in mail_get_first_header (mail=0x80d4088, field=0x80b0ff5 "Date") at mail.c:82 #17 0x0807b054 in index_mail_get_date (_mail=0x80cfca0, timezone=0x0) at index-mail.c:253 #18 0x08093370 in mail_get_date (mail=0x80d4088, timezone=0x0) at mail.c:61 #19 0x0805fe97 in mail_thread_input (ctx=0x80bd580, mail=0x80cfca0) at imap-thread.c:456 #20 0x0805f933 in imap_thread (cmd=0x80c37f4, charset=0x80c3b00 "US-ASCII", args=0x80cc628, type=MAIL_THREAD_REFERENCES) at imap-thread.c:143 #21 0x08059863 in cmd_thread (cmd=0x80c37f4) at cmd-thread.c:66 #22 0x08059949 in cmd_uid (cmd=0x80c37f4) at cmd-uid.c:19 #23 0x0805a1a7 in client_handle_input (cmd=0x80c37f4) at client.c:355 #24 0x0805a2c6 in _client_input (context=0x80c37b0) at client.c:406 #25 0x080a4ba8 in io_loop_handler_run (ioloop=0x80c1740) at ioloop-poll.c:189 #26 0x080a4201 in io_loop_run (ioloop=0x80c1740) at ioloop.c:235 #27 0x08061053 in main (argc=1, argv=0xbfffe2d4, envp=0xbfffe2dc) at main.c:238 #28 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) i r eax 0x0 0 ecx 0x6 6 edx 0x42130a14 1108544020 ebx 0x3ac3 15043 esp 0xbfffdc10 0xbfffdc10 ebp 0xbfffdc18 0xbfffdc18 esi 0x80c9cb8 135044280 edi 0x14c 332 eip 0xffffe002 0xffffe002 eflags 0x246 582 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x33 51 (gdb) ======================================================================================

3 9

[Dovecot] beta2: strange assert
by Jeff A. Earickson 31 Jan '06

31 Jan '06

Hi, My setup: Solaris 9, mbox format, mailboxes NFS mounted from another S9 system, imap. I got the following assert yesterday: Jan 30 19:57:15 emerald dovecot: [ID 107833 mail.error] imap(user): file index-mail-headers.c: line 258 (index_mail_parse_header): assertion failed: (part != NULL) Jan 30 19:57:16 emerald dovecot: [ID 107833 mail.error] imap(user): file index-mail-headers.c: line 258 (index_mail_parse_header): assertion failed: (part != NULL) The gdb work on the core file is attached. This is a mystery because part was NULL in the core file. Weird. Jeff Earickson Colby College

2 1

[Dovecot] adding a select or a poll?
by ohp＠pyrenet.fr 31 Jan '06

31 Jan '06

Since I put my hands on dovecot, I had this recuring problem at startup: Imap-login would'nt connect to the 'default' socket. I've been strugling with the fact that it would eventually connect on a slow or moderely fast machine but not on a really fast machine. According to Unixware doc, the FIRST connect ALWAYS returns -1 and errno=EINPROGRESS AFAICT, imap loop is like this: while (1) connect if no error bu EINPROGRESS send_handshake if successfull break; end while Now, I think the time between connect and send_handshake might be too short for async connect to finish in most case. I recognize is a Unixware probleme but I think this senario can happen on any system that CAN set errno to EINPROGRESS. I think the two possibilities are 1 - remove the setting of NONBLOCK before connecting to unix sock, but that would ruin the way dovecot is written. 2 - add a poll or a select before send_handshake to be sure the socket is writable. The third way would be to tell unixware users to disable async connects methink would be a pitta. What do you think? -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 15, Chemin des Monges +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp(a)pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)

1 0

[Dovecot] Re: extremely slow delete/move operations?
by Roger Weeks 31 Jan '06

31 Jan '06

On Jan 30, 2006, at 11:12 AM, Gregory K. Ruiz-Ade wrote: > Message: 10 > The thing is, these mail servers _are_ the local storage for the > mail spools > and home directories. There is no network involved when dovecot is > accessing the INBOX or saved mail folders. Yes, they're "NFS" > mounted, but > this is RHEL4 (linux 2.6.x), and when an NFS share is mounted from > the local > host, it's done as a bind mount, which means local disk access. I don't understand this at all. Mounting anything via NFS should only be done from a remote server, where when you type "mount" at a commandline you see something like: 10.10.10.2:/data on /var/mail type nfs (rw,nfsvers=3,addr=10.10.10.2) If your /etc/fstab or your automounts on your RHEL4 box are mouting LOCAL disks as type NFS, you have set up something very very wrong. Local disks should be mounted at boot like so: /dev/sda3 on / type ext3 (rw) Let's be certain we're clear about this. The disks where dovecot reads and writes your mboxes are local to the RHEL4 machine? If so, you're not using NFS at all. Roger Weeks

2 1

[Dovecot] Dovecot-lda/deliver - how to deliver root's mail?
by Shayne Hardesty 30 Jan '06

30 Jan '06

There's a good chance I could be missing something obvious here, as I can't find much documentation on the LDA other than on the wiki, which is sparse. Local mail delivery works for normal users, but when attempting to deliver mail for root I get this error: Jan 30 13:42:50 apollo deliver: Running as root isn't permitted Both sendmail and dovecot run as root on my server - since I don't allow shell accounts and have multiple layers of security around the server I'm okay with that. How can I tell dovecot-lda it's okay to deliver root's mail as root? Thanks, Shayne

2 1

[Dovecot] poprelay, ip v4 vs ip v6 adresses
by Gerhard Hofmann 30 Jan '06

30 Jan '06

Hi list, I would like to use poprelay as described here: http://wiki.dovecot.org/PopRelay Excerpt from my /var/log/mail.log: dovecot: 2006-01-26 11:27:52 Info: imap-login: Login: user=<planat99>, method=plain, rip=192.168.1.202, lip=192.168.1.5 dovecot: 2006-01-26 13:10:10 Info: pop3-login: Login: user=<planat99>, method=PLAIN, rip=192.168.1.202, lip=192.168.1.5 (these are only two lines but wrapped by my news client) As you can see, the logged remote ip address is in normal ip v4 style, no "ffff" stuff here. The WIKI page says that ip addresses should be in v6-mapped style, how can I achieve this? Respectively, how must I change poprelay.conf to work with my current mail.log style? TIA Gerhard

1 0