- dovecot - dovecot.org

[Dovecot] any limitations running on a Mac?
by Terry Barnum 28 Feb '10

28 Feb '10

I have postfix/dovecot/mysql installed using MacPorts on a quad-core 2.8GHz MacPro running Snow Leopard (10.6.2). I moved the base mail directory to a pair of 10k RPM Raptors that are mirrored (/Volumes/email/) and everything seems to be working fine on an unused domain with very little traffic. I used imapsync to pull everything from the current mailserver to this test server to play with. Our current mailserver's IMAP performance *really* suffers when an IMAP folder exceeds ~500MB or ~5k messages. I've become very tired of being the mailbox police trying to get my 20 users to delete email or divide into smaller mailboxes. Are there folks on the list running postfix/dovecot on similar Mac hardware that can share their experiences? Specifically, are there any limitations (file descriptors, other?) that can impact performance I should be aware of? How does dovecot on the Mac deal with >500GB maildirs? Is there a recommended tool for loading and testing the server? Postal? I have 20 IMAP users, with a total of ~37GB of mail currently. Thanks for any help and insight. -Terry Terry Barnum digital OutPost San Diego, CA http://www.dop.com 800/464-6434

6 14

[Dovecot] Possible CPU Denial-Of-Service attack to dovecot IMAP.
by Kostik 28 Feb '10

28 Feb '10

Hi All! Some time ago, we received e-mail message, which makes our server CPU exhaustion attack. --- PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 26319 5751796 20 0 2868 1868 1484 R 99.2 0.1 22:04.77 imap --- It happens when I try open mail folder with this buggy message. Our setup: -- slackware 11.0, x86_32 -- linux 2.6.31.6 -- dovecot 1.2.10 -- mailbox(not maildir) via NFS storage. Details: 1. Buggy message 1219733 byte size. And most of this size is mail header. 2. Mail header mostly consists of a repeating block: --- [...] MIME-Version: 1.0 Content-type: text/plain; charset=windows-1251 Content-transfer-encoding: 8bit Date: Thu, 25 Feb 2010 11:13:03 +0300 X-Priority: 0 X-MSMail-Priority: Normal X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Sender: torrents(a)rutracker.org Reply-to: torrents(a)rutracker.org From: torrents(a)rutracker.org Message-ID: <4052c9f301d0956f3fa1e855cca02d39(a)rutracker.org> [...] MIME-Version: 1.0 Content-type: text/plain; charset=windows-1251 Content-transfer-encoding: 8bit Date: Thu, 25 Feb 2010 11:13:02 +0300 X-Priority: 0 X-MSMail-Priority: Normal X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Sender: torrents(a)rutracker.org Reply-to: torrents(a)rutracker.org From: torrents(a)rutracker.org Message-ID: <6162ed30245a86b8cb26e4a0bf4de562(a)rutracker.org> [...] --- 3.strace show, that dovecot many-many time do: --- pread64(6, "-Sender: torrents(a)rutracker.org\n"..., 4038, 814845) = 4038 pread64(6, "s(a)rutracker.org\nMessage-ID: <461"..., 4083, 818883) = 4083 pread64(6, "arset=windows-1251\nContent-trans"..., 4068, 822966) = 4068 pread64(6, "ail-Priority: Normal\nX-Mailer: M"..., 4091, 827034) = 4091 --- 4. and gdb in that time: --- (gdb) where #0 0xb7604060 in memset () from /lib/tls/libc.so.6 #1 0x080d7483 in buffer_write (_buf=0x8131db0, pos=135470571, data=0x8132d52, data_size=2) at buffer.c:54 #2 0x080d74fb in buffer_append (buf=0x0, data=0x8132d52, data_size=2) at buffer.c:168 #3 0x080cebfe in read_header (mstream=0x8131be8) at istream-header-filter.c:214 #4 0x080cef07 in i_stream_header_filter_read (stream=0x8131be8) at istream-header-filter.c:301 #5 0x080cef79 in parse_header (mstream=0x8131be8) at istream-header-filter.c:323 #6 0x080cefce in i_stream_header_filter_seek (stream=0x8131be8, v_offset=128810, mark=false) at istream-header-filter.c:337 #7 0x080ddfdb in i_stream_seek (stream=0x8131c10, v_offset=128810) at istream.c:198 #8 0x0809c99c in i_stream_mail_stats_read_mail_stats (stream=0x812c460) at istream-mail-stats.c:43 #9 0x080ddbfc in i_stream_read (stream=0x812c488) at istream.c:85 #10 0x080de389 in i_stream_read_data (stream=0x812c488, data_r=0xbf9d82e4, size_r=0xbf9d82e8, threshold=2) at istream.c:366 #11 0x080d400b in message_get_header_size (input=0x812c488, hdr=0x81298fc, has_nuls=0x0) at message-size.c:56 #12 0x0808f976 in index_mail_init_stream (mail=0x81297d0, hdr_size=0xbf9d83b0, body_size=0xbf9d8390, stream_r=0x0) at index-mail.c:852 #13 0x080838a1 in mbox_mail_get_stream (_mail=0x81297d0, hdr_size=0xbf9d83b0, body_size=0xbf9d8390, stream_r=0xbf9d838c) at mbox-mail.c:322 #14 0x0808e9ac in index_mail_get_virtual_size (_mail=0x81297d0, size_r=0xbf9d8400) at index-mail.c:397 #15 0x080685a3 in fetch_rfc822_size (ctx=0x0, mail=0x81297d0, context=0x0) at imap-fetch-body.c:894 #16 0x0806607c in imap_fetch_more (ctx=0x811e030) at imap-fetch.c:472 #17 0x0805e370 in cmd_fetch (cmd=0x811dec8) at cmd-fetch.c:228 #18 0x0806292b in cmd_uid (cmd=0x811dec8) at cmd-uid.c:27 #19 0x08063726 in client_command_input (cmd=0x811dec8) at client.c:612 #20 0x080636d1 in client_command_input (cmd=0x811dec8) at client.c:661 #21 0x080638b3 in client_handle_input (client=0x811cc08) at client.c:701 #22 0x080642d6 in client_input (client=0x811cc08) at client.c:753 #23 0x080e1bf1 in io_loop_handler_run (ioloop=0x0) at ioloop-epoll.c:208 #24 0x080e0f79 in io_loop_run (ioloop=0x8119ab0) at ioloop.c:335 #25 0x0806c21a in main (argc=3, argv=0xbf9d86b4, envp=0xbf9d86c4) at main.c:327 --- 5. I can provide download link to this buggy mailbox file if needed. =koc

2 1

[Dovecot] client cert handling not working properly on centos 4.8
by zhong ming wu 28 Feb '10

28 Feb '10

Dear List I've successfully installed/configured dovecot 1.2.10 with "require client cert" on centos 5.4 and ubuntu server 9.10 I also need to install on centos 4.8 and after the following the exact same procedure I can only get it working if I commented out ssl_require_client_cert =yes and ssl_username_from_cert = yes from the working config file. This is even after compiling dovecot with openssl 0.9.8l on centos 4.8 If I copy the same "client_ca.crt" from centos 4.8 to centos 5.4 then centos 5.4 does not problem in verifying client cert. That file contain CRL as well as certificate which signs the pkcs12 file installed on the client. The following log entries do not appear on centos 5.4 ------------------ Feb 27 21:17:33 localhost dovecot: pop3-login: Invalid certificate: unable to get certificate CRL: /C=US/ST=New York/L=Astoria/O=SnakeOil Inc./OU=Email Administration/CN=web(a)example.com Feb 27 21:17:33 localhost dovecot: pop3-login: Valid certificate: /C=US/ST=NY/L=TEST/O=Internet Widgits Pty Ltd ------------------- $ dovecot -n # OS: Linux 2.6.9-89.0.20.EL i686 CentOS release 4.8 (Final) ext3 base_dir: /var/run/dovecot/ protocols: pop3 listen: 192.168.0.110 ssl_ca_file: /etc/pki/certs/dovecot/client_ca.crt ssl_cert_file: /etc/pki/certs/vrane.com/pop.crt ssl_key_file: /etc/pki/private/vrane.com/pop.key ssl_parameters_regenerate: 29 ssl_verify_client_cert: yes verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/libexec/dovecot/pop3-login mail_location: maildir:/home/vmail/%d/%n mail_executable: /usr/libexec/dovecot/pop3 mail_plugin_dir: /usr/lib/dovecot/pop3 auth default: user: squab debug: yes ssl_require_client_cert: yes ssl_username_from_cert: yes passdb: driver: passwd-file args: /etc/dovecot/shadow/%d userdb: driver: static args: uid=2000 gid=2000 home=/home/vmail/%d/%n socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix

1 0

[Dovecot] IMAP Folders Don't Make Sense
by Carlos Williams 27 Feb '10

27 Feb '10

I noticed for some reason that I am missing an IMAP folder called 'Symantec' that was under my Inbox. I see all my other folders listed on my mail client except 'Symantec'. I then login to my mail server which runs Postfix / Dovecot & there I can see it: /home/cwilliams/Maildir [root@mail Maildir]# ls -la total 608 drwx------ 33 cwilliams it 4096 Feb 25 13:23 . drwx------ 5 cwilliams it 4096 Feb 23 13:24 .. drwx------ 2 cwilliams it 53248 Feb 25 13:22 cur -rw------- 1 cwilliams it 3048 Feb 25 12:42 dovecot.index -rw------- 1 cwilliams it 178176 Feb 25 13:22 dovecot.index.cache -rw------- 1 cwilliams it 54936 Feb 25 13:23 dovecot.index.log -rw------- 1 cwilliams it 131124 Feb 10 16:18 dovecot.index.log.2 -rw------- 1 cwilliams it 121 Jul 15 2009 dovecot-keywords -rw------- 1 cwilliams it 8190 Feb 25 13:13 dovecot-uidlist drwx------ 5 cwilliams it 4096 Feb 25 12:47 .Drafts drwx------ 2 cwilliams it 4096 Feb 25 13:27 .INBOX drwx------ 5 cwilliams it 4096 Feb 25 08:55 .INBOX.ClamAV drwx------ 5 cwilliams it 4096 Feb 25 08:54 .INBOX.Dell.Quotes drwx------ 5 cwilliams it 4096 Feb 25 08:54 .INBOX.Dell.Warranty drwx------ 5 cwilliams it 4096 Feb 25 08:55 .INBOX.IBM drwx------ 5 cwilliams it 4096 Oct 2 2008 .INBOX.Logs drwx------ 5 cwilliams it 4096 Feb 25 08:55 .INBOX.Red Hat drwx------ 5 cwilliams it 4096 Feb 23 09:26 .INBOX.Symantec drwx------ 2 cwilliams it 32768 Feb 25 13:13 new drwx------ 5 cwilliams it 4096 Feb 25 13:25 .Sent drwx------ 5 cwilliams it 4096 Feb 17 08:37 .Spam -rw------- 1 cwilliams it 393 Feb 23 10:23 subscriptions drwx------ 2 cwilliams it 4096 Feb 25 13:13 tmp drwx------ 5 cwilliams it 4096 Feb 25 13:22 .Trash As you can see above there is a .INBOX.Symantec directory and I can 'cd' into it and see all my messages: /home/cwilliams/Maildir/.INBOX.Symantec [root@mail .INBOX.Symantec]# ls -l total 280 drwx------ 2 cwilliams it 12288 Jan 21 17:39 cur -rw------- 1 cwilliams it 2352 Jan 21 17:39 dovecot.index -rw------- 1 cwilliams it 233472 Feb 22 08:17 dovecot.index.cache -rw------- 1 cwilliams it 8428 Feb 22 08:17 dovecot.index.log -rw------- 1 cwilliams it 41 Oct 12 09:50 dovecot-keywords -rw------- 1 cwilliams it 6665 Jan 21 17:39 dovecot-uidlist drwx------ 2 cwilliams it 4096 Oct 2 2008 new drwx------ 2 cwilliams it 4096 Jan 21 17:39 tmp However when I open Thunderbird & Webmail (both use IMAP), I see my Symantec folder under a 'Drafts' folder. When I try and click on the Symantec folder as a sub folder in 'Drafts', I get an error: "The mail server responded:Mailbox doesn't exist: Drafts.Symantec" So why does it show up there and why can't I see it normally with all my other IMAP folders under my Inbox. I am confused. Please help! Thanks!

8 22

[Dovecot] panic in istream-raw-mbox.c - 2.0.beta3+
by Mark Sapiro 27 Feb '10

27 Feb '10

Saw the following in the log after shutting down my client. Feb 22 20:44:47 sbh16 dovecot: imap(mark): Disconnected: Logged out bytes=1755/9765 Feb 22 20:44:47 sbh16 dovecot: imap(mark): Panic: file istream-raw-mbox.c: line503 (istream_raw_mbox_get_start_offset): assertion failed: (rstream->seeked) Feb 22 20:44:47 sbh16 dovecot: imap(mark): Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0 [0x5ad950] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x5ad9a5] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x5ad008] -> /usr/local/lib/dovecot/libdovecot-storage.so.0 [0xe387a7] -> /usr/local/lib/dovecot/libdovecot-storage.so.0 [0xe406e4] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_get_special+0x1d) [0xdcaaed] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_expunge+0x34) [0xe0aed4] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_expunge+0xf) [0xdcab9f] -> dovecot/imap(imap_expunge+0xc5) [0x8056eb5] -> dovecot/imap(cmd_close+0x73) [0x804eab3] -> dovecot/imap [0x80550cc] -> dovecot/imap [0x805517b] -> dovecot/imap(client_handle_input+0x33) [0x80552d3] -> dovecot/imap(client_continue_pending_input+0x79) [0x8055529] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x110) [0x5b98d0] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x2f) [0x5b858f] -> /usr/local/lib/dovec (the above appears truncated, but that's all that was logged) Feb 22 20:44:47 sbh16 dovecot: master: service(imap): child 27281 killed with signal 6 (core dumps disabled) Feb 22 20:44:47 sbh16 dovecot: imap(gpc): Disconnected: Logged out bytes=274/1356 This is 2.0.beta3 plus http://hg.dovecot.org/dovecot-2.0/rev/32d6dd7ce7cb [root@sbh16 dovecot]# dovecot -n # 2.0.beta3: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-8.1.14.el5 i686 CentOS release 5 (Final) auth_mechanisms = plain apop login auth_worker_max_count = 5 mail_location = mbox:~/Mail:INBOX=/var/spool/mail/%u mail_privileged_group = mail mbox_write_locks = fcntl dotlock passdb { args = /usr/local/etc/dovecot.passwd deny = no driver = passwd-file master = no pass = no } passdb { deny = no driver = pam master = no pass = no } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } ssl_cert = </etc/postfix/sbh16-cert.pem ssl_key = </etc/postfix/sbh16-key.pem userdb { driver = passwd } [root@sbh16 dovecot]# -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 4

[Dovecot] Little Help with Plugin
by Alex Baule 26 Feb '10

26 Feb '10

Hi everyone..... I Build a plugin to concat my body and header splited.... Its have 2 "versions". One is with the body without modifications. The second, is with the body with crypto, and this crypto increase a little the size of email. So, without crypto, works fine, but with crypto i got stuck in some modifications. Because the email is splited, to handle it, i made a copy from istream-concat.c, and in i_stream_concat_read i made this changes. Every place that have the i_stream_get_data i get the index from concat (if is the frist or second stream) and if is the second, (my body) the unsigned char *data is the return from my function. if(cstream->cur_idx == 1){ data = decrypt_data(cstream->cur_input, &data_size); }else{ data = i_stream_get_data(cstream->cur_input, &data_size); } In this part, i get the function i_stream_get_data and put in the function too, to get the data in the same way, but i dont return the (_stream->buffer + _stream->skip;), i put this in a unsigned char * too, and work with this variable. The size_r, with the _stream->pos - _stream->skip, i update it, with the new value get from my total length from the decrypted buffer. In resume, every (inside istream_concat_read and istream_concat_read_next ) i_stream_get_data(cstream->cur_input, &data_size); // (in istream_concat_read is "pos" variable) I change to if(cstream->cur_idx == 1){ data = decrypt_data(cstream->cur_input, &data_size); //(in istream_concat_read is "pos" variable) }else{ data = i_stream_get_data(cstream->cur_input, &data_size); //(in istream_concat_read is "pos" variable) } Setting the unsigned char *data, with my decrypted data and the pos or data_size, with the length of buffer data , decrypted. I Think there is Ok with this, and must be work, but i got some errors with this, when the Email is read to get them, the total length read is diferent from the W flag in header name. FETCH [] for mailbox INBOX UID 2 got too little data: 1351 vs 1352 FETCH [] for mailbox INBOX UID 1 got too little data: 1357 vs 1360 FETCH [] for mailbox INBOX UID 3 got too little data: 691 vs 1303 There is some variable or something that i forgot to update with this news values ? I use the Zlib plugin for exemple to do this. Tks Guys !!!

1 0

[Dovecot] Thunderbird 3.0.2 released (CONDSTORE fixed)
by David Halik 25 Feb '10

25 Feb '10

Just wanted to mentioned that those of you who were having issues with unread messages in Thunderbird should see your problem fixed with TB 3.0.2 that was released today. http://www.rumblingedge.com/2010/02/25/thunderbird-3-0-2-released/ If you previously turned off CONDSTORE support, don't forget to enable it. -- ================================ David Halik System Administrator OIT-CSS Rutgers University dhalik(a)jla.rutgers.edu ================================

1 0

[Dovecot] Duplicate Elimination
by Aravind Divakaran 25 Feb '10

25 Feb '10

Hi All, In my mail server i have two groups and one user is belonging to both the groups. Previously i was using cyrus at that time when i am sending a mail to the two groups there will only one mail in the user mailbox . But now the user is getting one more copy of same mail. Is there is a way i can avoid this. Rgds, Aravind M D

8 13

[Dovecot] Regarding: **OFF LIST** subject declaration
by Jerry 25 Feb '10

25 Feb '10

Seriously, I just have to ask this question. Why mark via the subject line a message as "OFF LIST" and then send it via the normal list framework. Doing so only insures that the message is actually "ON LIST" irregardless of what nomenclature is used in the subject line. If a message is truly supposed to be "OFF LIST", then why not send it directly to its intended recipient(s)? If, on the other hand, it is meant for general review by the groups members, then why mark it "OFF LIST" to begin with? I am aware of the prefix "OT" used in several forums; however, that is usually only used to thwart the moderator of the forum from banning the message or chastising the OP. Its use is of dubious success. -- Jerry gesbbb(a)yahoo.com |::::======= |::::======= |=========== |=========== | The course of true anything never does run smooth. Samuel Butler

4 6

[Dovecot] FTS cron script to force index updates
by Stan Hoeppner 25 Feb '10

25 Feb '10

From: http://wiki.dovecot.org/Plugins/FTS "You could also build a cronjob to index users' mailboxes once in a while (by selecting each mailbox and issuing a SEARCH TEXT xyzzyx command)." Has anyone written a script to perform the above? If not, how would I do this? Would I write a shell script to telnet to the IMAP port and then issue the commands? Can this be done with a bash script? Or would I need something more serious like a perl script? Thanks. -- Stan

2 1