- dovecot - dovecot.org

[Dovecot] Speed and memory probs writing large Maildir
by Robin Whittle 14 Jun '03

14 Jun '03

I have Dovecot 0.99.9.1 compiled from the source, without SSL, running on a Red Hat 9.0 system - EXT3 file system, Pentium Pro 233 MHz, Intel motherboard, 128M RAM, IBM 40Gig 7200 RPM HD. I find reading and searching to be nice and fast, and comparing it with Courier IMAP on a Celeron 824 MHz, I think Dovecot is generally not much slower, which means it may well be faster than Courier in some or many ways if it was running on the same machine. My only problem is when writing mailboxes with large numbers of messages. I use Mailir - the same arrangement as Courier IMAP. (I also find that Netscape 4.77 can't use mailboxes which are subfolders of others, but that is probably a Netscape config problem.) I am using Netscape 4.77 (Windows 2000 1.3 GHz Celeron, via 100Mbps switched Ethernet to the Linux machines) to do these tests since I find Netscape 7.02 to be much slower. Netscape 7.02 seems to want to resynch its notion of the mailbox repeatedly as it is writing to it. Without looking at the IMAP traffic (I don't know how) I would say that Netscape 4.77 is not doing this at all. Indeed, after the write, looking at the mailbox causes Netscape 4.77 to read the headers from the IMAP server - so I feel sure that its way of writing the messages is purely "write". Writing a mailbox with a small number of large messages, such as 20 x 100k messages is fine. There is no obvious speed problem or excessive memory use. When writing 2000 x 1k messages, it is totally different. Here are some times for writing from a Netscape 4.77 local mailbox to Dovecot and to Courier IMAP. Message Total Time CPU max Memory RSS size number size Seconds Approx Megs - max Dovecot 100k 100 10M 23 15% ~2 100k 20 2M ~5 ? 1.2 10k 200 2M 20 15% 4 ! 1k 2000 2M 230 60 32 !!!! Courier IMAP 100k 100 10M 14 5% ~1 100k 20 2M ~2 ? ? 10k 200 2M 9 5% ~0.6 1k 2000 2M 88 2% ~0.6 This 2000 message test is small compared to some of the mailboxes I have. I used a real-live mailbox, 6878 messages - 44 Megabytes in a single Mbox for a more demanding test. Using Netscape 4.77 to copy this to a Dovecot Maildir took a very long time - over 30 minutes. The process was fast at first and got slower and slower. As the number of messages in the destination Maildir increased, the memory usage went up and the speed reduced. It seems that RSS memory usage (as reported by "top -d 0.3" hit some kind of limit at about 49 Megs after 4000 messages and then went down a little to about 48 or less. CPU usage was 30 to 82%, fluctuating according to the moment top looked. At the end, I think it was only writing 1.5 messages a second. RSS is the "total amount of physical memory used by the task". The "SIZE" was even bigger - "code plus data plus stack". This got up to 114 Megs by the end of the writing process! After the write had finished, I made Netscape 4.77 view the destination mailbox. This caused Dovecot to send it all the headers, and again Dovecot had these huge memory usage figures. It seemed to read the mailbox at a decent speed - 20 seconds or so. But if I made Dovecot look at another mailbox, and then again at this big target mailbox (say with another client - Netscape 7), it would read the big mailbox at Dovecot's usual high speed - without the high memory usage and I think without high CPU usage as well. Without knowing anything about how Dovecot works, I imagine that there is some kind of caching algorithm for writes, and that this is retained until Dovecot (or at least the instance serving this client) is asked to look at another Maildir. It seems that reading is not slowed by this process much or at all - but writing gets progressively slower as the cache gets bigger. The memory usage - even 49 Megs for RSS - seems excessive. It reached that about halfway through writing the mailbox, so the entire message contents at that state would have been about 22 Megs. But the "SIZE" memory usage is even larger - perhaps limited by the available RAM. This slow speed for writing Maildirs with large numbers of messages is currently a barrier to me using Dovecot. I am about to go back and try again to get Courier IMAP to do what I want . . . The Courier IMAP documentation and what I regard as the difficult "configure -> build -> install -> wonder why it doesn't run" process has cost me so much time that I would be very happy to use something simpler and easier (for me, at least) to install. Thanks for developing this new IMAP server! - Robin

4 8

[Dovecot] DRAC support?
by Charlie Allom 14 Jun '03

14 Jun '03

Hi, I was hoping someone had a quick and dirty (is there any other kind) of patch for DRAC support for dovecot. This daemon is very slick, and I wish to replace my complex courier-imap setup. Regards, C. -- charlie(a)rubberduck.com - Melbourne, Australia http://rubberduck.com/~yeled/ PGP: 0x14AA7941 || finger yeled(a)lazy.spodder.com > http://groups.google.com/groups?selm=3266D499.342E%40student.uq.edu.au <

5 44

[Dovecot] --enable-debug bug?
by Gasha 14 Jun '03

14 Jun '03

Hello all, when i'm bulding rpm with --enable-debug get this: I tried 2 different versions 0.99.8 and 0.99.9. Jun 2 20:41:14 mezgls dovecot: Dovecot starting up Jun 2 20:41:15 mezgls dovecot: child 9409 (login) killed with signal 11 Jun 2 20:41:15 mezgls dovecot: child 9408 (login) killed with signal 11 Jun 2 20:41:15 mezgls dovecot: child 9407 (login) killed with signal 11 Jun 2 20:41:15 mezgls dovecot: child 9406 (auth) killed with signal 11 Jun 2 20:41:15 mezgls dovecot: Login process died too early - shutting down Gasha

2 1

[Dovecot] squirrelmail setup
by Gasha 14 Jun '03

14 Jun '03

Hi all, My setup: INBOX -> /var/spool/mail/user directories -> /home/user/Maildir it works with Netscape, but squirrel mail shows only subfolder content. nothing in INBOX. any ideas? Gasha

2 3

[Dovecot] Dovecot configuration with passwd-file
by Spectron International, Inc. 14 Jun '03

14 Jun '03

I have tried to configure dovecot to use passwd-file authentication and I havent been able to. I changed the following in the dovecot.conf file (I want to have all the mailboxes in one directory): default_mail_env = maildir:/opt/mail/%u auth_userdb = passwd-file /etc/passwd.imap auth_passdb = passwd-file /etc/passwd.imap auth_verbose = yes It gives me the following error (I hoped it would be more verbose than this): Internal login failure: f_rojas [192.168.10.4]. passwd.imap exists, the user and password are ok What am I missing? Fawzib Rojas

2 1

[Dovecot] Debian and dovecot
by Spectron International, Inc. 14 Jun '03

14 Jun '03

I have Debian 3.0 (woody) and installed the dovecot packages and it works ok with the default configuration. I want to change the default configuration, is any of this possible? a) I don't want to create accounts for each user, can I have a mailboxes directory with the mailboxes of all the users? They are not many (10-15). b) The authentication right now is done through PAM, the pam.d/dovecot file is: auth required pam_unix_auth.so account required pam_unix_acct.so password required pam_unix_passwd.so session required pam_unix_session.so I want to authenticate against an SMB server, I added the following /etc/pam_smb.conf: SPECTRON-MSIM MAIN MAIN # documentation says I need 2 servers (only have one, is this ok) and changed the pam.d/dovecot to: auth required pam_smb_auth.so account required pam_smb_auth.so # there is no pam_smb_acct password required pam_smb_auth.so # there is no pam_smb_passwd session required pam_smb_auth.so # there is no pam_smb_session and it doesn't work. I obviously did something wrong, but don't know what, and I found nothing that could help me on google. I have samba installed and it works so It's PAM that I'm configuring incorrectly. Hope someone here can help me. Fawzib Rojas

2 1

[Dovecot] Quotas once again
by Mike Machado 14 Jun '03

14 Jun '03

Its been a while since I tried dovecot on my mail server where I have quota issues. I am using mbox file format. There still seems to be a quota problem. I have only /var with quotas and basedir is under /usr (not quotaed). default_mail_env = mbox:~/mail/:INBOX=/var/mail/%u. I tried with mbox_locks = dotlock fcntl (1st set of logs below) and with mbox_locks = fcntl (2nd set of logs) dotlock: Jun 12 17:02:49 newrook pop3(mike): open(/var/mail/mike.lock) failed: Disk quota exceeded Jun 12 17:02:49 newrook pop3(mike): file_lock_dotlock() failed with mbox file /var/mail/mike: Disk quota exceeded Jun 12 17:02:49 newrook pop3(mike): open(/var/mail/mike.lock) failed: Disk quota exceeded Jun 12 17:02:49 newrook pop3(mike): file_lock_dotlock() failed with mbox file /var/mail/mike: Disk quota exceeded Jun 12 17:02:49 newrook pop3: I/O leak: 0x8050930 (0) Jun 12 17:02:49 newrook pop3: I/O leak: 0x807baac (1) fcntl: Jun 12 16:45:32 newrook pop3-login: Login: mike [127.0.0.1] Jun 12 16:45:33 newrook pop3(mike): file_set_size() failed with mbox file /var/mail/mike: Disk quota exceeded Jun 12 16:45:33 newrook pop3(mike): file_set_size() failed with mbox file /var/mail/mike: Disk quota exceeded Jun 12 16:45:33 newrook pop3: I/O leak: 0x8073a2c (1) Jun 12 16:45:33 newrook pop3: I/O leak: 0x804a470 (0) I am using the latest CVS (as of 5 minutes ago). Wasn't this supposed to be working? Have I missed something? My sole goal is to allow people to delete mail when they are out of disk space. -- Mike Machado mike(a)innercite.com InnerCite Inc. Engineering Director / CTO

4 3

[Dovecot] trouble with ssl
by tom hensel 13 Jun '03

13 Jun '03

hi list, first of all, many thanks to the developers of dovecot for this great piece of software! i'm running dovecot 0.99.9.1 on netbsd-current - runs very fine until i try to use sylpheed from my client to connect via ssl: --snip-- Jun 13 08:48:32 devbox imap-login: SSL_accept() failed: error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac [192.168.100.88] Jun 13 08:48:32 devbox imap-login: Disconnected [192.168.100.8 --snip-- doesn't say much to me... thanks for any help, TOM

3 3

Re: [Dovecot] when/where to set MAIL env variable?
by Mike Brown 12 Jun '03

12 Jun '03

Andrew Basterfield wrote (in a msg I don't think made it to the list): > In dovecot.conf > > default_mail_env = maildir:~/Maildir Ah, silly me... default_mail_env = mbox:~/Mailbox works wonderfully, and is easier than trying to figure out where the MAIL environment variable actually comes into play. Thanks...

1 0

[Dovecot] when/where to set MAIL env variable?
by Mike Brown 12 Jun '03

12 Jun '03

I am trying to get dovecot to work on a system that uses qmail as its MTA. As is typical on such systems, the user's primary inbox is an mbox file called Mailbox, in their home directory. Global shell config files /etc/csh.cshrc and /etc/profile set the MAIL environment variable to ~/Mailbox for the users, which is enough to satisfy their interactive mail clients such as pine, mutt, and elm. The mail-storages.txt file in the dovecot docs seems to say that dovecot will not have a problem with this kind of setup. However, I can't figure out how to make it work. The MAIL variable is only set for users who are using shells like tcsh, bash, etc. The server apparently does not operate in a shell, so it is looking in some other location, perhaps /var/mail/$USER, so everyone's INBOX appears to be empty. It would help if mail-storages.txt, when referring to environment variables such as MAIL, would clarify *whose* environment variables are being referred to. Is it the environment of the user who built dovecot? Is it the environment of the user running the main dovecot process? Is it the environment of the user who is checking their mail / the user that dovecot runs setuid as? When and where do I set the MAIL variable so that dovecot will see it? Or is there some other problem? Any help appreciated...

1 0

[Dovecot] Help setting up SSL?
by Mike Matz 11 Jun '03

11 Jun '03

Hi, Sorry to bother, I don't know if this mailing list is the right place to look for help setting up dovecot.. Anyway I've got standard IMAP working fine, and I used mkcert.sh after editing the openssl config, and it seemed to work properly (I had to change some directories, I am using the FreeBSD 5.1-CURRENT port).. Anyway, when I try to connect using Outlook, here is what the log shows: (without the exact ip address shown, dont hax0r my windows machine!) imap-login: Jun 10 13:13:14 Warning: SSL_accept() failed: error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac [12.106.77.x] imap-login: Jun 10 13:13:14 Warning: SSL_accept() syscall failed: EOF [12.106.77.x] imap-login: Jun 10 13:13:14 Panic: key not found from hash dovecot: Jun 10 13:13:14 Error: child 32935 (login) killed with signal 6 imap-login: Jun 10 13:13:14 Warning: SSL_accept() failed: error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac [12.106.77.x] imap-login: Jun 10 13:13:14 Warning: SSL_accept() syscall failed: EOF [12.106.77.x] imap-login: Jun 10 13:13:14 Panic: key not found from hash dovecot: Jun 10 13:13:14 Error: child 32937 (login) killed with signal 6 so it says "decryption failed or bad record mac".. Does this mean I don't have my .pem files set up correctly? Or is there a problem with the OpenSSL libraries? I'd sure appreciate any help you could give me. Thanks, Mike

2 1

[Dovecot] gnutls_handshake problem
by Jacob Elder 07 Jun '03

07 Jun '03

I'm trying to bring dovecot up on Debian/stable using Christof Meerwald's unofficial packages (cmeerw.org/debian/) This package seems to work fine on Debian/testing. There error I'm getting on the stable box is: gnutls_handshake: A TLS fatal alert has been received.(Bad record MAC) Any guesses? -- Jacob Elder

4 3

[Dovecot] Dovecot for Gentoo portage
by Zach Bagnall 04 Jun '03

04 Jun '03

Hi all - I've added an ebuild for 0.99.9.1 to bugzilla: http://bugs.gentoo.org/show_bug.cgi?id=22080 It includes the CR+LF patch. Should be trivial to update for .10 when it makes an appearance :-). Feel free to fix/improve this ebuild - I'm fairly new at them. Zach. -- Services & Support Bulletin Wireless | www.bulletinwireless.com This email is digitally signed | Key ID 0x3F9AA9A2

1 0

[Dovecot] error
by Farkas Levente 03 Jun '03

03 Jun '03

hi, I've got the following alert window in mozilla (with dovecot): --------------------------------------- The current command did not succeed. The mail server responded: Error in IMAP command: Too long argument.. --------------------------------------- although there is not any messages in my imap.log file. -- Levente "Si vis pacem para bellum!"

2 1

[Dovecot] dovecot 0.99.10-test10
by Andrew Basterfield 31 May '03

31 May '03

Hiya builds fine on BSD now (thanks) Now I am getting index corruption again May 30 03:11:53 snigger imap(bob): Corrupted binary tree file /home/bob/Mail/.INBOX/.imap.index.tree: UID to be inserted isn't higher than existing (1 <= 1) and so on until the client hangs test7 is stable here. Also what are the correct permissions for /var/run/dovecot/login? I get a message about correcting the permissions every time I start dovecot --Andrew -- http://cemetery.homeunix.org

2 3

[Dovecot] the most serius problems [part 3]
by Farkas Levente 30 May '03

30 May '03

hi, so the third problem is ssl. the imaps is still not working (and I assume the same apply to pop3s too). I already send the log file which was generated when verbose ssl is set. -- Levente "Si vis pacem para bellum!"

4 3

[Dovecot] the most serius problems [part 2]
by Farkas Levente 30 May '03

30 May '03

hi, the second thing is the webmail. we use squirrelmail. actualy I don't know whose fault is, but they told that them imap server's:-) first problems is that those mails which has empty body (even if they has an attachments) simple not shown in the mail list (as they even not exists). it'd be nice to check this and find the reason of the problem. this happend with more people. -- Levente "Si vis pacem para bellum!"

2 2

[Dovecot] the most serius problems [part 1]
by Farkas Levente 30 May '03

30 May '03

hi, I try to collect out most serious problem with dovecot. so I write a series of mails:-) the first on is the load. has anybody else use dovecot in a real enviromnt with at least hundreds of users and at least 50 concurrent user at the same time? it seems to me no one else just me. the weekend I move the whole mail system to a quad P4 with 1Gb ram and this server do nothing just imap. yesterday evening I have to delete all .imap.index and other dovecot specific files. and as I wrote today morning the load was about 20-30 !!!. but even in an average day the load is over 2-3 white the cpu load is 1-2% in each of the 4 processor. the system is totaly unusable. what more today I've got other errors. of course all the time when I can't find the colution I try to escape to the latest cvs. today finally I compile a new courier install it to another port and those clients which can't connect to dovecot reconfigure to courier. but they are not compatible!!! I have to set root directory to INBOX in courier while not for dovecot and this is a real big problem. is there any way to compile dovecot to use INBOX as a root directory? since in this case at least there is a chance to switch to courier without reconfigure 300 OE6. I try to help to find out the reason for the load in any way, but it seems to me than it's better to change to courier or cyrus since our users getting angry not to mention my boss. -- Levente "Si vis pacem para bellum!"

3 7

[Dovecot] 0.99.10-test8
by Timo Sirainen 30 May '03

30 May '03

http://dovecot.procontrol.fi/test/ Again major maildir syncing updates: - uidlist file is accessed and rewritten a lot less often now than before - mail duplicates are detected and handled properly by renaming them to new base name - message flag updates and expunging before did two full syncs for cur/ directory. now it does only one. - some fixes to syncing in read-only / out-of-quota conditions - before we did always a full sync in cur/ directory when mailbox was opened. now it's done only when needed. - fixed some crash/index corruption cases I think syncing should be now very close to having perfect possible performance.

2 3

[Dovecot] new problems
by Farkas Levente 29 May '03

29 May '03

hi, so yesterday I move the whole mail system to a better machine. the only thing which is running on this system is dovect and: -------------------------------------------------- 10:44am up 6 days, 25 min, 3 users, load average: 28.03, 17.96, 9.05 233 processes: 231 sleeping, 2 running, 0 zombie, 0 stopped CPU0 states: 3.1% user, 29.4% system, 0.0% nice, 66.4% idle CPU1 states: 7.1% user, 63.2% system, 0.0% nice, 29.1% idle CPU2 states: 1.3% user, 3.1% system, 0.0% nice, 94.4% idle CPU3 states: 1.1% user, 7.4% system, 0.0% nice, 90.3% idle Mem: 1030496K av, 1021836K used, 8660K free, 0K shrd, 104064K buff Swap: 2040244K av, 55544K used, 1984700K free 776180K cached -------------------------------------------------- so the problem here is again with the load 28 (!!!)..... in the imap.log: -------------------------------------------------- imap(arvays): May 28 10:39:32 Error: Timeout while waiting for release of fcntl() lock for index file /home/arvays/Maildir/.INBOX/.imap.index imap(lfarkas): May 28 10:42:05 Panic: unreached dovecot: May 28 10:42:07 Error: child 19933 (imap) killed with signal 6 imap(ahegedus): May 28 11:05:57 Panic: file maildir-sync.c: line 472 (maildir_sync_uidlist): assertion failed: (ACTION(hash_rec) == MAILDIR_FILE_ACTION_NEW) dovecot: May 28 11:05:58 Error: child 29103 (imap) killed with signal 6 -------------------------------------------------- -- Levente "Si vis pacem para bellum!"

2 7

[Dovecot] still has error
by Farkas Levente 28 May '03

28 May '03

hi, after download the current cvs recompile, start I still has these errors: ------------------------------------- imap-login: May 28 13:49:02 Info: Login: jcsinalos [192.168.0.106] imap(jcsinalos): May 28 13:49:02 Panic: unreached dovecot: May 28 13:49:02 Error: child 31683 (imap) killed with signal 6 imap-login: May 28 13:49:03 Info: Login: jcsinalos [192.168.0.106] imap-login: May 28 13:49:03 Info: Login: jcsinalos [192.168.0.106] imap(jcsinalos): May 28 13:49:03 Panic: unreached dovecot: May 28 13:49:03 Error: child 31687 (imap) killed with signal 6 ------------------------------------- -- Levente "Si vis pacem para bellum!"

2 3

[Dovecot] 0.99.10-test7
by Timo Sirainen 28 May '03

28 May '03

- APPEND memory leak fixed - maildir syncing crash / some other errors fixed - pop3 mail fetching fixed - pop3 didn't advertise STLS in capabilities before http://dovecot.procontrol.fi/test/

1 0

[Dovecot] a few more error
by Farkas Levente 28 May '03

28 May '03

in the imap.log: imap(arvays): May 28 10:39:32 Error: Timeout while waiting for release of fcntl() lock for index file /home/arvays/Maildir/.INBOX/.imap.index imap(lfarkas): May 28 10:42:05 Panic: unreached dovecot: May 28 10:42:07 Error: child 19933 (imap) killed with signal 6 -- Levente "Si vis pacem para bellum!"

1 0

[Dovecot] serious problem
by Farkas Levente 28 May '03

28 May '03

hi, I've got dozen such errors: ---------------------------- imap-login: May 28 10:28:56 Info: Login: gpajlocz [192.168.1.182] imap(gpajlocz): May 28 10:28:56 Panic: file maildir-sync.c: line 472 (maildir_sync_uidlist): assertion failed: (ACTION(hash_rec) == MAILDIR_FILE_ACTION_NEW) dovecot: May 28 10:28:56 Error: child 17207 (imap) killed with signal 6 ---------------------------- and this users can't get into the system... -- Levente "Si vis pacem para bellum!"

1 0

[Dovecot] Dovecot 0.99.9.1 and openssl-0.9.7a-5
by Robin Whittle 28 May '03

28 May '03

Hi Timo and others, Thanks for Dovecot! I just got it running under Red Hat 9.0 after spending many hours unsuccessfully trying to get Courier IMAP to run, although I have done it in the past. Here is a compilation glitch which may already be well known: I compiled dovecot-0.99.9.1 on my newly installed and recently updated (via Red Hat Network) system and the compilation stopped due to a missing file in the SSL headers: In file included from /usr/include/openssl/ssl.h:179, from ssl-proxy-openssl.c:14: /usr/include/openssl/kssl.h:72:18: krb5.h: No such file or directory In file included from /usr/include/openssl/ssl.h:179, from ssl-proxy-openssl.c:14: The currently installed OpenSSL RPM was openssl-0.9.7a-5. This lacks the file /usr/include/openssl/krb5.h . I understand that openssl-0.9.7b is OK, but I wasn't sure about whether installing this as a tarball from http://www.openssl.org would upset any other programs on this machine, or confuse the RHN update system. I didn't really need SSL and I couldn't see a specific option for disabling it, other than a general instruction about disabling modules with a ./configure option. So I tried ./configure --help and compiled it with: ./configure --without-ssl This worked fine. I really like the report at the end of the ./configure stage saying how the program will be compiled! - Robin // Robin Whittle http://www.firstpr.com.au // Melbourne, Australia http://fondlyandfirmly.com

1 0