[Dovecot] OS X-centricities? Too many files, temp, indexes, etc...
I'm just writing to see if anyone else is seeing the following issues, and particularly if they are running OS X (especially 10.4), and/or have compatible solutions (I've been reading the mailing list quite a bit, and every now and then, I find a solution that just "doesn't work" because of the differences in OS:
Too many files open - no plimit command on OS X, so I can't figure out how to increase the max number of open files... Also, are there any alternative means of CLOSING the files? stopping/restarting dovecot didn't seem to help;
I was running 1.0beta1 last week, and am now on 1.0beta3, but I'm still not seeing corrupt indexes/other dovecot files rebuilding themselves. I really can't stick with this solution if I'll always have to worry about manually fixing broken caches/indexes.
Any advice on fcntl/flock on OS X, particularly? They both appear to work, but I'd prefer to use the fastest/least resource intensive options for everything if possible.
Thanks!
-deano
On Tue, Mar 21, 2006 at 02:26:00PM -0800, Dean Blackburn wrote:
I'm just writing to see if anyone else is seeing the following issues, and particularly if they are running OS X (especially 10.4), and/or have compatible solutions (I've been reading the mailing list quite a bit, and every now and then, I find a solution that just "doesn't work" because of the differences in OS:
- Too many files open - no plimit command on OS X, so I can't figure out how to increase the max number of open files... Also, are there any alternative means of CLOSING the files? stopping/restarting dovecot didn't seem to help;
You can use limit/ulimit - depending on how you start dovecot. (If you use launchd, take a look at 'launchctl limit maxfiles'.)
If you need to change it for the system, you can use sysctl - though at least on my machine the default seem more than adequate:
[p7:1004] ~%sysctl -a | grep files 6:01PM kern.maxfiles = 12288 kern.maxfilesperproc = 10240
-- Nicholas Riley njriley@uiuc.edu | http://www.uiuc.edu/ph/www/njriley
On Tue, 2006-03-21 at 14:26 -0800, Dean Blackburn wrote:
I'm just writing to see if anyone else is seeing the following issues, and particularly if they are running OS X (especially 10.4), and/or have compatible solutions (I've been reading the mailing list quite a bit, and every now and then, I find a solution that just "doesn't work" because of the differences in OS:
- Too many files open - no plimit command on OS X, so I can't figure out how to increase the max number of open files... Also, are there any alternative means of CLOSING the files? stopping/restarting dovecot didn't seem to help;
Which process is running out of them? There are multiple Dovecot processes, and each could have a different reason of running out of file handles.
- I was running 1.0beta1 last week, and am now on 1.0beta3, but I'm still not seeing corrupt indexes/other dovecot files rebuilding themselves. I really can't stick with this solution if I'll always have to worry about manually fixing broken caches/indexes.
First of all they shouldn't anymore be corrupting themselves at least easily. What exactly are the error messages you're seeing?
Also as far as I know they're rebuilding themselves automatically in most situations. Only if Dovecot crashes because of the corruption it doesn't get rebuilt. If it's not crashing with you, could it be just that they keep re-breaking all the time?
Timo Sirainen wrote:
- Too many files open - no plimit command on OS X, so I can't figure out how to increase the max number of open files... Also, are there any alternative means of CLOSING the files? stopping/restarting dovecot didn't seem to help;
Which process is running out of them? There are multiple Dovecot processes, and each could have a different reason of running out of file handles.
dovecot: Mar 24 11:35:51 Error: imap(yukimurashige): pipe() failed: Too many open files dovecot: Mar 24 11:35:51 Error: child 21465 (imap) returned error 89
- I was running 1.0beta1 last week, and am now on 1.0beta3, but I'm still not seeing corrupt indexes/other dovecot files rebuilding themselves. I really can't stick with this solution if I'll always have to worry about manually fixing broken caches/indexes.
First of all they shouldn't anymore be corrupting themselves at least easily. What exactly are the error messages you're seeing?
Lots of this:
dovecot: Mar 24 12:18:45 Error: imap(mikitanaka): Corrupted index file /Users/mikitanaka/Maildir/dovecot.index: uid_validity = 0, next_uid = 1366
Some of this:
dovecot: Mar 24 10:55:45 Error: imap(kristinegivas): imap(19936) malloc: *** error for object 0x602200: incorrect checksum for freed object - object was probably modified after being freed, break at szone_error to debug
And occasionally this:
dovecot: Mar 24 11:00:14 Error: imap(deanblackburn): Maildir /Users/deanblackburn/Maildir sync: UID inserted in the middle of mailbox (1042 > 1041, file = 1143071249.Ve000008I8e7f3b40.mailserv.local:2,Sk)
That's just yesterday's errors, we've seen some others as well. But they all go away if one deletes dovecot* from the affected directory, and they don't go away on their own. Also, we'll see a huge number of temp.* files build up within the maildir once these files "go bad"...
From the client perspective, you either get a "connection timed out" error, or on some less robust clients, "connection refused by IMAP server".
Also as far as I know they're rebuilding themselves automatically in most situations. Only if Dovecot crashes because of the corruption it doesn't get rebuilt. If it's not crashing with you, could it be just that they keep re-breaking all the time?
Definitely possible. ;) How can we tell?
If we can solve these issues, we'll be very happy with Dovecot... Courier seems a lot more solid, in retrospect, but required so much disk access it was completely unusable by our staff. In contrast, Dovecot is fast, light on resources, but will break around 5-10 individuals' mailboxes every day. So far, we just can't win!
-deano
participants (3)
-
Dean Blackburn
-
Nicholas Riley
-
Timo Sirainen