On Fri, 2007-03-30 at 15:39 +0200, Steffen Kaiser wrote:
dovecot: Mar 30 08:49:27 Error: IMAP(dvtest2) [12578]: Transaction log file /var/cache/dovecot/31022/index/.box1/dovecot.index.log: marked corrupted dovecot: Mar 30 08:49:28 Error: IMAP(dvtest2) [12581]: Corrupted transaction log file /var/cache/dovecot/31022/index/.box1/dovecot.index.log: Append with UID 168554, but next_uid = 168555 dovecot: Mar 30 08:49:28 Error: IMAP(dvtest2) [12488]: Transaction log file /var/cache/dovecot/31022/index/.box1/dovecot.index.log: marked corrupted
These and the duplicate link errors are because of maildir_copy_preserve_filename=yes. With =no it shouldn't log any errors.
box1 (the copybox) contains 168778 messages. When I first selected the box1, there were some "Fixed a duplicate link" Warnings, then Thunderbird tried to download all the message headers. It never returned to something useful state. straceing the imap process I saw that Dovecot reads all messages fully, header & body, then sends the client:
- 49136 FETCH (UID 49235 RFC822.SIZE 7426 FLAGS (\Recent) BODY[HEADER.FIELDS (FROM TO CC SUBJECT DATE MESSAGE-ID PRIORITY X-PRIORITY REFERENCES NEWSGROUPS IN-REPLY-TO CONTENT-TYPE)] {668} And the headers.
For all the messages I guess? So was there a Dovecot bug related to this, or was it just that Thunderbird doesn't like that many messages?
When I tried to delete all the messages through the "Trashbox", the copy-to-Trash process fails with (see end of mail). Then the process starts again with uid=1, again the same error is issued at the same uid. After the first "Out-of-mem" condition, I see "delete uid 1" in the log, but the messages are still there and the COPY is still performed again.
Hmm. So copying 180k messages eats 250MB of memory? That sounds a bit much, but I guess it's possible.. Maildir code takes around 80-100 bytes/message, index file code takes 8 bytes/message + more for keywords. But even assuming about the worst case scenarios, it shouldn't take more than 30MB. Wonder where the rest is going.
Dunno, I simply suggest that the admin may limit the number of items a COPY command may be performed for. I mean, one can move/copy/delete-to-trash mails in chunks of 50'000 messages - nobody sane has that many message in one mailbox, hasn't one?
Right, so this practically never happens with real users. I don't want to add any more settings that are mostly useless.
You can anyway make it work by growing mail_process_size.