[Dovecot] Copy from INBOX to INBOX corrupts file
Aloha, again, Timo.
Concerning the problem of copying within one folder...
I'm still not sure it's not a problem I created myself, but I think it's caused by the use of the buffered IO routines in ostream-file.c ... you use io_add() to schedule write operations for later while simultanously reading new chunks from the very same file. This seems to confuse dovecot and make it write the copy to the strangest locations. I think when working within one folder I should use either blocking IO or a lot of calls to o_stream_flush()... what do you think?
Greetings, Andy
On Thu, 2003-10-16 at 19:06, Andreas Jaekel wrote:
Concerning the problem of copying within one folder...
I'm still not sure it's not a problem I created myself,
No, it's with mbox too.
but I think it's caused by the use of the buffered IO routines in ostream-file.c ... you use io_add() to schedule write operations for later while simultanously reading new chunks from the very same file. This seems to confuse dovecot and make it write the copy to the strangest locations. I think when working within one folder I should use either blocking IO or a lot of calls to o_stream_flush()... what do you think?
I think it is using blocking I/O? I'm not actually sure if Dovecot ever really uses the io_add() there anymore..
mbox-save.c: ctx->output = o_stream_create_file(ibox->index->mbox_fd, default_pool, 4096, FALSE); o_stream_set_blocking(ctx->output, 60000, NULL, NULL);
Aloha Timo.
I had some further looks into the folder corruption issue and it looks like non-blocking routines are being used. Could it be possible that you are setting the istream to non-blocking and that influences the ostream's/iostream's timeout_msecs settings?
I got rid of the problem by using your pread() patch and doing seeks on output in mail_storage_save() and write_func(). Because ostream-file.c/_read() is somehow thinking ostream is non-blocking I had to do a call to o_stream_flush() after writing the body, too, or else a stray llseek() on input would make me write the body to a very unfortunate position in the file.
io_add() is not being used, but the last bits of data were still just being written to an iov and write() was called only after some other operation.
At 19:15 16/10/2003 +0300, Timo Sirainen wrote:
On Thu, 2003-10-16 at 19:06, Andreas Jaekel wrote:
Concerning the problem of copying within one folder...
I'm still not sure it's not a problem I created myself,
No, it's with mbox too.
but I think it's caused by the use of the buffered IO routines in ostream-file.c ... you use io_add() to schedule write operations for later while simultanously reading new chunks from the very same file. This seems to confuse dovecot and make it write the copy to the strangest locations. I think when working within one folder I should use either blocking IO or a lot of calls to o_stream_flush()... what do you think?
I think it is using blocking I/O? I'm not actually sure if Dovecot ever really uses the io_add() there anymore..
mbox-save.c: ctx->output = o_stream_create_file(ibox->index->mbox_fd, default_pool, 4096, FALSE); o_stream_set_blocking(ctx->output, 60000, NULL, NULL);
On Tue, 2003-10-28 at 13:39, Andreas Jaekel wrote:
Aloha Timo.
I had some further looks into the folder corruption issue and it looks like non-blocking routines are being used. Could it be possible that you are setting the istream to non-blocking and that influences the ostream's/iostream's timeout_msecs settings?
I don't think it should be set non-blocking.. What makes it look like they're used?
io_add() is not being used, but the last bits of data were still just being written to an iov and write() was called only after some other operation.
Yes, flushing has to be done to ostream since it does buffer data even in blocking mode.
I just did several changes in CVS to make copying inside same mbox working. Lets see if those changes corrupt my mboxes.. :)
participants (2)
-
Andreas Jaekel
-
Timo Sirainen