[Dovecot] compressed mboxes very slow

[Ed W]

On 12/01/2012 10:39, Kamil Jońca wrote:
> kjonca at o2.pl (Kamil Jońca) writes:
>
>> I have some archive mails in gzipped mboxes. I could use them with
>> dovecot 1.x without problems.
>> But recently I have installed dovecot 2.0.12, and they are slow. very
>> slow.
>
> Recently I have to read some compressed mboxes again, and no progress :(
> I took 2.0.17 sources and   put some
> 	i_debug ("#kjonca["__FILE__",%d,%s] %d",  __LINE__,__func__,...some parameters ...);	
>
> lines into istream-bzlib.c, istream-raw-mbox.c and istream-limit.c
> and found that:
>
> in istream-limit.c in function around lines 40-45:
> --8<---------------cut here---------------start------------->8---
>   i_stream_seek(stream->parent, lstream->istream.parent_start_offset +
>                     stream->istream.v_offset);
>   stream->pos -= stream->skip;
>   stream->skip = 0;
> --8<---------------cut here---------------end--------------->8---
> seeks stream, (calling i_stream_raw_mbox_seek in file istream-raw-mbox.c )
>
> and then (line 50 )
> --8<---------------cut here---------------start------------->8---
>          if ((ret = i_stream_read(stream->parent)) == -2)
>              return -2;
> --8<---------------cut here---------------end--------------->8---
>
> tries to read some data earlier in stream, and with compressed mboxes it
> cause reread file from the beginning.
>

Just wanted to bump this since it seems interesting.  Timo do you have a 
comment?

I definitely see your point that skipping backwards in a compressed 
stream is going to be very CPU intensive.

Ed W