hi, so after upgrade to the current cvs all of our problem disappear:-) but we don't have time to test the ssl part... although after I compile and install the new dovecot I stop it I do a
find /home/*/Maildir -name '.imap*' -exec rm {} \; find /home/*/Maildir -name 'dovecot-uidlist' -exec rm {} \; find /home/*/Maildir -name 'msgid.cache' -exec rm {} \; find /home/*/Maildir -name '.customflags' -exec rm {}\;
just to be sure that we use a fresh version and no other problem can be caused by the old index or any other cached files. this was a BIG mistake!!! we've got about 300 users and about 100 online at the same time. after a two minutes break (while I reinstall and delete) every user check his mail which casue a load about 90!!! the cpu usage 1-2% but the avarage load was 90 (curently about 0.22)! the reason was IO. we have to stop imap and wait for a while... I don't know the exact reason bu I assume dovecot is regenerate everybody's index a other cache at the same time. is there any way to reduce the possible number of IO load? this would be useful if never happend again!:-)
this happend around 4:40. the output of sar -b:
03:00:00 PM tps rtps wtps bread/s bwrtn/s 03:10:00 PM 48.18 29.19 18.99 1456.53 771.29 03:20:00 PM 41.66 25.80 15.86 1054.88 830.05 03:30:00 PM 42.64 25.67 16.97 1223.03 759.77 03:40:00 PM 44.98 31.95 13.03 1959.15 467.40 03:50:00 PM 26.68 14.63 12.05 741.48 424.12 04:00:00 PM 25.80 11.70 14.10 516.21 450.04 04:10:00 PM 59.55 34.91 24.64 1755.07 939.11 04:20:00 PM 95.98 61.15 34.83 4453.24 1563.25 04:30:00 PM 95.08 55.45 39.63 2182.22 1591.56 04:40:00 PM 168.85 145.99 22.86 11155.73 1037.25 04:50:00 PM 135.98 106.50 29.48 7908.21 1604.76 05:00:00 PM 125.87 97.65 28.23 5876.03 1060.50 05:10:00 PM 80.83 62.92 17.90 4321.82 590.19 05:20:00 PM 102.04 77.65 24.40 4907.27 1076.92 05:30:00 PM 129.04 106.58 22.46 6387.18 883.13 05:40:00 PM 89.44 73.20 16.24 4905.64 782.94 05:50:00 PM 88.23 73.16 15.06 5539.93 965.34 06:00:00 PM 92.49 72.67 19.81 5486.25 692.18 06:10:00 PM 45.24 32.82 12.42 1711.35 618.31 06:20:00 PM 42.17 28.16 14.01 960.11 622.33 06:30:00 PM 29.90 20.24 9.66 1538.27 376.17 06:40:00 PM 25.89 12.14 13.74 796.73 833.25 06:50:00 PM 20.50 4.98 15.52 264.91 993.62 07:00:00 PM 15.08 7.57 7.52 453.41 282.63
the output of sar -d
04:10:00 PM dev22-3 0.00 0.00 04:10:00 PM dev33-0 1.76 109.55 04:10:00 PM dev34-0 8.51 919.94 04:20:00 PM dev3-0 8.56 183.93 04:20:00 PM dev22-2 67.62 3798.02 04:20:00 PM dev22-3 0.00 0.00 04:20:00 PM dev33-0 4.80 463.81 04:20:00 PM dev34-0 15.01 1570.73 04:30:00 PM dev3-0 4.40 126.63 04:30:00 PM dev22-2 77.02 2440.08 04:30:00 PM dev22-3 0.00 0.00 04:30:00 PM dev33-0 1.94 124.80 04:30:00 PM dev34-0 11.73 1082.28 04:40:00 PM dev3-0 4.78 124.56 04:40:00 PM dev22-2 137.27 10306.18 04:40:00 PM dev22-3 0.00 0.00 04:40:00 PM dev33-0 13.50 410.44 04:40:00 PM dev34-0 13.29 1351.80 04:50:00 PM dev3-0 6.33 194.27 04:50:00 PM dev22-2 111.50 7961.44 04:50:00 PM dev22-3 0.00 0.00 04:50:00 PM dev33-0 5.64 156.51
-- Levente "Si vis pacem para bellum!"
On Tue, 2003-04-15 at 20:21, Farkas Levente wrote:
hi, so after upgrade to the current cvs all of our problem disappear:-) but we don't have time to test the ssl part... although after I compile and install the new dovecot I stop it I do a
find /home/*/Maildir -name '.imap*' -exec rm {} \;
This is fine, but as you noticed regenerating indexes isn't very fast :) Also currently Dovecot doesn't behave too nicely if the index gets deleted while it was open.
find /home/*/Maildir -name 'dovecot-uidlist' -exec rm {} \;
There shouldn't be any need to do this. Point of it was that the UIDs will be remembered if indexes are deleted so client doesn't have to invalidate local cache.
find /home/*/Maildir -name 'msgid.cache' -exec rm {} \;
Not created by Dovecot.
find /home/*/Maildir -name '.customflags' -exec rm {}\;
Should never be deleted with maildir. This contains user-defined flag names so it could cause some data loss.
I don't know the exact reason bu I assume dovecot is regenerate everybody's index a other cache at the same time. is there any way to reduce the possible number of IO load? this would be useful if never happend again!:-)
Currently it re-reads all the mails in the directory and indexes them. I'll make this configurable some day.
Timo Sirainen wrote:
On Tue, 2003-04-15 at 20:21, Farkas Levente wrote:
hi, so after upgrade to the current cvs all of our problem disappear:-) but we don't have time to test the ssl part... although after I compile and install the new dovecot I stop it I do a
find /home/*/Maildir -name '.imap*' -exec rm {} \;
This is fine, but as you noticed regenerating indexes isn't very fast :) Also currently Dovecot doesn't behave too nicely if the index gets deleted while it was open.
find /home/*/Maildir -name 'dovecot-uidlist' -exec rm {} \;
There shouldn't be any need to do this. Point of it was that the UIDs will be remembered if indexes are deleted so client doesn't have to invalidate local cache.
find /home/*/Maildir -name 'msgid.cache' -exec rm {} \;
Not created by Dovecot.
find /home/*/Maildir -name '.customflags' -exec rm {}\;
Should never be deleted with maildir. This contains user-defined flag names so it could cause some data loss.
I don't know the exact reason bu I assume dovecot is regenerate everybody's index a other cache at the same time. is there any way to reduce the possible number of IO load? this would be useful if never happend again!:-)
Currently it re-reads all the mails in the directory and indexes them. I'll make this configurable some day.
the whole mails or just the headers? as you know we've a lots of big mails so this makes a real different!
ok so what is the best way to be sure that no cached information can confuse the new version? just:
- stop dovecot
- kill all imap process
- find /home/*/Maildir -name '.imap*' -exec rm {} \;
- start dovecot ?
-- Levente "Si vis pacem para bellum!"
On Wed, 2003-04-16 at 11:34, Farkas Levente wrote:
Currently it re-reads all the mails in the directory and indexes them. I'll make this configurable some day.
the whole mails or just the headers? as you know we've a lots of big mails so this makes a real different!
The whole file, but you can disable that by removing MessagePart from mail_cache_fields in config file. I think I'll fix this even more so that the file isn't opened at all if nothing is wanted to be cached.
ok so what is the best way to be sure that no cached information can confuse the new version? just:
- stop dovecot
- kill all imap process
- find /home/*/Maildir -name '.imap*' -exec rm {} \;
- start dovecot
Well, yes, to be safe. But usually you shouldn't need to delete the .imap* files at all. If there's any changes in format or whatever, it's rebuilt automatically. It's needed only if it contains actually invalid data that can't be detected, like there cached ENVELOPE texts were wrong before.
Timo Sirainen wrote:
On Wed, 2003-04-16 at 11:34, Farkas Levente wrote:
Currently it re-reads all the mails in the directory and indexes them. I'll make this configurable some day.
the whole mails or just the headers? as you know we've a lots of big mails so this makes a real different!
The whole file, but you can disable that by removing MessagePart from mail_cache_fields in config file. I think I'll fix this even more so that the file isn't opened at all if nothing is wanted to be cached.
at noon we switch to mail_cache_fields = after that the the clients are usable again. BUT what we can do? what would be the best choise for us? the parameters:
- most people don't know anything about computer, mailer, internet!!
- they use to write email in word's doc!
- they attach pictures, exe, doc etc to their mails
- they use OE6 which can't create rules on imap folder, can't delete mails one by one there is no thrash (only purge all deleted mails) which casue mailbox with a lots of mails (most od them already deleted, but wouldn't like to delete them since cant select them).
so:
- we've got about 300 mailbox
- most people just has mails in his inbox and sent folder
- this folder are real huge (a few thousend mails)
- 300 mailbox has 100000 mails which is about 16GB this means the avarage message size is about 160K (actualy the size are 5-10K or 1-5M)
so what extra (non default) parameters should I have to use (eg: mail_cache_fields etc.)
ok so what is the best way to be sure that no cached information can confuse the new version? just:
- stop dovecot
- kill all imap process
- find /home/*/Maildir -name '.imap*' -exec rm {} \;
- start dovecot
Well, yes, to be safe. But usually you shouldn't need to delete the .imap* files at all. If there's any changes in format or whatever, it's rebuilt automatically. It's needed only if it contains actually invalid data that can't be detected, like there cached ENVELOPE texts were wrong before.
they were wrong in any previous version? or how can I know that?
-- Levente "Si vis pacem para bellum!"
On Wed, 2003-04-16 at 17:20, Farkas Levente wrote:
at noon we switch to mail_cache_fields = after that the the clients are usable again.
test5 should be even more usable with this.
BUT what we can do? what would be the best choise for us? the parameters: ..
- we've got about 300 mailbox
- most people just has mails in his inbox and sent folder
- this folder are real huge (a few thousend mails)
- 300 mailbox has 100000 mails which is about 16GB this means the avarage message size is about 160K (actualy the size are 5-10K or 1-5M)
so what extra (non default) parameters should I have to use (eg: mail_cache_fields etc.)
There's not really other settings that affect it. I've tried to make Dovecot handle large mailboxes fast by default, but I haven't really thought of the rebuild slowdown before.
Well, yes, to be safe. But usually you shouldn't need to delete the .imap* files at all. If there's any changes in format or whatever, it's rebuilt automatically. It's needed only if it contains actually invalid data that can't be detected, like there cached ENVELOPE texts were wrong before.
they were wrong in any previous version? or how can I know that?
NEWS file contains everything that's needed to know when upgrading, including if index files should be deleted because of old bugs. Except I forgot to update it for 0.99.8.1 which fixed one of these.
participants (2)
-
Farkas Levente
-
Timo Sirainen