[Dovecot] Purging a large folder... anything I need to be careful of?
I've got a large message folder (about 40G in 'cur') that I need to empty to a more reasonable size. Oddly, my Thunderbird e-mail client only sees about 2.4G of that, and has "looped" a few times. What I mean is, I thought someone was cleaning the folder out, but Thunderbird apparently was actually just looping back to 0 every 4 gigs and starting to count up again Ha. But that's a sidenote. So, since Thunderbird apparently is not seeing all of the messages, what is the safest way to delete the files straight from the 'cur' folder? IS that safe? Will I mess up any of the dovecot indexes in any way?? Dave
Dave schrieb:
But it's a strange one. I have some fairly big folders (>>10G) too but there is no problem at all with TB 2.0.0.23.
Yes it is safe. Dovecot has self healing abilities which will fix the index files.
Dave
On 2010-02-10 12:59 AM, Dennis Guhl wrote:
Not entirely accurate. TB does indeed have a big problem with mbox files larger than 4GB. But as long as you don't set an IMAP folder that is larger than 4GB to full offline mode, you should be fine, BUT... the calculated size will be displayed using a 32-bit value, so as it calculates the size, when it hits 4GB, it will roll over and start again
- so, a 6GB and a 10GB folder will both be displayed as 2GB...
apparently is not seeing all of the messages,
Yes it is - it is simply not calculating the folder size correctly. I don't know if this an underlying OS issue (32-bit vs 64-bit), or a TB bug...
I don't think dovecots indexes are affected by this - it is merely a cosmetic TB issue.
--
Best regards,
Charles
Charles Marcus schrieb:
Oh, I think that should be a bug, since today files with more then 4GB are nor rare. But using only IMAP I never encountered this problem and wasn't aware of it.
OK, this I never observed. But there are no quotas on the folder so I never took a look at the size with TB.
And what is displayed as the message count if exceed 32 bit?
When you go to ~/Maildir/cur/ to remove messages (as dave intend) dovecots indexes will indeed be affected. But it will cause no trouble beside throwing an entry in the log.
On Wed, 2010-02-10 at 13:40 +0100, Dennis Guhl wrote:
It shouldn't even log anything. Dovecot's Maildir code was built so that it expected there to be other non-Dovecot MUAs accessing and changing the Maildir. Only recently I added maildir_very_dirty_syncs=yes setting where it assumes it's the only thing accessing maildir to improve performance.
On 2010-02-10 7:40 AM, Dennis Guhl wrote:
Oh, I think that should be a bug, since today files with more then 4GB are nor rare.
I agree... I've been considering opening a bug about this, but it hasn't been high on my list since I don't have my large folders set to full offline mode (just my Inbox mainly)...
But using only IMAP I never encountered this problem and wasn't aware of it.
You would if you had set any of the folders larger than 4GB to full offline mode and sync'd them... ;)
But there are no quotas on the folder so I never took a look at the size with TB.
I always enable extra columns in the folder pane and add the size column, so I can always see how big my folders are...
And what is displayed as the message count if exceed 32 bit?
Message count is correctly displayed, as far as I've been able to determine.
I don't think dovecots indexes are affected by this - it is merely a cosmetic TB issue.
Yes, of course the indexes will be rebuilt, but that is normal when messages are deleted/added. I meant they won't be affected in any abnormal way...
--
Best regards,
Charles
Charles Marcus schrieb:
On 2010-02-10 7:40 AM, Dennis Guhl wrote:
Charles Marcus schrieb:
I have no folder at all set to offline mode. So it would make no sense for to file this bug since I can't give any hints what goes wrong. Maybe someone like Dave with a more urgent need will file it.
[..]
Yes, of course.
Ahh, you were right about this, I did see later that all the messages *were* there in fact, but it does "loop" the folder size as was stated (6G looks like 2G in TB, etc). I appreciate all the info from everyone, and I was able just to just move all of the 40G around using Thunderbird once I saw that all the messages were indeed there. Took a while :), but now we have some things archived and folders are down to a more "reasonable" size (reasonable for us anyway).
It is good to know that removing messages directly from the source folders won't mess up anything as far as Dovecot or its indexes is concerned though, so, thanks for that information everyone!! Dave
Ahh, you were right about this, I did see later that all the messages *were* there in fact, but it does "loop" the folder size as was stated (6G looks like 2G in TB, etc). I appreciate all the info from everyone, and I was able just to just move all of the 40G around using Thunderbird once I saw that all the messages were indeed there. Took a while :), but now we have some things archived and folders are down to a more "reasonable" size (reasonable for us anyway).
It is good to know that removing messages directly from the source folders won't mess up anything as far as Dovecot or its indexes is concerned though, so, thanks for that information everyone!! Dave
participants (5)
-
Charles Marcus
-
Dave
-
David Salisbury
-
Dennis Guhl
-
Timo Sirainen