On Tue, 2014-05-27 at 15:44 +0200, Robert Schetterer wrote:

Am 27.05.2014 14:56, schrieb Arthur Dent:

...

Hello All,

I have a Dovecot v2.2.12 installation running on a Fedora 20 box which is the mailserver for my family. This means that it is serving only 4 accounts.

I use fetchmail->procmail to feed Dovecot and I read my mail on clients such as Evolution on (another) Fedora 20 machine, iPad/iPhone or squirrelmail on the web (running on the same Fedora 20 server).

The average mail throughput for the whole family is around a couple of hundred messages per day, but each (and every) day I get between 3 and 10 of the following messages:

dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s)

I have googled, but with little success. What I have found seemed to suggest that removing the .index files and allowing Dovecot to rebuild them will solve this. I did: find /home -type f -name "dovecot.index.*" -exec rm -f {} \; but it has made no difference.

Can anyone help me solve this please?

at this small setup simply use maildir as format and use getmail to fetch, sieve may do sorting

see https://sys4.de/de/blog/2013/04/12/abholdienst-fur-mail/

sorry german but setup should speak for its own

getmail does also deliver into maildir directly ( without dovecot deliver ) if you dont need to sort through sieve

Well thanks - but I have been using an mbox setup for about 10 years. It is only since I upgraded to F20 that I have been experiencing this problem. I may look at switching to maildir for the future, but for now that's too big a job...

Any other ideas?

Thanks

Mark

On Tue, 2014-05-27 at 16:09 +0200, Robert Schetterer wrote:

Am 27.05.2014 16:04, schrieb Arthur Dent:

...

On Tue, 2014-05-27 at 15:44 +0200, Robert Schetterer wrote:

...

Am 27.05.2014 14:56, schrieb Arthur Dent:

...

Hello All,

I have a Dovecot v2.2.12 installation running on a Fedora 20 box which is the mailserver for my family. This means that it is serving only 4 accounts.

I use fetchmail->procmail to feed Dovecot and I read my mail on clients such as Evolution on (another) Fedora 20 machine, iPad/iPhone or squirrelmail on the web (running on the same Fedora 20 server).

The average mail throughput for the whole family is around a couple of hundred messages per day, but each (and every) day I get between 3 and 10 of the following messages:

dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s)

I have googled, but with little success. What I have found seemed to suggest that removing the .index files and allowing Dovecot to rebuild them will solve this. I did: find /home -type f -name "dovecot.index.*" -exec rm -f {} \; but it has made no difference.

Can anyone help me solve this please?

at this small setup simply use maildir as format and use getmail to fetch, sieve may do sorting

see https://sys4.de/de/blog/2013/04/12/abholdienst-fur-mail/

sorry german but setup should speak for its own

getmail does also deliver into maildir directly ( without dovecot deliver ) if you dont need to sort through sieve

Well thanks - but I have been using an mbox setup for about 10 years. It is only since I upgraded to F20 that I have been experiencing this problem. I may look at switching to maildir for the future, but for now that's too big a job...

Any other ideas?

Thanks

Mark

http://wiki2.dovecot.org/MboxProblems

OK - Unless I'm missing something, all that page (which I'd already read) tells me is that I may have to delete the dovecot.index file(s) - which I've don (several times). I still have the same problem. Every single day.

Anything else?

On Tue, 2014-05-27 at 16:45 +0200, Patrick De Zordo wrote:

You would *really* not change do Maildir format? It's not so difficult to switch over.. and you will have working and simple mailfolders.. (also backup and restore would be a simple task)..

So, why won't you change at all?

Well - the more I read about this, the more I think you're right. Perhaps I should convert to Maildir - but I'm terrified...

I have seen that there is a script - mb2md - that I can use, but what changes do I need to make elsewhere?

I get my mail from a variety of ISP and online email accounts using fetchmail. I guess there's no change needed there?

That feeds into procmail. I think I will have to change the mail procmail configuration file and ALL my procmail recipe rc files (I have many) - which filter into several different mboxs?

Within one of the procmail recipes I call spamassassin and clamscan. Will they be a problem?

Then Dovecot. What changes do i need to make within Dovecot?

What about the clients? Do I need to configure squirrelmail, Evolution or iPad/iPhone email clients?

I archive old mail using archivemail. I think this can cope with maildir format, but it archives to a mbox - which I guess is not a problem except when I want to retrieve something from the archive - but I think I can use Mutt for that?

Is there anything else I should consider?

I have the day off tomorrow. I might have a go - what do you think?

Thanks

Mark

-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von Arthur Dent Gesendet: Dienstag, 27. Mai 2014 19:27 An: dovecot@dovecot.org Betreff: Re: [Dovecot] Corrupted Mail?

On Tue, 2014-05-27 at 16:45 +0200, Patrick De Zordo wrote:

...

You would *really* not change do Maildir format? It's not so difficult to switch over.. and you will have working and simple

mailfolders.. (also backup and restore would be a simple task)..

...

So, why won't you change at all?

Well - the more I read about this, the more I think you're right. Perhaps I should convert to Maildir - but I'm terrified...

I have seen that there is a script - mb2md - that I can use, but what changes do I need to make elsewhere?

I would prefer work with dsync (maybe in 2 steps), it preserves message UIDs.. and that's what you really want.

I get my mail from a variety of ISP and online email accounts using fetchmail. I guess there's no change needed there?

Think it wouldn't change anything, it just gets mail from remote server and gives it to the local one.. Are all the messages available as a copy on the ISP side or are they purged on every fetch and so they are just available on your server locally?

That feeds into procmail. I think I will have to change the mail procmail configuration file and ALL my procmail recipe rc files (I have many) - which filter into several different mboxs?

Personally never use procmail.. working with sieve scripts.

Within one of the procmail recipes I call spamassassin and clamscan. Will they be a problem?

You could use amavis; it "glues" alltogether.. Are you using postfix as MTA?

Then Dovecot. What changes do i need to make within Dovecot?

What about the clients? Do I need to configure squirrelmail, Evolution or iPad/iPhone email clients?

I archive old mail using archivemail. I think this can cope with maildir format, but it archives to a mbox - which I guess is not a problem except when I want to retrieve something from the archive - but I think I can use Mutt for that?

Are you working with virtualized server or bare metal?

Is there anything else I should consider?

I have the day off tomorrow. I might have a go - what do you think?

Thanks

Mark

I am seriously considering the change to Maildir format. I am however more than a little concerned that this could all go horribly wrong. This is what I am planning. Does it look sane?

Checklist

1. Stop the mail queue (essentially stop fetchmail)
2. Stop Dovecot
3. Backup all mbox files
4. Edit all procmail recipes (actually I will probably do this first - working with copies - because this will be time-consuming) See below.
5. Convert mbox files to maildir using either mb2md or dsync (to be decided)
6. Change Dovecot config Actually - As far as I can tell no configuration change is required. Dovecot should just recognise the directories as maildirs. Is this right?
7. Restart Dovecot
8. Restart Fetchmail

Have I missed anything?

Procmail scripts:

I am quite concerned about this. I have a directory structure such as:

Home Inbox <-mbox Work Inbox <-mbox HobbyFile <-mbox Malware <-directory L> Spam <-mbox L> Virus <-mbox MLists <-directory L> Fedora <-mbox L> Dovecot <-mbox L> Spamassassin <-mbox L> ...etc...

My current procmail recipe for one example look like this: :0: * ^List-Id:.*users.lists.fedoraproject.org $DESTDIR/MLists/Fedora

I'm guessing it would need to change to: :0 * ^List-Id:.*users.lists.fedoraproject.org $DESTDIR/.MLists.Fedora/

Is that right??? (Note: remove locking colon ":" and use of "." and trailing "/" in path)

Please advise - I am rather nervous of making a rather big mess here!

Thanks

Mark

Am 27.05.2014 16:28, schrieb Arthur Dent:

On Tue, 2014-05-27 at 16:09 +0200, Robert Schetterer wrote:

...

Am 27.05.2014 16:04, schrieb Arthur Dent:

...

On Tue, 2014-05-27 at 15:44 +0200, Robert Schetterer wrote:

...

Am 27.05.2014 14:56, schrieb Arthur Dent:

...

Hello All,

I have a Dovecot v2.2.12 installation running on a Fedora 20 box which is the mailserver for my family. This means that it is serving only 4 accounts.

I use fetchmail->procmail to feed Dovecot and I read my mail on clients such as Evolution on (another) Fedora 20 machine, iPad/iPhone or squirrelmail on the web (running on the same Fedora 20 server).

The average mail throughput for the whole family is around a couple of hundred messages per day, but each (and every) day I get between 3 and 10 of the following messages:

dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s)

I have googled, but with little success. What I have found seemed to suggest that removing the .index files and allowing Dovecot to rebuild them will solve this. I did: find /home -type f -name "dovecot.index.*" -exec rm -f {} \; but it has made no difference.

Can anyone help me solve this please?

at this small setup simply use maildir as format and use getmail to fetch, sieve may do sorting

see https://sys4.de/de/blog/2013/04/12/abholdienst-fur-mail/

sorry german but setup should speak for its own

getmail does also deliver into maildir directly ( without dovecot deliver ) if you dont need to sort through sieve

Well thanks - but I have been using an mbox setup for about 10 years. It is only since I upgraded to F20 that I have been experiencing this problem. I may look at switching to maildir for the future, but for now that's too big a job...

Any other ideas?

Thanks

Mark

http://wiki2.dovecot.org/MboxProblems

OK - Unless I'm missing something, all that page (which I'd already read) tells me is that I may have to delete the dovecot.index file(s) - which I've don (several times). I still have the same problem. Every single day.

Anything else?

... Avoiding crashes and errors

Since the problems usually have been related to broken headers, you should be able to avoid them by filtering out all the Dovecot's internal metadata headers. This is a good idea to do in any case. If you use Dovecot LDA it does this filtering automatically

do you use dovecot lda ?

Best Regards MfG Robert Schetterer

-- [*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein