Re: [Dovecot] Corrupted Mail?
Arthur Dent <arthurdent.london@gmail.com> writes:
I get my mail from a variety of ISP and online email accounts using fetchmail. I guess there's no change needed there?
That feeds into procmail. I think I will have to change the mail procmail configuration file and ALL my procmail recipe rc files (I have many) - which filter into several different mboxs?
As others have intimated, the use of procmail as your LDA is causing your mailbox to become inconsistent with respect to your indices. Procmail is appending your messages to mailboxes, but is ignorant of the existence of dovecot indices, and does not update them. When your IMAP service access those mailboxes, it detects this condition, and that's the log messages you are seeing.
I think those messages are benign -- you can probably ignore it. Dovecot will automatically rebuild them.
The alternative is to let dovecot's be responsible for message delivery: it can then update the indices, and IMAP/POP will find consistent indices.
Try this at the end of procmailrc for regular delivery to the INBOX
################################################################################
# Replace default action using dovecot-lda to maintains caches.
################################################################################
:0 w
| /location/of/dovecot-lda -d $USERFor for non-INBOX filtering, try something like
:0 Hw
*^Subject: TEST
|/location/of/dovecot-lda -d $USER -m testboxJoseph Tam <jtam.home@gmail.com>
On Tue, 2014-05-27 at 16:31 -0700, Joseph Tam wrote:
Arthur Dent <arthurdent.london@gmail.com> writes:
I get my mail from a variety of ISP and online email accounts using fetchmail. I guess there's no change needed there?
That feeds into procmail. I think I will have to change the mail procmail configuration file and ALL my procmail recipe rc files (I have many) - which filter into several different mboxs?
As others have intimated, the use of procmail as your LDA is causing your mailbox to become inconsistent with respect to your indices. Procmail is appending your messages to mailboxes, but is ignorant of the existence of dovecot indices, and does not update them. When your IMAP service access those mailboxes, it detects this condition, and that's the log messages you are seeing.
I think those messages are benign -- you can probably ignore it. Dovecot will automatically rebuild them.
The alternative is to let dovecot's be responsible for message delivery: it can then update the indices, and IMAP/POP will find consistent indices.
Try this at the end of procmailrc for regular delivery to the INBOX
################################################################################ # Replace default action using dovecot-lda to maintains caches. ################################################################################ :0 w | /location/of/dovecot-lda -d $USER
For for non-INBOX filtering, try something like
:0 Hw *^Subject: TEST |/location/of/dovecot-lda -d $USER -m testbox
Thanks Joseph - this is really useful.
By coincidence (whilst researching all of this) last night I cam across this page: http://wiki2.dovecot.org/procmail which says exactly what you suggested. I tried it as an experiment on just one of my mailing list recipes. It didn't work and dropped through to my default inbox. This is what I have:
:0 w
- ^List-Id:.*users.lists.fedoraproject.org | $DELIVER -m $DESTDIR/MLists/Fedora
where: DELIVER=/usr/libexec/dovecot/deliver DESTDIR=/home/mark/mail/ are defined in a the main procmailrc
This is what I found in the procmail log: lda: Fatal: destination user parameter (-d user) not given procmail: Program failure (64) of "/usr/libexec/dovecot/deliver" procmail: Couldn't determine implicit lockfile from "spamc"
From users-bounces@lists.fedoraproject.org Wed May 28 01:39:03 2014 Subject: Five Things in Fedora This Week (2014-05-27)
Why did this mail not get delivered to /home/mark/mail/MLists/Fedora?
Thanks again
Mark
On May 28, 2014, at 2:03 AM, Arthur Dent <arthurdent.london@gmail.com> wrote:
Try this at the end of procmailrc for regular delivery to the INBOX
################################################################################ # Replace default action using dovecot-lda to maintains caches. ################################################################################ :0 w | /location/of/dovecot-lda -d $USER
For for non-INBOX filtering, try something like
:0 Hw *^Subject: TEST |/location/of/dovecot-lda -d $USER -m testbox
Thanks Joseph - this is really useful.
By coincidence (whilst researching all of this) last night I cam across this page: http://wiki2.dovecot.org/procmail which says exactly what you suggested. I tried it as an experiment on just one of my mailing list recipes. It didn't work and dropped through to my default inbox. This is what I have:
:0 w
- ^List-Id:.*users.lists.fedoraproject.org | $DELIVER -m $DESTDIR/MLists/Fedora
But that’s not exactly what was suggested above. It’s missing the “-d $USER”.
where: DELIVER=/usr/libexec/dovecot/deliver DESTDIR=/home/mark/mail/ are defined in a the main procmailrc
This is what I found in the procmail log: lda: Fatal: destination user parameter (-d user) not given procmail: Program failure (64) of "/usr/libexec/dovecot/deliver" procmail: Couldn't determine implicit lockfile from "spamc" From users-bounces@lists.fedoraproject.org Wed May 28 01:39:03 2014 Subject: Five Things in Fedora This Week (2014-05-27)
Why did this mail not get delivered to /home/mark/mail/MLists/Fedora?
As the error above says, it’s missing the “-d $USER” from the example above.
-- Larry Stone lstone19@stonejongleux.com http://www.stonejongleux.com/
Larry Stone <lstone19@stonejongleux.com> wrote:
On May 28, 2014, at 2:03 AM, Arthur Dent <arthurdent.london@gmail.com> wrote:
DELIVER=/usr/libexec/dovecot/deliver DESTDIR=/home/mark/mail/ :0 w
- ^List-Id:.*users.lists.fedoraproject.org | $DELIVER -m $DESTDIR/MLists/Fedora
But that’s not exactly what was suggested above. It’s missing the “-d $USER”.
And I guess the mailbox name at -m is wrong.
This gets expanded to:
| /usr/libexec/dovecot/deliver -d mark -m /home/mark/mail/MLists/Fedora
But with "-m" you have to use the mailbox name dovecot uses, which could be for example
"INBOX.MLists.Fedora" or "INBOX/MLists/Fedora" or "MLists.Fedora" or even "MLists/Fedora" depending on your namespace configuration.
I suggest to use "doveadm mailbox list" to find out what the dovecot name of the target mailbox is.
Grüße, Sven.
-- Sigmentation fault. Core dumped.
On Wed, 2014-05-28 at 14:27 +0200, Sven Hartge wrote:
Larry Stone <lstone19@stonejongleux.com> wrote:
On May 28, 2014, at 2:03 AM, Arthur Dent <arthurdent.london@gmail.com> wrote:
DELIVER=/usr/libexec/dovecot/deliver DESTDIR=/home/mark/mail/ :0 w
- ^List-Id:.*users.lists.fedoraproject.org | $DELIVER -m $DESTDIR/MLists/Fedora
But that’s not exactly what was suggested above. It’s missing the “-d $USER”.
And I guess the mailbox name at -m is wrong.
This gets expanded to:
| /usr/libexec/dovecot/deliver -d mark -m /home/mark/mail/MLists/Fedora
But with "-m" you have to use the mailbox name dovecot uses, which could be for example
"INBOX.MLists.Fedora" or "INBOX/MLists/Fedora" or "MLists.Fedora" or even "MLists/Fedora" depending on your namespace configuration.
I suggest to use "doveadm mailbox list" to find out what the dovecot name of the target mailbox is.
Thanks to everyone who helped me sort this out. This was the most helpful reply - I had been trying both with and without the -d switch without success. Using "doveadm mailbox list" showed me that the problem was with the delivery address.
This is what I now have - and it works - Thanks to all: :0 w
- ^List-Id:.*users.lists.fedoraproject.org | /usr/libexec/dovecot/deliver -d mark -m MLists/Fedora
Another two questions now please if I may:
I have tried both /usr/libexec/dovecot/deliver and /usr/libexec/dovecot/dovecot-lda and they both work. Are they equivalent? If not which should I use?
Switching to maildir, should the new delivery line now read: | /usr/libexec/dovecot/deliver -d mark -m .MLists.Fedora/ (i.e. a dot before and after MLists and a trailing slash)?
Thanks again for all the help and support. Much appreciated...
Mark
Arthur Dent <arthurdent.london@gmail.com> wrote:
- Switching to maildir, should the new delivery line now read: | /usr/libexec/dovecot/deliver -d mark -m .MLists.Fedora/ (i.e. a dot before and after MLists and a trailing slash)?
That depends on the namespace configuration in dovecot.
An excerpt from the output of "doveadm mailbox list" on my server looks like this:
[...] Admin.DVZ Admin.DVZ.server Admin.DVZ.server.events Admin.DVZ.server.events.nagios Admin.DVZ.server.events.nagios.FM Admin.DVZ.server.events.nagios.dkm Admin.DVZ.server.events.nagios.verw Admin.DVZ.server.events.nagios.net Admin.DVZ.server.events.nagios.sonstiges [...]
And my namespace configuration:
namespace inbox { hidden = no inbox = yes list = yes location = maildir:~/Maildir prefix = separator = . subscriptions = yes type = private }
Depending on your "prefix" and "separator" settings the names of the mailboxes change.
Again: use "doveadm mailbox list" to find out, what the names should be.
Grüße, S°
-- Sigmentation fault. Core dumped.
On May 29, 2014, at 6:35 AM, Arthur Dent <arthurdent.london@gmail.com> wrote:
- I have tried both /usr/libexec/dovecot/deliver and /usr/libexec/dovecot/dovecot-lda and they both work. Are they equivalent? If not which should I use?
A ls -l /usr/libexec/dovecot will (or at least it does on my system) show you that deliver is a link to dovecot-lda. lrwxr-xr-x 1 root wheel 11 May 15 22:16 deliver -> dovecot-lda
-- Larry Stone lstone19@stonejongleux.com http://www.stonejongleux.com/
On Thu, 2014-05-29 at 06:53 -0500, Larry Stone wrote:
On May 29, 2014, at 6:35 AM, Arthur Dent <arthurdent.london@gmail.com> wrote:
- I have tried both /usr/libexec/dovecot/deliver and /usr/libexec/dovecot/dovecot-lda and they both work. Are they equivalent? If not which should I use?
A ls -l /usr/libexec/dovecot will (or at least it does on my system) show you that deliver is a link to dovecot-lda. lrwxr-xr-x 1 root wheel 11 May 15 22:16 deliver -> dovecot-lda
Oh yes. Mine too. Thanks for clearing that up!
Well, with thanks to everyone on this list I have now successfully (I hope) switched from mbox to Maildir format. Moreover I am now using dovecot-lda to deliver.
I went step by step, and tested it with just one account before opening up the whole system.
I had to rewrite my procmail recipes twice because I first changed them to: | /usr/libexec/dovecot/deliver -d mark -m .MLists.Fedora/
but after testing with the Maildir format up and running I found that they had to be: | /usr/libexec/dovecot/deliver -d mark -m MLists.Fedora
I will now leave it running for a couple of days. I will keep an eye on the logs, but I hope that I have now seen the last of the "Error: Next message unexpectedly corrupted ..." messages!
Thanks again for all the help. Much appreciated!
Mark
-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von Arthur Dent Gesendet: Freitag, 30. Mai 2014 11:25 An: dovecot@dovecot.org Betreff: Re: [Dovecot] Change to Maildir format (Was:Corrupted Mail?)
Well, with thanks to everyone on this list I have now successfully (I hope) switched from mbox to Maildir format. Moreover I am now using dovecot-lda to deliver.
I went step by step, and tested it with just one account before opening up the whole system.
Great!
I had to rewrite my procmail recipes twice because I first changed them to: | /usr/libexec/dovecot/deliver -d mark -m .MLists.Fedora/
but after testing with the Maildir format up and running I found that they had to be: | /usr/libexec/dovecot/deliver -d mark -m MLists.Fedora
I will now leave it running for a couple of days. I will keep an eye on the logs, but I hope that I have now seen the last of the "Error: Next message unexpectedly corrupted ..." messages!
Sure!
Thanks again for all the help. Much appreciated!
Mark
participants (5)
-
Arthur Dent
-
Joseph Tam
-
Larry Stone
-
Patrick De Zordo
-
Sven Hartge