[Dovecot] General questions about TCP replication with dsync
Hi Guys,
I would like to question why I see this errors in my logs.
Let explain me!
Dovecot version: 2.2.9 (on Ubuntu 14.04 LTS)
Ive configured our dovecot servers (2 servers) to do the TCP replication with dsync.
This is working flawless.
- Question:
In logs I can see this errors on every sync (on the slave server / server without changes which should be updated):
---8<-----------------------------------------------------------------------
dsync-server(email@example.com): Error: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
---8<-----------------------------------------------------------------------
Googling a bit gave me the information This should be normal..?
Well ok, errors in logs are not nice to see, but can someone confirm this is really ok? Timo?
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<-----------------------------------------------------------------------
dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<-----------------------------------------------------------------------
So whats going on there?
This is strange and should not be, right?
Any hints?
Thank you very much!
Mit freundlichen Grüßen / Distinti saluti / Kind regards
De Zordo Patrick
patrick@spamreducer.eu
Am 04.06.2014 19:35, schrieb Patrick De Zordo:
Hi Guys,
I would like to question why I see this errors in my logs.
Let explain me!
Dovecot version: 2.2.9 (on Ubuntu 14.04 LTS)
I’ve configured our dovecot servers (2 servers) to do the TCP replication with dsync.
This is working flawless.
- Question:
In logs I can see this errors on every sync (on the “slave” server / server without changes which should be updated):
---8<-----------------------------------------------------------------------
dsync-server(email@example.com): Error: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
---8<-----------------------------------------------------------------------
Googling a bit gave me the information “This should be normal”..?
Well ok, errors in logs are not nice to see, but can someone confirm this is really ok? Timo?
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<-----------------------------------------------------------------------
dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<-----------------------------------------------------------------------
So what’s going on there?
This is strange and should not be, right?
Any hints?
Thank you very much!
Mit freundlichen Grüßen / Distinti saluti / Kind regards
De Zordo Patrick
patrick@spamreducer.eu
2.2.9 is not recent, before debug you might upgrade
see http://hg.dovecot.org/dovecot-2.2/log?rev=replicator
also you missed to attach your dove conf
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von Robert Schetterer Gesendet: Mittwoch, 4. Juni 2014 19:42 An: dovecot@dovecot.org Betreff: Re: [Dovecot] General questions about TCP replication with dsync
Am 04.06.2014 19:35, schrieb Patrick De Zordo:
Hi Guys,
I would like to question why I see this errors in my logs.
Let explain me!
Dovecot version: 2.2.9 (on Ubuntu 14.04 LTS)
I’ve configured our dovecot servers (2 servers) to do the TCP replication with dsync.
This is working flawless.
- Question:
In logs I can see this errors on every sync (on the “slave” server / server without changes which should be updated):
---8<-----------------------------------------------------------------------
dsync-server(email@example.com): Error: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
---8<-----------------------------------------------------------------------
Googling a bit gave me the information “This should be normal”..?
Well ok, errors in logs are not nice to see, but can someone confirm this is really ok? Timo?
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<-----------------------------------------------------------------------
dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<-----------------------------------------------------------------------
So what’s going on there?
This is strange and should not be, right?
Any hints?
Thank you very much!
Mit freundlichen Grüßen / Distinti saluti / Kind regards
De Zordo Patrick
patrick@spamreducer.eu
2.2.9 is not recent, before debug you might upgrade
Well, not so easy.. we are working on a productive server; this version ships as default for this distro.. I don't even know how to compile my own dovecot version..
see http://hg.dovecot.org/dovecot-2.2/log?rev=replicator
also you missed to attach your dove conf
Relevant config parts: --- /etc/dovecot/conf.d/10-mail.conf --- ---8<----------------------------------------------------------------------- mail_plugins = $mail_plugins notify replication ---8<-----------------------------------------------------------------------
--- /etc/dovecot/conf.d/99-replication.conf --- ---8<----------------------------------------------------------------------- service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service replicator { unix_listener replicator-doveadm { mode = 0600 } } service doveadm { inet_listener { port = 999 } } doveadm_password = secret replication_max_conns = 1 plugin { mail_replica = tcp:server2.example.com:999 } ---8<-----------------------------------------------------------------------
Thank you for any hint!
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Am 04.06.2014 19:53, schrieb Patrick De Zordo:
Well, not so easy.. we are working on a productive server; this version ships as default for this distro.. I don't even know how to compile my own dovecot version..
see
http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages
or
https://launchpad.net/~mamarley/+archive/updates/+packages
recompile
https://sys4.de/de/blog/2013/06/17/dovecot-patching-mit-debian-und-ubuntu/
sorry german
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
How does this affect the other packages and the dependencies ? For example, my test system is an Ubuntu 14.04/trusty box ...
$ dpkg -l dove* | fgrep ii ii dovecot-antispam 2.0+20130822-2build1 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.9-1ubuntu2.1 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.9-1ubuntu2.1 secure POP3/IMAP server - IMAP daemon ii dovecot-lmtpd 1:2.2.9-1ubuntu2.1 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.9-1ubuntu2.1 secure POP3/IMAP server - ManageSieve server ii dovecot-sieve 1:2.2.9-1ubuntu2.1 secure POP3/IMAP server - Sieve filters support ii dovecot-solr 1:2.2.9-1ubuntu2.1 secure POP3/IMAP server - Solr support
So rather than the stock
_sudo apt-get install dovecot-imapd dovecot-sieve dovecot-antispam dovecot-managesieved dovecot-lmtpd dovecot-solr_
one would install _dovecot_ from the MAMARLEY PPA ? To integrate cleanly it would have to "Provides: dovecot-common" and "Replaces: dovecot-common, mailavenger" like the stock dovecot-core does. While we can and do use non-stock packages, we really try to stay with stock(ish) packages as much as possible to ease upgrade administration. I'm dealing with a slew of custom apache2 installs, all unique, on a bunch of old Cent 5.2 boxes right now, and it's a mighty pain.
D.
On 2014-06-04 16:41, Robert Schetterer wrote:
Am 04.06.2014 19:53, schrieb Patrick De Zordo:
Well, not so easy.. we are working on a productive server; this version ships as default for this distro.. I don't even know how to compile my own dovecot version..
see
http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages
or
recompile
https://sys4.de/de/blog/2013/06/17/dovecot-patching-mit-debian-und-ubuntu/
Am 05.06.2014 02:50, schrieb deano-dovecot@areyes.com:
we really try to stay with stock(ish) packages as much as possible to ease upgrade administration
I understand, but Timo does heavy debug, so its up to the ubuntu mantainers to keep dove version up2date.
I never used the Ubuntu Distro packages.
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 2014-06-04 19:53:19(+0200), Patrick De Zordo wrote:
-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von Robert Schetterer Gesendet: Mittwoch, 4. Juni 2014 19:42 An: dovecot@dovecot.org Betreff: Re: [Dovecot] General questions about TCP replication with dsync
Am 04.06.2014 19:35, schrieb Patrick De Zordo:
Dovecot version: 2.2.9 (on Ubuntu 14.04 LTS)
- Question:
In logs I can see this errors on every sync (on the “slave” server / server without changes which should be updated):
---8<----------------------------------------------------------------------
dsync-server(email@example.com): Error: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
---8<----------------------------------------------------------------------
Googling a bit gave me the information “This should be normal”..?
Well ok, errors in logs are not nice to see, but can someone confirm this is really ok? Timo?
I have a similar setup and have been getting the same errors about replicator-doveadm permissions on 2.2.12. I found this message in the list archives which say's its not important
http://www.dovecot.org/list/dovecot/2013-April/089559.html
and suggests changing the permissions on the socket. The right permissions would depend on which user(s) the doveadm-server process doing the sync is running as. From your questions below it looks like you are using virtual users with the vmail user owning the mailboxes, try
service replicator { unix_listener replicator-doveadm { mode = 0660 group = vmail } }
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<---------------------------------------------------------------------- dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<----------------------------------------------------------------------
So what’s going on there?
This is strange and should not be, right?
Any hints?
It looks like it thinks the .dovecot.sieve file should be a Maildir. Are you using the Maildir++ format for your mail stores? That uses "." as a separator for mailbox names. You might want to try changing the "sieve" config option from ".dovecot.sieve" to something which doesn't contain dots and rename the files to match.
James
-- }-<James TD Smith>-email/ahktenzero@mohorovi.cc-{
-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von James TD Smith Gesendet: Donnerstag, 5. Juni 2014 17:34 An: dovecot@dovecot.org Betreff: Re: [Dovecot] General questions about TCP replication with dsync
On 2014-06-04 19:53:19(+0200), Patrick De Zordo wrote:
-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von Robert Schetterer Gesendet: Mittwoch, 4. Juni 2014 19:42 An: dovecot@dovecot.org Betreff: Re: [Dovecot] General questions about TCP replication with dsync
Am 04.06.2014 19:35, schrieb Patrick De Zordo:
Dovecot version: 2.2.9 (on Ubuntu 14.04 LTS)
- Question:
In logs I can see this errors on every sync (on the “slave” server / server without changes which should be updated):
---8<-------------------------------------------------------------
dsync-server(email@example.com): Error: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
---8<-------------------------------------------------------------
Googling a bit gave me the information “This should be normal”..?
Well ok, errors in logs are not nice to see, but can someone confirm this is really ok? Timo?
I have a similar setup and have been getting the same errors about replicator-doveadm permissions on 2.2.12. I found this message in the list archives which say's its not important
http://www.dovecot.org/list/dovecot/2013-April/089559.html
and suggests changing the permissions on the socket. The right permissions would depend on which user(s) the doveadm-server process doing the sync is running as. From your questions below it looks like you are using virtual users with the vmail user owning the mailboxes, try
service replicator { unix_listener replicator-doveadm { mode = 0660 group = vmail } }
OK, thanks! So I'm going on to read this in my logs.. ;-)
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<-------------------------------------------------------------
dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<-------------------------------------------------------------
So what’s going on there?
This is strange and should not be, right?
Any hints?
It looks like it thinks the .dovecot.sieve file should be a Maildir. Are you using the Maildir++ format for your mail stores? That uses "." as a separator for mailbox names. You might want to try changing the "sieve" config option from ".dovecot.sieve" to something which doesn't contain dots and rename the files to match.
I'm using "/" as separator ("separator = /" in configuration).. SO it shouldn't be the problem..
Is your sync up and running? With sieve scripts?
James
-- }-<James TD Smith>-email/ahktenzero@mohorovi.cc-{
On 2014-06-05 17:39:26(+0200), Patrick De Zordo wrote:
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<-------------------------------------------------------------
dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<-------------------------------------------------------------
So what’s going on there?
This is strange and should not be, right?
Any hints?
It looks like it thinks the .dovecot.sieve file should be a Maildir. Are you using the Maildir++ format for your mail stores? That uses "." as a separator for mailbox names. You might want to try changing the "sieve" config option from ".dovecot.sieve" to something which doesn't contain dots and rename the files to match.
I'm using "/" as separator ("separator = /" in configuration).. SO it shouldn't be the problem..
That parameter controls how the mailbox names are presented over IMAP, not how they are stored on disk. If you run "doveadm mailbox list -u email@example.com" you'll get a list like
INBOX Sent Drafts MailingLists/dovecot
etc, but if you look in /var/vmail/example.com/email you'll see something like
cur/ new/ tmp/ .Sent/ .Drafts/ .MailingList.dovecot/
If you add LAYOUT=fs to the mail_location option it will store the mailboxes as a directory hierarchy. You'll need to convert all the mailboxes to the new layout, changing the sieve file name will be much less work.
http://wiki2.dovecot.org/MailLocation/Maildir
Is your sync up and running? With sieve scripts?
Yes, my sync works. I don't use sieve though.
James
-- }-<James TD Smith>-email/ahktenzero@mohorovi.cc-{
-----Ursprüngliche Nachricht----- Von: dovecot [mailto:dovecot-bounces@dovecot.org] Im Auftrag von James TD Smith Gesendet: Donnerstag, 5. Juni 2014 18:29 An: dovecot@dovecot.org Betreff: Re: [Dovecot] General questions about TCP replication with dsync
On 2014-06-05 17:39:26(+0200), Patrick De Zordo wrote:
- Question (much more important):
We are using sieve scripts for our users, they are replicated to the replica-host but this errors appears in logs on every synchronized message:
---8<---------------------------------------------------------
dsync-server(email@example.com): Error: Failed to access mailbox dovecot/sieve: Internal error occurred. Refer to server log for more information.
dsync-server(email@example.com): Error: stat(/var/vmail/example.com/email/.dovecot.sieve/tmp) failed: Not a directory
---8<---------------------------------------------------------
So what’s going on there?
This is strange and should not be, right?
Any hints?
It looks like it thinks the .dovecot.sieve file should be a Maildir. Are you using the Maildir++ format for your mail stores? That uses "." as a separator for mailbox names. You might want to try changing the "sieve" config option from ".dovecot.sieve" to something which doesn't contain dots and rename the files to match.
I'm using "/" as separator ("separator = /" in configuration).. SO it shouldn't be the problem..
That parameter controls how the mailbox names are presented over IMAP, not how they are stored on disk. If you run "doveadm mailbox list -u email@example.com" you'll get a list like
INBOX Sent Drafts MailingLists/dovecot
etc, but if you look in /var/vmail/example.com/email you'll see something like
cur/ new/ tmp/ .Sent/ .Drafts/ .MailingList.dovecot/
If you add LAYOUT=fs to the mail_location option it will store the mailboxes as a directory hierarchy. You'll need to convert all the mailboxes to the new layout, changing the sieve file name will be much less work.
Absolutely TRUE! In http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration there is this little very small paragraph: " . Note: It is not wise to place this link inside your mail store, as it may be mistaken for a mail folder. Inside a maildir for instance, the default .dovecot.sieve would show up as phantom folder /dovecot/sieve in your IMAP tree."
But in the example configuration they use "sieve = ~/.dovecot.sieve", well probably no one is using Maildir? ;-)
DAMN! A day full of researches because of this little error..
Well thank you very much for pointing me on the right way!
http://wiki2.dovecot.org/MailLocation/Maildir
Is your sync up and running? With sieve scripts?
Yes, my sync works. I don't use sieve though.
James
-- }-<James TD Smith>-email/ahktenzero@mohorovi.cc-{
participants (4)
-
deano-dovecot@areyes.com
-
James TD Smith
-
Patrick De Zordo
-
Robert Schetterer