Re: Removing a mailbox from a dovecot cluster
Le mar. 26 févr. 2019 à 22:58, David Myers david.myers.24j74@gmail.com a écrit :
Hello Francis,
I wonder if this is due to how a cluster is configured to function internally.
Tell us more about the cluster, is it one of those ‘fancy pants’ high availability, auto backup heart beat things, or is it more a traditional multi server (master slave style) setup.
Either way you may need to disconnect the servers from one another and delete the offending files / directories either via dove or or via the os (although reading your original email it sounds like you are already attempting this).
If you have a fancy cluster this may actually be more difficult than it sounds and have interesting (unwanted) side effects, also the underlying database (if you are storing emails that way) may have a method to remove data
I assume you are keeping back up copies of all those emails somewhere, just in case you need them in the future.
See this wiki article to better understand what I mean by the ‘fancy pants’ clusters : https://en.m.wikipedia.org/wiki/High-availability_cluster They sound very cool, but I suspect are overkill for a mail server, unless your database is already inside one then it would make sense I guess.
Hello,
I just use the cluster/replication functionality integrated into dovecot, nothing more. There is no database involved. I use LDAP for the authentication. The mails are stored locally on each server and replicated with the replication feature of dovecot.
I followed this wiki article: https://wiki.dovecot.org/Replication
Thanks!
-- Francis
Le jeu. 28 févr. 2019 à 11:18, Francis francisd@gmail.com a écrit :
Le mar. 26 févr. 2019 à 22:58, David Myers david.myers.24j74@gmail.com a écrit :
Hello Francis,
I wonder if this is due to how a cluster is configured to function internally.
Tell us more about the cluster, is it one of those ‘fancy pants’ high availability, auto backup heart beat things, or is it more a traditional multi server (master slave style) setup.
Either way you may need to disconnect the servers from one another and delete the offending files / directories either via dove or or via the os (although reading your original email it sounds like you are already attempting this).
If you have a fancy cluster this may actually be more difficult than it sounds and have interesting (unwanted) side effects, also the underlying database (if you are storing emails that way) may have a method to remove data
I assume you are keeping back up copies of all those emails somewhere, just in case you need them in the future.
See this wiki article to better understand what I mean by the ‘fancy pants’ clusters : https://en.m.wikipedia.org/wiki/High-availability_cluster They sound very cool, but I suspect are overkill for a mail server, unless your database is already inside one then it would make sense I guess.
Hello,
I just use the cluster/replication functionality integrated into dovecot, nothing more. There is no database involved. I use LDAP for the authentication. The mails are stored locally on each server and replicated with the replication feature of dovecot.
I followed this wiki article: https://wiki.dovecot.org/Replication
Hello,
So nobody use the replication feature? or nobody never ever remove a mailbox? or maybe my question is so dumb and I should RTFM something? :)
Do you need more information to help me to debug that issue?
Thanks.
-- Francis
Am 04.03.2019 um 16:35 schrieb Francis via dovecot dovecot@dovecot.org:
Le jeu. 28 févr. 2019 à 11:18, Francis
mailto:francisd@gmail.com> a écrit : Le mar. 26 févr. 2019 à 22:58, David Myers mailto:david.myers.24j74@gmail.com> a écrit : Hello Francis, I wonder if this is due to how a cluster is configured to function internally.
Tell us more about the cluster, is it one of those ‘fancy pants’ high availability, auto backup heart beat things, or is it more a traditional multi server (master slave style) setup.
Either way you may need to disconnect the servers from one another and delete the offending files / directories either via dove or or via the os (although reading your original email it sounds like you are already attempting this).
If you have a fancy cluster this may actually be more difficult than it sounds and have interesting (unwanted) side effects, also the underlying database (if you are storing emails that way) may have a method to remove data
I assume you are keeping back up copies of all those emails somewhere, just in case you need them in the future.
See this wiki article to better understand what I mean by the ‘fancy pants’ clusters : https://en.m.wikipedia.org/wiki/High-availability_cluster https://en.m.wikipedia.org/wiki/High-availability_clusterThey sound very cool, but I suspect are overkill for a mail server, unless your database is already inside one then it would make sense I guess.
Hello,
I just use the cluster/replication functionality integrated into dovecot, nothing more. There is no database involved. I use LDAP for the authentication. The mails are stored locally on each server and replicated with the replication feature of dovecot.
I followed this wiki article: https://wiki.dovecot.org/Replication https://wiki.dovecot.org/Replication
Hello,
So nobody use the replication feature? or nobody never ever remove a mailbox? or maybe my question is so dumb and I should RTFM something? :)
Do you need more information to help me to debug that issue?
Thanks.
Hallo Francis,
have you tried removing the account from your ldap? If dovecot has no information about a particular user, it won't replicate.
Then you would have to delete the mailbox (on both cluster nodes) from the filesystem (rm -rf /path/to/mailbox) During testing you could move the mailbox somewhere else instead of deleting it, just in case something does not work as expected.
Deleting files on another server could be automated with ssh (ssh keys).
Best regards, Gerald
Le lun. 4 mars 2019 à 12:48, Gerald Galster via dovecot dovecot@dovecot.org a écrit :
Hallo Francis,
have you tried removing the account from your ldap? If dovecot has no information about a particular user, it won't replicate.
Then you would have to delete the mailbox (on both cluster nodes) from the filesystem (rm -rf /path/to/mailbox) During testing you could move the mailbox somewhere else instead of deleting it, just in case something does not work as expected.
Deleting files on another server could be automated with ssh (ssh keys).
Hello,
I'm also using single instance storage for attachments. Because of that, I think I can't just remove the mdbox storage with rm because I'll be stuck with attachments from removed mailboxes. Am I wrong?
This is why I first use doveadm flags/expunge to mark as removed all messages then I use doveadm purge to remove them from storage. I can't use theses commands on deleted/disabled user, I get an error saying the user cannot be found, so I can't remove them from LDAP first.
-- Francis
Am 04.03.2019 um 22:19 schrieb Francis via dovecot dovecot@dovecot.org:
Le lun. 4 mars 2019 à 12:48, Gerald Galster via dovecot
mailto:dovecot@dovecot.org> a écrit : Hallo Francis,
have you tried removing the account from your ldap? If dovecot has no information about a particular user, it won't replicate.
Then you would have to delete the mailbox (on both cluster nodes) from the filesystem (rm -rf /path/to/mailbox) During testing you could move the mailbox somewhere else instead of deleting it, just in case something does not work as expected.
Deleting files on another server could be automated with ssh (ssh keys).
I'm also using single instance storage for attachments. Because of that, I think I can't just remove the mdbox storage with rm because I'll be stuck with attachments from removed mailboxes. Am I wrong?
you may be right. I don't know if any tools like doveadm deduplicate will check for orphans. On a new server you could disable sis and use a filesystem with deduplication instead (e.g. vdo)
This is why I first use doveadm flags/expunge to mark as removed all messages then I use doveadm purge to remove them from storage. I can't use theses commands on deleted/disabled user, I get an error saying the user cannot be found, so I can't remove them from LDAP first.
you could try to stop replication for the user you are deleting: doveadm replicator remove [-a replicator_socket_path] username
Best regards Gerald
Le mar. 5 mars 2019 à 10:08, Gerald Galster via dovecot dovecot@dovecot.org a écrit :
you could try to stop replication for the user you are deleting: doveadm replicator remove [-a replicator_socket_path] username
Good idea! But I have a problem. I tried to stop the replication (doveadm replicator remove <myuser>) for an user on both server. Verified with "doveadm replicator status <myuser>", it doesn't return, so I assume the replicator is off for this account. Then I try to send an email to this account to verify the replication is really stopped, but it activate again by itself? I receive the mail on both server and If I type "doveadm replicator status <myuser>", it seem like replication is back on.
-- Francis
Am 07.03.2019 um 20:43 schrieb Francis francisd@gmail.com:
Le mar. 5 mars 2019 à 10:08, Gerald Galster via dovecot
mailto:dovecot@dovecot.org> a écrit : you could try to stop replication for the user you are deleting: doveadm replicator remove [-a replicator_socket_path] username
Good idea! But I have a problem. I tried to stop the replication (doveadm replicator remove <myuser>) for an user on both server. Verified with "doveadm replicator status <myuser>", it doesn't return, so I assume the replicator is off for this account. Then I try to send an email to this account to verify the replication is really stopped, but it activate again by itself? I receive the mail on both server and If I type "doveadm replicator status <myuser>", it seem like replication is back on.
Why does "doveadm replicator status <myuser>" not return?
I've tried the following commands on a replicated server (2.2.33.2), which all returned immediately:
[root@server ~]# doveadm replicator status user@domain.com username priority fast sync full sync success sync failed user@domain.com none 12:30:32 12:30:32 12:30:31 -
[root@server ~]# doveadm replicator remove user@domain.com
[root@server ~]# doveadm replicator status user@domain.com username priority fast sync full sync success sync failed (no additional output as replication is stopped)
[root@server ~]# doveadm replicator add user@domain.com
[root@server ~]# doveadm replicator status user@domain.com
username priority fast sync full sync success sync failed
user@domain.com none 00:00:02 00:00:02 00:00:01 -
(replication working again)
Maybe replication had not been stopped yet on your server.
Best regards Gerald
Le jeu. 7 mars 2019 à 17:03, Gerald Galster via dovecot dovecot@dovecot.org a écrit :
Why does "doveadm replicator status <myuser>" not return?
I've tried the following commands on a replicated server (2.2.33.2), which all returned immediately:
[root@server ~]# doveadm replicator status user@domain.com username priority fast sync full sync success sync failed user@domain.com none 12:30:32 12:30:32 12:30:31 -
[root@server ~]# doveadm replicator remove user@domain.com
[root@server ~]# doveadm replicator status user@domain.com username priority fast sync full sync success sync failed (no additional output as replication is stopped)
Hi,
Sorry, I didn't express myself correctly. The output you see is the one I see too when the replication is off. When I say it doesn't return anything, I wanted to say it doesn't print anything else other than the header line. The replication has been disabled for more than 12 hours and I just sent an email to this account and the replication switched on by itself again. I see nothing about the replication in the logs.
-- Francis
Am 08.03.2019 um 14:46 schrieb Francis francisd@gmail.com:
Le jeu. 7 mars 2019 à 17:03, Gerald Galster via dovecot
mailto:dovecot@dovecot.org> a écrit : Why does "doveadm replicator status <myuser>" not return?
I've tried the following commands on a replicated server (2.2.33.2), which all returned immediately:
[root@server ~]# doveadm replicator status user@domain.com mailto:user@domain.com username priority fast sync full sync success sync failed user@domain.com mailto:user@domain.com none 12:30:32 12:30:32 12:30:31 -
[root@server ~]# doveadm replicator remove user@domain.com mailto:user@domain.com
[root@server ~]# doveadm replicator status user@domain.com mailto:user@domain.com username priority fast sync full sync success sync failed (no additional output as replication is stopped)
Hi,
Sorry, I didn't express myself correctly. The output you see is the one I see too when the replication is off. When I say it doesn't return anything, I wanted to say it doesn't print anything else other than the header line. The replication has been disabled for more than 12 hours and I just sent an email to this account and the replication switched on by itself again. I see nothing about the replication in the logs.
Maybe someone from the dovecot team can tell more about the internals on why and when replication is activated again.
I'm solving that problem at the mta level: the email is inserted into postfix' access table where access is denied, so emails won't reach dovecot. Maybe you can do something similar or set a flag in ldap that your mta can use to reject mails.
Best regards Gerald
participants (2)
-
Francis
-
Gerald Galster