Hello dovecot! I administer a dovecot installation with 30k users. I've got 4 dovecot directors as frontend and 10 backends. The mailbox are now in maildir format, with maildir++ quota, on a shared netapp filer. Indexes are local on each backend.
I'm reconfiguring the quota plugin: as a first step, I want to use the clone plugin to keep a copy of the quota on a redis database. Next, I'm going to use the "count" quota backend.
I've configured without (many) issues the quota clone plugin, but now I want to force the recalculation on all the mailboxes, because I've got some (not many, but some) mailboxes that are mostly unused and are not refreshed.
At first, I was going to use "doveadm quota recalc -A", but.... I want each backend to perform the recalculation ONLY for the users he it's managing. I can't perform "doveadm quota recalc -A" on the directors, because the quota plugin is enabled only on the backends.
I can parse the user mapping on the directors and split the calculation, one user a time, on the backends, but I feel I'm choosing a overly complicated path.
So.... which is the right way to do this?
Thanks.
-- *Simone Lazzaris* *Qcom S.p.A. a Socio Unico*
Via Roggia Vignola, 9 | 24047 Treviglio (BG)T +39 0363 47905 | D +39 0363 1970352
simone.lazzaris@qcom.it[1] | www.qcom.it[2]
- LinkedIn[3]* | *Facebook*[4]
[1] mailto:simone.lazzaris@qcom.it [2] https://www.qcom.it [3] https://www.linkedin.com/company/qcom-spa [4] http://www.facebook.com/qcomspa
Ciao Simone,
why you want each backend to recalc quota only for its managed users and not run "doveadm quota recalc -A" only one time from a backend tha recalc quota for all users?
Il 10/03/20 11:18, Simone Lazzaris ha scritto:
Hello dovecot!
I administer a dovecot installation with 30k users. I've got 4 dovecot directors as frontend and 10 backends.
The mailbox are now in maildir format, with maildir++ quota, on a shared netapp filer. Indexes are local on each backend.
I'm reconfiguring the quota plugin: as a first step, I want to use the clone plugin to keep a copy of the quota on a redis database. Next, I'm going to use the "count" quota backend.
I've configured without (many) issues the quota clone plugin, but now I want to force the recalculation on all the mailboxes, because I've got some (not many, but some) mailboxes that are mostly unused and are not refreshed.
At first, I was going to use "doveadm quota recalc -A", but.... I want each backend to perform the recalculation ONLY for the users he it's managing.
I can't perform "doveadm quota recalc -A" on the directors, because the quota plugin is enabled only on the backends.
I can parse the user mapping on the directors and split the calculation, one user a time, on the backends, but I feel I'm choosing a overly complicated path.
So.... which is the right way to do this?
Thanks.
--
Simone LazzarisStaff R&D Qcom S.p.A. a Socio UnicoSocietà soggetta all'attività di direzione e coordinamento di Intred S.p.A.
Via Roggia Vignola, 9 | 24047 Treviglio (BG) T +39 0363 47905 | D +39 0363 1970352 simone.lazzaris@qcom.it mailto:simone.lazzaris@qcom.it| www.qcom.it https://www.qcom.itQcom Official PagesLinkedIn https://www.linkedin.com/company/qcom-spa| Facebook http://www.facebook.com/qcomspa
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
In data mercoledì 18 marzo 2020 23:08:27 CET, Alessio Cecchi ha scritto:
Ciao Simone,
why you want each backend to recalc quota only for its managed users and not run "doveadm quota recalc -A" only one time from a backend tha recalc quota for all users?
Because the indexes are local to the backends.
If I make the recalc from one backend only, that one will have to reindex all the messages, and it will have the indexes for all users, while right now (and if every backend processes only its users) every backend has 1/10 of all the indexes.
And another point: I want to switch to count backend (now I'm using maildir++). With this backend, the quota is memorized on the indexes, but if I make all the recalc on one backend only, the others will not have the correct quota on their indexes.
Right now I've written a python script that iterates over the users, asks to the frontend (via doveadm http) which is the current backend, and then connect (again, doveadm http) to that to perform the recalculation.
A bit slow but working.
-- *Simone Lazzaris* *Qcom S.p.A. a Socio Unico*
Via Roggia Vignola, 9 | 24047 Treviglio (BG)T +39 0363 47905 | D +39 0363 1970352
simone.lazzaris@qcom.it[1] | www.qcom.it[2]
- LinkedIn[3]* | *Facebook*[4]
[1] mailto:simone.lazzaris@qcom.it [2] https://www.qcom.it [3] https://www.linkedin.com/company/qcom-spa [4] http://www.facebook.com/qcomspa
On 19/03/2020 10:06 Simone Lazzaris simone.lazzaris@qcom.it wrote:
In data mercoledì 18 marzo 2020 23:08:27 CET, Alessio Cecchi ha scritto:
Ciao Simone,
why you want each backend to recalc quota only for its managed users and
not run "doveadm quota recalc -A" only one time from a backend tha
recalc quota for all users?
Because the indexes are local to the backends.
If I make the recalc from one backend only, that one will have to reindex all the messages, and it will have the indexes for all users, while right now (and if every backend processes only its users) every backend has 1/10 of all the indexes.
And another point: I want to switch to count backend (now I'm using maildir++). With this backend, the quota is memorized on the indexes, but if I make all the recalc on one backend only, the others will not have the correct quota on their indexes.
Right now I've written a python script that iterates over the users, asks to the frontend (via doveadm http) which is the current backend, and then connect (again, doveadm http) to that to perform the recalculation.
A bit slow but working.
--
Simone Lazzaris
If you are using director, you can issue doveadm quota recalc -A on director, which will then direct it to correct backend.
For this to work, it requires that you setup doveadm server on the backend, and configure doveadm_port and doveadm_password on director. It also either needs iteratable user database on director, or a list of users in a file, which you can provide with option -F.
Aki
In data giovedì 19 marzo 2020 09:10:06 CET, Aki Tuomi ha scritto:
If you are using director, you can issue
doveadm quota recalc -Aon director, which will then direct it to correct backend.For this to work, it requires that you setup doveadm server on the backend, and configure doveadm_port and doveadm_password on director. It also either needs iteratable user database on director, or a list of users in a file, which you can provide with option -F.
Aki
Hi Aki, Nice to know!
The quota plugin has to be configured on the frontends? With which backend? Right now the "quota" subcommands are not recognised.
Thanks-- *Simone Lazzaris* *Qcom S.p.A. a Socio Unico*
Via Roggia Vignola, 9 | 24047 Treviglio (BG)T +39 0363 47905 | D +39 0363 1970352
simone.lazzaris@qcom.it[1] | www.qcom.it[2]
- LinkedIn[3]* | *Facebook*[4]
[1] mailto:simone.lazzaris@qcom.it [2] https://www.qcom.it [3] https://www.linkedin.com/company/qcom-spa [4] http://www.facebook.com/qcomspa
On 19/03/2020 10:13 Simone Lazzaris simone.lazzaris@qcom.it wrote:
In data giovedì 19 marzo 2020 09:10:06 CET, Aki Tuomi ha scritto:
If you are using director, you can issue
doveadm quota recalc -Aondirector, which will then direct it to correct backend.
For this to work, it requires that you setup doveadm server on the backend,
and configure doveadm_port and doveadm_password on director. It also either
needs iteratable user database on director, or a list of users in a file,
which you can provide with option -F.
Aki
Hi Aki,
Nice to know!
The quota plugin has to be configured on the frontends? With which backend?
Right now the "quota" subcommands are not recognised.
Thanks
Simone Lazzaris Staff R&D Qcom S.p.A. a Socio Unico Società soggetta all'attività di direzione e coordinamento di Intred S.p.A.
Via Roggia Vignola, 9 | 24047 Treviglio (BG) T +39 0363 47905 | D +39 0363 1970352simone.lazzaris@qcom.it|www.qcom.it (https://www.qcom.it)
Qcom Official PagesLinkedIn (https://www.linkedin.com/company/qcom-spa)|Facebook (http://www.facebook.com/qcomspa)
It's probably enough to just do:
protocol doveadm { mail_plugins = $mail_plugins quota }
Aki
participants (3)
-
Aki Tuomi
-
Alessio Cecchi
-
Simone Lazzaris