doveadm backup|sync works for every folder but INBOX
Aki Tuomi
aki.tuomi at open-xchange.com
Thu Nov 3 13:29:02 UTC 2022
> On 03/11/2022 13:43 EET Ralf Becker <rb at egroupware.org> wrote:
>
>
> Hi Aki,
>
> Am 03.11.22 um 12:27 schrieb Aki Tuomi:
> >> On 03/11/2022 13:23 EET Aki Tuomi <aki.tuomi at open-xchange.com> wrote:
> >>> On 03/11/2022 13:19 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>
> >>>
> >>> Hi Aki,
> >>>
> >>> Am 03.11.22 um 10:54 schrieb Aki Tuomi:
> >>>>> On 03/11/2022 11:46 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>>
> >>>>>
> >>>>> Hi Aki,
> >>>>>
> >>>>> Am 03.11.22 um 10:29 schrieb Aki Tuomi:
> >>>>>>> On 03/11/2022 11:27 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Aki,
> >>>>>>>
> >>>>>>> Am 03.11.22 um 09:12 schrieb Aki Tuomi:
> >>>>>>>>> On 03/11/2022 10:09 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Hi Aki,
> >>>>>>>>>
> >>>>>>>>> Am 03.11.22 um 08:50 schrieb Aki Tuomi:
> >>>>>>>>>>> On 03/11/2022 09:46 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I'm trying to migrate an old Cyrus 2.5 server to Dovecot 2.3.19 using
> >>>>>>>>>>> doveadm backup -R, which works for all folders but the INBOX itself,
> >>>>>>>>>>> which always stays empty.
> >>>>>>>>>>>
> >>>>>>>>>>> The Cyrus side uses altnamespace:no and unixhierarchysep:no, it's used
> >>>>>>>>>>> as imapc: remote in doveadm backup -R with imapc_list_prefix=INBOX
> >>>>>>>>>>>
> >>>>>>>>>>> Dovecot uses the following namespace to migrate into:
> >>>>>>>>>>>
> >>>>>>>>>>> namespace inboxes {
> >>>>>>>>>>> inbox = yes
> >>>>>>>>>>> location =
> >>>>>>>>>>> mailbox Sent {
> >>>>>>>>>>> auto = subscribe
> >>>>>>>>>>> special_use = \Sent
> >>>>>>>>>>> }
> >>>>>>>>>>> ### some more folders omitted ###
> >>>>>>>>>>> prefix = INBOX/
> >>>>>>>>>>> separator = /
> >>>>>>>>>>> subscriptions = no
> >>>>>>>>>>> }
> >>>>>>>>>> Hi!
> >>>>>>>>>>
> >>>>>>>>>> When syncing mailboxes from other server, you should use migration config file, which has **no** auto=subscribe or auto=create folders, as these can mess up with synchronization.
> >>>>>>>>>>
> >>>>>>>>>> Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ for more details.
> >>>>>>>>> Does a migration config file specified with doveadm -c <file> add to and
> >>>>>>>>> overwrite the existing Dovecot configuration for the time the command
> >>>>>>>>> runs, like the -o options, or do I need to start a separate server with
> >>>>>>>>> a full configuration to e.g. have my authentication and mailbox location
> >>>>>>>>> available?
> >>>>>>>>>
> >>>>>>>>> Ralf
> >>>>>>>>>
> >>>>>>>> It does not add/replace/overwrite configuration, you provide a fresh config file which is used *instead of* the default dovecot.conf.
> >>>>>>>>
> >>>>>>>> You don't need to run a separate instance necessarely, although in some larger migrations this has been used as well.
> >>>>>>> I created now a separate instance with a modified configuration file
> >>>>>>> with no auto=subscribe (or create), no replication and an empty storage.
> >>>>>>> doveadm config -n is attached.
> >>>>>>>
> >>>>>>> Unfortunately the result is identical to my previous tries:
> >>>>>>>
> >>>>>>> doveadm -o namespace/subs/location=mbox:/var/dovecot/subs -o
> >>>>>>> imapc_user='someuser' -o imapc_password='secret' -D backup -n INBOX/ -R
> >>>>>>> -u someuser at somedomain imapc: 2>&1 | tee /tmp/doveadm-backup.log
> >>>>>>>
> >>>>>>> Nov 03 09:06:35 dsync(someuser at somedomain): Warning: Mailbox changes
> >>>>>>> caused a desync. You may want to run dsync again: Remote lost mailbox
> >>>>>>> GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
> >>>>>>>
> >>>>>>> doveadm mailbox status -u someuser at somedomain all INBOX
> >>>>>>> INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
> >>>>>>> highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
> >>>>>>> firstsaved=never
> >>>>>>>
> >>>>>>> Any ideas what else to try or how to debug that further?
> >>>>>>>
> >>>>>>> I can send you the full log to your personal address, if that helps ...
> >>>>>>>
> >>>>>>> Ralf
> >>>>>> You should rm -rf the target folder first. Can you attach `doveadm -D backup` logs? Check that it won't contain passwords.
> >>>>> The mailbox directory did NOT exist before, therefore no need to rm -rf it.
> >>>>>
> >>>>> I send the logs to your private address only, I feel not comfortable to
> >>>>> post them on the list.
> >>>>>
> >>>>> Ralf
> >>>>>
> >>>> 1. You did not delete the mailbox.
> >>>>
> >>>> 2. You are using **mbox** for subscription namespace, please don't.
> >>>>
> >>>> Also
> >>>>
> >>>> Nov 03 09:05:33 dsync(): Debug: brain M: Local mailbox tree: INBOX guid=c8adef115c84636335000000effb6190 uid_validity=1667466332 uid_next=1 subs=no last_change=0 last_subs=0
> >>>> Nov 03 09:05:33 dsync(): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32746 subs=no last_change=0 last_subs=0
> >>>>
> >>>> This clearly shows that you did not, in fact, rm -rf the user's mailboxes prir running backup. Can you please try again, and clean up the user from t target mailsystem before running backup again?
> >>>>
> >>>> I would also strongly recommend not using ACL plugin while doing backup, unless you are backing up ACLs from source system.
> >>> I removed all ACL plugin and config from my Dovecot config and removed
> >>> the subscription namespace from my doveadm backup command, but now it
> >>> fails with an error:
> >>>
> >>> doveadm -o imapc_user='someuser' -o imapc_password='secret' -D backup -n
> >>> INBOX/ -R -u someuser at somedomain imapc:
> >>>
> >>> Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Effective
> >>> uid=90, gid=101, home=/var/dovecot/imap/somedomain/someuser
> >>> Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Home dir not
> >>> found: /var/dovecot/imap/somedomain/someuser
> >>>
> >>> Nov 03 11:12:15 doveadm(someuser at somedomain): Error: namespace
> >>> configuration error: subscriptions=yes namespace missing
> >>>
> >>> doveadm config -n is attached.
> >>>
> >>> Ralf
> >> You can keep the subscription namespace in your config, otherwise you have no place to store subscriptions into. Just don't use mbox driver for it, use sdbox instead.
> >>
> >> namespace subscriptions {
> >> hidden = yes
> >> list = no
> >> location = sdbox:/index/directory/%n/subs-shared/:LISTINDEX=root:SUBSCRIPTIONS=subscriptions-shared:LAYOUT=INDEX
> >> prefix =
> >> separator = /
> >> subscriptions = yes
> >> }
> >>
> >> Aki
> > Oh and forgot to mention, for INBOX namespace you can leave subscriptions turned on. This is only needed for shared/public namespaces in any case..
>
> I changed the config now, as you suggested (doveadm config -n is
> attached), unfortunately the result stays the same:
>
> doveadm -o imapc_user='someuser' -o imapc_password='secret' -D backup -n
> INBOX/ -R -u someuser at somedomain imapc: 2>&1|tee
> /tmp/doveadm-backup-someuser2.log
>
> Nov 03 11:35:39 dsync(someuser at somedomain): Warning: Mailbox changes
> caused a desync. You may want to run dsync again: Remote lost mailbox
> GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
>
> doveadm mailbox status -u someuser at somedomain all INBOX
> INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
> highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
> firstsaved=never
>
> I send the full log again to your private address.
>
> Ralf
>
Seems I can reproduce this issue, we'll look into this.
Aki
More information about the dovecot
mailing list