doveadm backup|sync works for every folder but INBOX
Ralf Becker
rb at egroupware.org
Thu Nov 3 13:42:02 UTC 2022
Am 03.11.22 um 14:29 schrieb Aki Tuomi:
>> On 03/11/2022 13:43 EET Ralf Becker <rb at egroupware.org> wrote:
>>
>>
>> Hi Aki,
>>
>> Am 03.11.22 um 12:27 schrieb Aki Tuomi:
>>>> On 03/11/2022 13:23 EET Aki Tuomi <aki.tuomi at open-xchange.com> wrote:
>>>>> On 03/11/2022 13:19 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>
>>>>>
>>>>> Hi Aki,
>>>>>
>>>>> Am 03.11.22 um 10:54 schrieb Aki Tuomi:
>>>>>>> On 03/11/2022 11:46 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi Aki,
>>>>>>>
>>>>>>> Am 03.11.22 um 10:29 schrieb Aki Tuomi:
>>>>>>>>> On 03/11/2022 11:27 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi Aki,
>>>>>>>>>
>>>>>>>>> Am 03.11.22 um 09:12 schrieb Aki Tuomi:
>>>>>>>>>>> On 03/11/2022 10:09 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi Aki,
>>>>>>>>>>>
>>>>>>>>>>> Am 03.11.22 um 08:50 schrieb Aki Tuomi:
>>>>>>>>>>>>> On 03/11/2022 09:46 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm trying to migrate an old Cyrus 2.5 server to Dovecot 2.3.19 using
>>>>>>>>>>>>> doveadm backup -R, which works for all folders but the INBOX itself,
>>>>>>>>>>>>> which always stays empty.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The Cyrus side uses altnamespace:no and unixhierarchysep:no, it's used
>>>>>>>>>>>>> as imapc: remote in doveadm backup -R with imapc_list_prefix=INBOX
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dovecot uses the following namespace to migrate into:
>>>>>>>>>>>>>
>>>>>>>>>>>>> namespace inboxes {
>>>>>>>>>>>>> inbox = yes
>>>>>>>>>>>>> location =
>>>>>>>>>>>>> mailbox Sent {
>>>>>>>>>>>>> auto = subscribe
>>>>>>>>>>>>> special_use = \Sent
>>>>>>>>>>>>> }
>>>>>>>>>>>>> ### some more folders omitted ###
>>>>>>>>>>>>> prefix = INBOX/
>>>>>>>>>>>>> separator = /
>>>>>>>>>>>>> subscriptions = no
>>>>>>>>>>>>> }
>>>>>>>>>>>> Hi!
>>>>>>>>>>>>
>>>>>>>>>>>> When syncing mailboxes from other server, you should use migration config file, which has **no** auto=subscribe or auto=create folders, as these can mess up with synchronization.
>>>>>>>>>>>>
>>>>>>>>>>>> Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ for more details.
>>>>>>>>>>> Does a migration config file specified with doveadm -c <file> add to and
>>>>>>>>>>> overwrite the existing Dovecot configuration for the time the command
>>>>>>>>>>> runs, like the -o options, or do I need to start a separate server with
>>>>>>>>>>> a full configuration to e.g. have my authentication and mailbox location
>>>>>>>>>>> available?
>>>>>>>>>>>
>>>>>>>>>>> Ralf
>>>>>>>>>>>
>>>>>>>>>> It does not add/replace/overwrite configuration, you provide a fresh config file which is used *instead of* the default dovecot.conf.
>>>>>>>>>>
>>>>>>>>>> You don't need to run a separate instance necessarely, although in some larger migrations this has been used as well.
>>>>>>>>> I created now a separate instance with a modified configuration file
>>>>>>>>> with no auto=subscribe (or create), no replication and an empty storage.
>>>>>>>>> doveadm config -n is attached.
>>>>>>>>>
>>>>>>>>> Unfortunately the result is identical to my previous tries:
>>>>>>>>>
>>>>>>>>> doveadm -o namespace/subs/location=mbox:/var/dovecot/subs -o
>>>>>>>>> imapc_user='someuser' -o imapc_password='secret' -D backup -n INBOX/ -R
>>>>>>>>> -u someuser at somedomain imapc: 2>&1 | tee /tmp/doveadm-backup.log
>>>>>>>>>
>>>>>>>>> Nov 03 09:06:35 dsync(someuser at somedomain): Warning: Mailbox changes
>>>>>>>>> caused a desync. You may want to run dsync again: Remote lost mailbox
>>>>>>>>> GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
>>>>>>>>>
>>>>>>>>> doveadm mailbox status -u someuser at somedomain all INBOX
>>>>>>>>> INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
>>>>>>>>> highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
>>>>>>>>> firstsaved=never
>>>>>>>>>
>>>>>>>>> Any ideas what else to try or how to debug that further?
>>>>>>>>>
>>>>>>>>> I can send you the full log to your personal address, if that helps ...
>>>>>>>>>
>>>>>>>>> Ralf
>>>>>>>> You should rm -rf the target folder first. Can you attach `doveadm -D backup` logs? Check that it won't contain passwords.
>>>>>>> The mailbox directory did NOT exist before, therefore no need to rm -rf it.
>>>>>>>
>>>>>>> I send the logs to your private address only, I feel not comfortable to
>>>>>>> post them on the list.
>>>>>>>
>>>>>>> Ralf
>>>>>>>
>>>>>> 1. You did not delete the mailbox.
>>>>>>
>>>>>> 2. You are using **mbox** for subscription namespace, please don't.
>>>>>>
>>>>>> Also
>>>>>>
>>>>>> Nov 03 09:05:33 dsync(): Debug: brain M: Local mailbox tree: INBOX guid=c8adef115c84636335000000effb6190 uid_validity=1667466332 uid_next=1 subs=no last_change=0 last_subs=0
>>>>>> Nov 03 09:05:33 dsync(): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32746 subs=no last_change=0 last_subs=0
>>>>>>
>>>>>> This clearly shows that you did not, in fact, rm -rf the user's mailboxes prir running backup. Can you please try again, and clean up the user from t target mailsystem before running backup again?
>>>>>>
>>>>>> I would also strongly recommend not using ACL plugin while doing backup, unless you are backing up ACLs from source system.
>>>>> I removed all ACL plugin and config from my Dovecot config and removed
>>>>> the subscription namespace from my doveadm backup command, but now it
>>>>> fails with an error:
>>>>>
>>>>> doveadm -o imapc_user='someuser' -o imapc_password='secret' -D backup -n
>>>>> INBOX/ -R -u someuser at somedomain imapc:
>>>>>
>>>>> Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Effective
>>>>> uid=90, gid=101, home=/var/dovecot/imap/somedomain/someuser
>>>>> Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Home dir not
>>>>> found: /var/dovecot/imap/somedomain/someuser
>>>>>
>>>>> Nov 03 11:12:15 doveadm(someuser at somedomain): Error: namespace
>>>>> configuration error: subscriptions=yes namespace missing
>>>>>
>>>>> doveadm config -n is attached.
>>>>>
>>>>> Ralf
>>>> You can keep the subscription namespace in your config, otherwise you have no place to store subscriptions into. Just don't use mbox driver for it, use sdbox instead.
>>>>
>>>> namespace subscriptions {
>>>> hidden = yes
>>>> list = no
>>>> location = sdbox:/index/directory/%n/subs-shared/:LISTINDEX=root:SUBSCRIPTIONS=subscriptions-shared:LAYOUT=INDEX
>>>> prefix =
>>>> separator = /
>>>> subscriptions = yes
>>>> }
>>>>
>>>> Aki
>>> Oh and forgot to mention, for INBOX namespace you can leave subscriptions turned on. This is only needed for shared/public namespaces in any case..
>> I changed the config now, as you suggested (doveadm config -n is
>> attached), unfortunately the result stays the same:
>>
>> doveadm -o imapc_user='someuser' -o imapc_password='secret' -D backup -n
>> INBOX/ -R -u someuser at somedomain imapc: 2>&1|tee
>> /tmp/doveadm-backup-someuser2.log
>>
>> Nov 03 11:35:39 dsync(someuser at somedomain): Warning: Mailbox changes
>> caused a desync. You may want to run dsync again: Remote lost mailbox
>> GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
>>
>> doveadm mailbox status -u someuser at somedomain all INBOX
>> INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
>> highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
>> firstsaved=never
>>
>> I send the full log again to your private address.
>>
>> Ralf
>>
> Seems I can reproduce this issue, we'll look into this.
>
> Aki
Thanks Aki :)
Please reply to this release, if you have any news, preferably with a
commit, so I can test ...
Ralf
--
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0
More information about the dovecot
mailing list