doveadm backup|sync works for every folder but INBOX

[Aki Tuomi]

> On 03/11/2022 13:19 EET Ralf Becker <rb at egroupware.org> wrote:
> 
>  
> Hi Aki,
> 
> Am 03.11.22 um 10:54 schrieb Aki Tuomi:
> >> On 03/11/2022 11:46 EET Ralf Becker <rb at egroupware.org> wrote:
> >>
> >>   
> >> Hi Aki,
> >>
> >> Am 03.11.22 um 10:29 schrieb Aki Tuomi:
> >>>> On 03/11/2022 11:27 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>
> >>>>    
> >>>> Hi Aki,
> >>>>
> >>>> Am 03.11.22 um 09:12 schrieb Aki Tuomi:
> >>>>>> On 03/11/2022 10:09 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>>>
> >>>>>>     
> >>>>>> Hi Aki,
> >>>>>>
> >>>>>> Am 03.11.22 um 08:50 schrieb Aki Tuomi:
> >>>>>>>> On 03/11/2022 09:46 EET Ralf Becker <rb at egroupware.org> wrote:
> >>>>>>>>
> >>>>>>>>      
> >>>>>>>> I'm trying to migrate an old Cyrus 2.5 server to Dovecot 2.3.19 using
> >>>>>>>> doveadm backup -R, which works for all folders but the INBOX itself,
> >>>>>>>> which always stays empty.
> >>>>>>>>
> >>>>>>>> The Cyrus side uses altnamespace:no and unixhierarchysep:no, it's used
> >>>>>>>> as imapc: remote in doveadm backup -R with imapc_list_prefix=INBOX
> >>>>>>>>
> >>>>>>>> Dovecot uses the following namespace to migrate into:
> >>>>>>>>
> >>>>>>>> namespace inboxes {
> >>>>>>>>        inbox = yes
> >>>>>>>>        location =
> >>>>>>>>        mailbox Sent {
> >>>>>>>>          auto = subscribe
> >>>>>>>>          special_use = \Sent
> >>>>>>>>        }
> >>>>>>>>        ### some more folders omitted ###
> >>>>>>>>        prefix = INBOX/
> >>>>>>>>        separator = /
> >>>>>>>>        subscriptions = no
> >>>>>>>> }
> >>>>>>> Hi!
> >>>>>>>
> >>>>>>> When syncing mailboxes from other server, you should use migration config file, which has **no** auto=subscribe or auto=create folders, as these can mess up with synchronization.
> >>>>>>>
> >>>>>>> Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ for more details.
> >>>>>> Does a migration config file specified with doveadm -c <file> add to and
> >>>>>> overwrite the existing Dovecot configuration for the time the command
> >>>>>> runs, like the -o options, or do I need to start a separate server with
> >>>>>> a full configuration to e.g. have my authentication and mailbox location
> >>>>>> available?
> >>>>>>
> >>>>>> Ralf
> >>>>>>
> >>>>> It does not add/replace/overwrite configuration, you provide a fresh config file which is used *instead of* the default dovecot.conf.
> >>>>>
> >>>>> You don't need to run a separate instance necessarely, although in some larger migrations this has been used as well.
> >>>> I created now a separate instance with a modified configuration file
> >>>> with no auto=subscribe (or create), no replication and an empty storage.
> >>>> doveadm config -n is attached.
> >>>>
> >>>> Unfortunately the result is identical to my previous tries:
> >>>>
> >>>> doveadm -o namespace/subs/location=mbox:/var/dovecot/subs -o
> >>>> imapc_user='someuser' -o imapc_password='secret' -D backup -n INBOX/ -R
> >>>> -u someuser at somedomain imapc: 2>&1 | tee /tmp/doveadm-backup.log
> >>>>
> >>>> Nov 03 09:06:35 dsync(someuser at somedomain): Warning: Mailbox changes
> >>>> caused a desync. You may want to run dsync again: Remote lost mailbox
> >>>> GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
> >>>>
> >>>> doveadm mailbox status -u someuser at somedomain all INBOX
> >>>> INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
> >>>> highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
> >>>> firstsaved=never
> >>>>
> >>>> Any ideas what else to try or how to debug that further?
> >>>>
> >>>> I can send you the full log to your personal address, if that helps ...
> >>>>
> >>>> Ralf
> >>> You should rm -rf the target folder first. Can you attach `doveadm -D backup` logs? Check that it won't contain passwords.
> >> The mailbox directory did NOT exist before, therefore no need to rm -rf it.
> >>
> >> I send the logs to your private address only, I feel not comfortable to
> >> post them on the list.
> >>
> >> Ralf
> >>
> > 1. You did not delete the mailbox.
> >
> > 2. You are using **mbox** for subscription namespace, please don't.
> >
> > Also
> >
> > Nov 03 09:05:33 dsync(): Debug: brain M: Local mailbox tree: INBOX guid=c8adef115c84636335000000effb6190 uid_validity=1667466332 uid_next=1 subs=no last_change=0 last_subs=0
> > Nov 03 09:05:33 dsync(): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32746 subs=no last_change=0 last_subs=0
> >
> > This clearly shows that you did not, in fact, rm -rf the user's mailboxes prir running backup. Can you please try again, and clean up the user from t target mailsystem before running backup again?
> >
> > I would also strongly recommend not using ACL plugin while doing backup, unless you are backing up ACLs from source system.
> 
> I removed all ACL plugin and config from my Dovecot config and removed 
> the subscription namespace from my doveadm backup command, but now it 
> fails with an error:
> 
> doveadm -o imapc_user='someuser' -o imapc_password='secret' -D backup -n 
> INBOX/ -R -u someuser at somedomain imapc:
> 
> Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Effective 
> uid=90, gid=101, home=/var/dovecot/imap/somedomain/someuser
> Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Home dir not 
> found: /var/dovecot/imap/somedomain/someuser
> 
> Nov 03 11:12:15 doveadm(someuser at somedomain): Error: namespace 
> configuration error: subscriptions=yes namespace missing
> 
> doveadm config -n is attached.
> 
> Ralf

You can keep the subscription namespace in your config, otherwise you have no place to store subscriptions into. Just don't use mbox driver for it, use sdbox instead.

namespace subscriptions {
  hidden = yes
  list = no
  location = sdbox:/index/directory/%n/subs-shared/:LISTINDEX=root:SUBSCRIPTIONS=subscriptions-shared:LAYOUT=INDEX
  prefix = 
  separator = /
  subscriptions = yes
}

Aki