doveadm backup|sync works for every folder but INBOX
Ralf Becker
rb at egroupware.org
Thu Nov 3 11:19:45 UTC 2022
Hi Aki,
Am 03.11.22 um 10:54 schrieb Aki Tuomi:
>> On 03/11/2022 11:46 EET Ralf Becker <rb at egroupware.org> wrote:
>>
>>
>> Hi Aki,
>>
>> Am 03.11.22 um 10:29 schrieb Aki Tuomi:
>>>> On 03/11/2022 11:27 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>
>>>>
>>>> Hi Aki,
>>>>
>>>> Am 03.11.22 um 09:12 schrieb Aki Tuomi:
>>>>>> On 03/11/2022 10:09 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>>
>>>>>>
>>>>>> Hi Aki,
>>>>>>
>>>>>> Am 03.11.22 um 08:50 schrieb Aki Tuomi:
>>>>>>>> On 03/11/2022 09:46 EET Ralf Becker <rb at egroupware.org> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm trying to migrate an old Cyrus 2.5 server to Dovecot 2.3.19 using
>>>>>>>> doveadm backup -R, which works for all folders but the INBOX itself,
>>>>>>>> which always stays empty.
>>>>>>>>
>>>>>>>> The Cyrus side uses altnamespace:no and unixhierarchysep:no, it's used
>>>>>>>> as imapc: remote in doveadm backup -R with imapc_list_prefix=INBOX
>>>>>>>>
>>>>>>>> Dovecot uses the following namespace to migrate into:
>>>>>>>>
>>>>>>>> namespace inboxes {
>>>>>>>> inbox = yes
>>>>>>>> location =
>>>>>>>> mailbox Sent {
>>>>>>>> auto = subscribe
>>>>>>>> special_use = \Sent
>>>>>>>> }
>>>>>>>> ### some more folders omitted ###
>>>>>>>> prefix = INBOX/
>>>>>>>> separator = /
>>>>>>>> subscriptions = no
>>>>>>>> }
>>>>>>> Hi!
>>>>>>>
>>>>>>> When syncing mailboxes from other server, you should use migration config file, which has **no** auto=subscribe or auto=create folders, as these can mess up with synchronization.
>>>>>>>
>>>>>>> Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ for more details.
>>>>>> Does a migration config file specified with doveadm -c <file> add to and
>>>>>> overwrite the existing Dovecot configuration for the time the command
>>>>>> runs, like the -o options, or do I need to start a separate server with
>>>>>> a full configuration to e.g. have my authentication and mailbox location
>>>>>> available?
>>>>>>
>>>>>> Ralf
>>>>>>
>>>>> It does not add/replace/overwrite configuration, you provide a fresh config file which is used *instead of* the default dovecot.conf.
>>>>>
>>>>> You don't need to run a separate instance necessarely, although in some larger migrations this has been used as well.
>>>> I created now a separate instance with a modified configuration file
>>>> with no auto=subscribe (or create), no replication and an empty storage.
>>>> doveadm config -n is attached.
>>>>
>>>> Unfortunately the result is identical to my previous tries:
>>>>
>>>> doveadm -o namespace/subs/location=mbox:/var/dovecot/subs -o
>>>> imapc_user='someuser' -o imapc_password='secret' -D backup -n INBOX/ -R
>>>> -u someuser at somedomain imapc: 2>&1 | tee /tmp/doveadm-backup.log
>>>>
>>>> Nov 03 09:06:35 dsync(someuser at somedomain): Warning: Mailbox changes
>>>> caused a desync. You may want to run dsync again: Remote lost mailbox
>>>> GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
>>>>
>>>> doveadm mailbox status -u someuser at somedomain all INBOX
>>>> INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
>>>> highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
>>>> firstsaved=never
>>>>
>>>> Any ideas what else to try or how to debug that further?
>>>>
>>>> I can send you the full log to your personal address, if that helps ...
>>>>
>>>> Ralf
>>> You should rm -rf the target folder first. Can you attach `doveadm -D backup` logs? Check that it won't contain passwords.
>> The mailbox directory did NOT exist before, therefore no need to rm -rf it.
>>
>> I send the logs to your private address only, I feel not comfortable to
>> post them on the list.
>>
>> Ralf
>>
> 1. You did not delete the mailbox.
>
> 2. You are using **mbox** for subscription namespace, please don't.
>
> Also
>
> Nov 03 09:05:33 dsync(): Debug: brain M: Local mailbox tree: INBOX guid=c8adef115c84636335000000effb6190 uid_validity=1667466332 uid_next=1 subs=no last_change=0 last_subs=0
> Nov 03 09:05:33 dsync(): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32746 subs=no last_change=0 last_subs=0
>
> This clearly shows that you did not, in fact, rm -rf the user's mailboxes prir running backup. Can you please try again, and clean up the user from t target mailsystem before running backup again?
>
> I would also strongly recommend not using ACL plugin while doing backup, unless you are backing up ACLs from source system.
I removed all ACL plugin and config from my Dovecot config and removed
the subscription namespace from my doveadm backup command, but now it
fails with an error:
doveadm -o imapc_user='someuser' -o imapc_password='secret' -D backup -n
INBOX/ -R -u someuser at somedomain imapc:
Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Effective
uid=90, gid=101, home=/var/dovecot/imap/somedomain/someuser
Nov 03 11:12:15 doveadm(someuser at somedomain 156): Debug: Home dir not
found: /var/dovecot/imap/somedomain/someuser
Nov 03 11:12:15 doveadm(someuser at somedomain): Error: namespace
configuration error: subscriptions=yes namespace missing
doveadm config -n is attached.
Ralf
--
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Geschäftsführer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0
-------------- next part --------------
# 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 4.15.0-140-generic x86_64
# Hostname: 6cc0d9aef9ad
auth_cache_negative_ttl = 2 mins
auth_cache_size = 10 M
auth_cache_ttl = 5 mins
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@#"
default_client_limit = 3500
default_process_limit = 512
disable_plaintext_auth = no
doveadm_port = 26
first_valid_uid = 90
imapc_features = rfc822.size fetch-headers
imapc_host = 10.44.88.3
imapc_list_prefix = INBOX
listen = *
log_path = /dev/stderr
login_greeting = Dovecot FRA.khs ready
mail_access_groups = dovecot
mail_attribute_dict = file:%h/dovecot-metadata
mail_fsync = never
mail_gid = dovecot
mail_location = mdbox:~/mdbox
mail_log_prefix = "%s(%u %p): "
mail_max_userip_connections = 200
mail_plugins = quota notify replication mail_log
mail_prefetch_count = 20
mail_uid = dovecot
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.debug
mbox_min_index_size = 1000 B
mdbox_rotate_size = 50 M
namespace inboxes {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix = INBOX/
separator = /
subscriptions = no
}
passdb {
args = /etc/dovecot/dovecot-dict-master-auth.conf
driver = dict
master = yes
}
passdb {
args = /etc/dovecot/dovecot-dict-auth.conf
driver = dict
}
plugin {
mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
mail_log_fields = uid box msgid size
quota = dict:User quota::ns=INBOX/:file:%h/dovecot-quota
quota_rule = *:storage=200GB
sieve = ~/sieve/dovecot.sieve
sieve_after = /var/dovecot/sieve/after.d/
sieve_before = /var/dovecot/sieve/before.d/
sieve_dir = ~/sieve
sieve_extensions = +editheader
sieve_user_log = ~/.sieve.log
}
postmaster_address = admins at egroupware.org
protocols = imap pop3 lmtp sieve
quota_full_tempfail = yes
service auth-worker {
user = $default_internal_user
}
service auth {
drop_priv_before_exec = no
inet_listener {
port = 113
}
}
service doveadm {
inet_listener {
port = 26
}
vsz_limit = 640 M
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
process_min_avail = 5
service_count = 1
vsz_limit = 64 M
}
service imap {
executable = imap
process_limit = 2048
vsz_limit = 640 M
}
service lmtp {
inet_listener lmtp {
port = 24
}
unix_listener lmtp {
mode = 0666
}
vsz_limit = 512 M
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
inet_listener sieve_deprecated {
port = 2000
}
}
service pop3-login {
inet_listener pop3 {
port = 110
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service pop3 {
executable = pop3
}
service postlogin {
executable = script-login -d rawlog -b -t
}
ssl_cert = </etc/certs/mail.egroupware.org.pem
ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
userdb {
args = /etc/dovecot/dovecot-dict-auth.conf
driver = dict
}
verbose_proctitle = yes
protocol lda {
mail_plugins = quota notify replication mail_log sieve quota
}
protocol imap {
imap_metadata = yes
mail_max_userip_connections = 200
mail_plugins = quota notify replication mail_log quota imap_quota
}
protocol lmtp {
mail_max_lock_timeout = 25 secs
mail_plugins = quota notify replication mail_log sieve quota
}
More information about the dovecot
mailing list