[Dovecot] Doubts about dsync, mdbox, SIS
Jan-Frode Myklebust
janfrode at tanso.net
Thu Feb 2 13:31:20 EET 2012
On Thu, Feb 02, 2012 at 12:23:01PM +0200, Timo Sirainen wrote:
>
> Note that with SIS the attachments aren't compressed.
Yes, I know.
>
> > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is
> > missing, but there are 205 other copies of this file named
> > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with
> > identical sha1sum.
>
> All of them have a link count of 2, with the other link being in hashes/
> directory?
No, these has link count=207. I don't know what you mean by link being
in hashes directory.
# ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|head
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0069222e0c080f4f754a0000bf34f6ab
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-00ffb9312a370e4f6b610000bf34f6ab
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0442c5163ad3114fb4780000bf34f6ab
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-04f288390052144f012d0000bf34f6ab
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-053b6c0f185a0d4fc4210000bf34f6ab
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-06c98213c3b30e4fac3c0000bf34f6ab
-rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-076573234fbd0b4fa8620000bf34f6ab
This is just one example, I can provide tons of other examples.. Hmm,
I see now that there are 206 files of that first example with the 207
links, and here's another other example with numlinks=7:
# ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|wc -l
206
and numlinks=4:
# ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l
3
is dovecot somehow creating numlinks+1 copies of every file it
hardlinks?? Would explain my diskusage :-)
> That is most likely related to your troubles. If the dsync runs crash,
> the result could leave extra files lying around etc..
If dsync backup is supposed to be a viable backup solution, I think it
should fail much better. If it see errors on the target side it should
clear the target and do a full sync. Manually cleaning up after it's
problems is too much work.
>
> > Some samples:
> >
> > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX
>
> This is a little bit strange. What is the doveconf -n output of the
> source server?
----------------------------------------------------------------------------
# 2.0.14: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server
# release 5.5 (Tikanga)
auth_cache_size = 100 M
auth_verbose = yes
auth_verbose_passwords = sha1
disable_plaintext_auth = no
login_trusted_networks = 192.168.0.0/16
mail_gid = 3000
mail_home = /srv/mailstore/%256RHu/%d/%n
mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u
mail_max_userip_connections = 20
mail_plugins = quota zlib
mail_uid = 3000
maildir_stat_dirs = yes
maildir_very_dirty_syncs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date
mmap_disable = yes
namespace {
inbox = yes
location =
prefix = INBOX.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
quota = dict:UserQuota::file:%h/dovecot-quota
sieve = /sieve/%1u/%1.1u/%u/.dovecot.sieve
sieve_dir = /sieve/%1u/%1.1u/%u
sieve_max_script_size = 1M
zlib_save = gz
zlib_save_level = 6
}
postmaster_address = postmaster at example.net
protocols = imap pop3 lmtp sieve
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 4521
unix_listener auth-userdb {
group =
mode = 0600
user = atmail
}
}
service imap-login {
inet_listener imap {
address = *
port = 143
}
process_min_avail = 4
service_count = 0
vsz_limit = 1 G
}
service imap-postlogin {
executable = script-login /usr/local/sbin/imap-postlogin.sh
}
service imap {
executable = imap imap-postlogin
process_limit = 2048
}
service lmtp {
client_limit = 1
inet_listener lmtp {
address = *
port = 24
}
process_limit = 25
}
service managesieve-login {
inet_listener sieve {
address = *
port = 4190
}
service_count = 1
}
service pop3-login {
inet_listener pop3 {
address = *
port = 110
}
process_min_avail = 4
service_count = 0
vsz_limit = 1 G
}
service pop3-postlogin {
executable = script-login /usr/local/sbin/pop3-postlogin.sh
}
service pop3 {
executable = pop3 pop3-postlogin
process_limit = 2048
}
ssl = no
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = quota zlib sieve
}
protocol imap {
imap_client_workarounds = delay-newmail
mail_plugins = quota zlib imap_quota
}
protocol pop3 {
mail_plugins = quota zlib
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_uidl_format = UID%u-%v
}
protocol sieve {
managesieve_logout_format = bytes=%i/%o
}
----------------------------------------------------------------------------
>
> > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name
>
> Is this a namespace prefix? It shouldn't be trying to sync a mailbox
> named this (there's an extra "." suffix).
I believe it's a folder named "INBOX.ferie 2006.", with the user using
the namespace separator in the folder name.. I believe dovecot allows
this, so it should also handle backing it up.
>
> > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda
>
> I guess some kind of mismatch related to namespace configuration.
They both have same namespace config I think, but Maildir vs. mdbox
maybe changes something..
>
> It would help if I could reproduce the errors that you're seeing. Can
> you easily reproduce them with some accounts? If so, if you can give
> enough details for me to reproduce the problems I can fix them. (Except
> for the "file not found" issues, since that problems occurred earlier
> already. I should probably somehow make Dovecot fix those missing files
> though..)
I'll look trough the logs to see if there are any errors that keep
repeating for the same accounts.
-jf
More information about the dovecot
mailing list