On Thu, Feb 02, 2012 at 12:23:01PM +0200, Timo Sirainen wrote:
Note that with SIS the attachments aren't compressed.
Yes, I know.
"/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is missing, but there are 205 other copies of this file named /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with identical sha1sum.
All of them have a link count of 2, with the other link being in hashes/ directory?
No, these has link count=207. I don't know what you mean by link being in hashes directory.
# ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|head -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0069222e0c080f4f754a0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-00ffb9312a370e4f6b610000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0442c5163ad3114fb4780000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-04f288390052144f012d0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-053b6c0f185a0d4fc4210000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-06c98213c3b30e4fac3c0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-076573234fbd0b4fa8620000bf34f6ab
This is just one example, I can provide tons of other examples.. Hmm, I see now that there are 206 files of that first example with the 207 links, and here's another other example with numlinks=7:
# ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|wc -l
206and numlinks=4:
# ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l
3is dovecot somehow creating numlinks+1 copies of every file it hardlinks?? Would explain my diskusage :-)
That is most likely related to your troubles. If the dsync runs crash, the result could leave extra files lying around etc..
If dsync backup is supposed to be a viable backup solution, I think it should fail much better. If it see errors on the target side it should clear the target and do a full sync. Manually cleaning up after it's problems is too much work.
Some samples:
Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX
This is a little bit strange. What is the doveconf -n output of the source server?
# 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server # release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 mail_gid = 3000 mail_home = /srv/mailstore/%256RHu/%d/%n mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 mail_plugins = quota zlib mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1u/%1.1u/%u/.dovecot.sieve sieve_dir = /sieve/%1u/%1.1u/%u sieve_max_script_size = 1M zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster@example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib imap_quota } protocol pop3 { mail_plugins = quota zlib pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o }
Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name
Is this a namespace prefix? It shouldn't be trying to sync a mailbox named this (there's an extra "." suffix).
I believe it's a folder named "INBOX.ferie 2006.", with the user using the namespace separator in the folder name.. I believe dovecot allows this, so it should also handle backing it up.
Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda
I guess some kind of mismatch related to namespace configuration.
They both have same namespace config I think, but Maildir vs. mdbox maybe changes something..
It would help if I could reproduce the errors that you're seeing. Can you easily reproduce them with some accounts? If so, if you can give enough details for me to reproduce the problems I can fix them. (Except for the "file not found" issues, since that problems occurred earlier already. I should probably somehow make Dovecot fix those missing files though..)
I'll look trough the logs to see if there are any errors that keep repeating for the same accounts.
-jf