Hi Dovecot community,
I would like to share a problem I saw a few days ago. Any comment is highly appreciated.
*Steps to reproduce* 0. Version of Dovecot I use is v2.3.11.3.
Prepare 2 Dovecot backends:
- One backend is used as the source of dsync command. Other backend is used as the destination of the dsync command.
- Mailbox format is Maildir. Mail dir of two test accounts is empty.
Prepare the test account in the source of dsync:
- Send a test message to the test account. For example, I use swaks:
swaks --server 127.0.0.1 --port 24 --protocol LMTP --to user1@local
- Use doveadm expunge command to clean mail dir of the test account:
doveadm expunge -u user1@local mailbox INBOX all 3. Prepare the test account in the destination of dsync:
- Log in test account via IMAP then select INBOX first:
telnet 127.0.0.1 imap ... s login user1@local 1 ... s select INBOX ... s logout
- Then perform dsync:
doveadm -o imapc_host=backend2.local -o imapc_port=143 -o imapc_user=user1@local -o imapc_password=1 backup -s '' -R -u user1@local imapc:
- Confirm error occurs when using dsync:
dsync(user1@local): Info: imapc(backend2.local:143): Connected to 192.168.100.2:143 (local 192.168.100.27:58850) dsync(user1@local): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. AQAAAHm4+Jk= 4. Log in the test account in dsync destination then select INBOX. Repeat many times. I can see the size of the transaction log file is increasing continuously. *Before*: -rw-------. 1 vmail vmail 456 Apr 22 10:54 dovecot.index.log
*After*: -rw-------. 1 vmail vmail 544 Apr 22 10:59 dovecot.index.log
-rw-------. 1 vmail vmail 632 Apr 22 11:00 dovecot.index.log
-rw-------. 1 vmail vmail 720 Apr 22 11:00 dovecot.index.log
*Problems*
- Size of the transaction log file is increasing continuously. Do you think this is an issue of Dovecot?
- The value of *mail_index_log_rotate_max_size* is 1M by default. However the transaction log file dovecot.index.log is not rotated when its size exceeds limit.
- Because the size of the transaction log file is too big and it is not rotated, I think "out of memory" error will occur when I try to use IMAP SELECT or POP3 STAT command (if the virtual size limit is small enough).
- If "size of the transaction log file is increasing continuously" is an issue, what should I do to avoid it or is there any workaround? I understand I should fix the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* first before doing anything next. I think removing the transaction log file is an understandable workaround.
*Investigaton*
- About the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted*, I can understand the cause after reading this thread https://dovecot.org/pipermail/dovecot/2014-April/095588.html
- Maildir + INBOX + backup/sync/replicate
In the test scenarios where the INBOX on one side was to be completely removed, e.g. doveadm backup -R the dsync failed and nothing was synced to the target. This is because before moving the source mails to the mailbox, dsync cleans out the old ones ( -R preserves nothing) and in Maildir the INBOX can not be removed. This is a feature/not easily solvable, because in Maildir INBOX is different from other folders.
The transaction log file is rotated whenever there's any change in index file. I can confirm by sending a test message to the test account in dsync destination. I can see that logic in *src/lib-index/mail-index-sync.c* and *src/lib-index/mail-index-write.c*. After that, the size of the new transaction log file is not increasing anymore.
When the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* occurred, I tried to dump the transaction log file and I saw these records:
record: offset=432, type=index-deleted (ext), size=12 record: offset=444, type=index-undeleted (ext), size=12
- Do you think these records relate to the issue that the size of the transaction log file is increasing continuously?
- Did the error *Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted* make content of transaction log file incomplete/incorrect?
Regards, Anh Do