Hello Adam, and the dovecot list

Just a question, I hate to pollute the thread, so feel free to push these questions into a new thread if deemed necessary. So as you can guess I'm a bit of a newb here, so rather obvious questions are about to arrive....

As you are using GIT for your archive (which is a cool idea by the way) I'm sure you are well aware that not all files types play nicely with version control, my question therefore is : How do you plan to handle attachments ?

Also, although I appreciate the idea of using git, emails generally don't 'change', but I guess that also depends on how you are storing them (single email with links to previous / next ... etc, or as a single big file for each specific thread). Although this is hitting my limits of understanding for how dovecot works, so I probably need educating on this (a pointer to the docs would be good).

You seem concerned regarding the files that you are ignoring that you will need to 'recreate them', so why not do a complete git add . prior to adding them into the git ignore, then you have an initial state for those files too.

Final thought, what advantage do you envisage by using git as opposed to simply using a filter to select the files over a certain age, and place them into a zipped TAR archive ? Although I guess you could eventually zip the git archive too, and in the interim it would remain searchable by your users mail clients whilt in git.

Thanks in advance, and apologies once again for polluting your question with my own.

David

On Tue, Oct 6, 2020 at 7:22 PM Adam <adam.ranek@seznam.cz> wrote:

Hi Everybody,

I'd like to start archiving e-mails by moving them to a server with running Dovecot.

I installed "dovecot-core" and "dovecot-imapd" (version 2.3.4.1) on Debian 10.

One of a few configurations I made is to use Maildir:

# grep '^mail_location = ' /etc/dovecot/conf.d/10-mail.conf
mail_location = maildir:~/Maildir

I successfully moved some e-mails (at this moment it's just a test) to subfolders that describe a year of the backup.

I'd like to start using git on the whole Maildir. After going through the Maildir directory and subdirectories I realized that Dovecot saves some extra files that I might not need to backup (version control).

Based on my understanding and after reading https://wiki.dovecot.org/IndexFiles I believe that the only files I need to archive are files in "cur/" directory and these files should start with Unix time.

So I decided to write a ".gitignore" whitelist that will include only files in "cur/" that start with a number:

# cat .gitignore
-----------------------------------
!.gitignore
!*/
!/Maildir/cur/[0-9]*
!/Maildir/*/cur/[0-9]*
-----------------------------------

and it's working:

# git add . --dry-run
-----------------------------------
add '.gitignore'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M110302P17113.mail1,S=8066,W=8276:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M134554P17113.mail1,S=8024,W=8234:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M157569P17113.mail1,S=8005,W=8215:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M180965P17113.mail1,S=8021,W=8231:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M205518P17113.mail1,S=8017,W=8227:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M355896P17113.mail1,S=8042,W=8252:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M52133P17113.mail1,S=8053,W=8263:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M555847P17113.mail1,S=8029,W=8239:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M581502P17113.mail1,S=8047,W=8257:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M606379P17113.mail1,S=8044,W=8254:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M628850P17113.mail1,S=8036,W=8246:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M657431P17113.mail1,S=8069,W=8279:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M681265P17113.mail1,S=8039,W=8249:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M706585P17113.mail1,S=8073,W=8283:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M730046P17113.mail1,S=8059,W=8269:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M753487P17113.mail1,S=8034,W=8244:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M776202P17113.mail1,S=8019,W=8229:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M799238P17113.mail1,S=8085,W=8295:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M822106P17113.mail1,S=8081,W=8291:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M844397P17113.mail1,S=8102,W=8312:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M867952P17113.mail1,S=7964,W=8174:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M889882P17113.mail1,S=7992,W=8202:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M912429P17113.mail1,S=8005,W=8215:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M935326P17113.mail1,S=8027,W=8237:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M959570P17113.mail1,S=8037,W=8247:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986418.M987388P17113.mail1,S=8018,W=8228:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M112704P17113.mail1,S=8044,W=8254:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M13262P17113.mail1,S=8025,W=8235:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M321294P17113.mail1,S=8012,W=8222:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M347297P17113.mail1,S=8014,W=8224:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M36960P17113.mail1,S=8037,W=8247:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M371029P17113.mail1,S=8005,W=8215:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M395270P17113.mail1,S=8024,W=8234:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M418277P17113.mail1,S=8046,W=8256:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M441021P17113.mail1,S=8047,W=8257:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M466329P17113.mail1,S=8025,W=8235:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M491134P17113.mail1,S=8038,W=8248:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M515650P17113.mail1,S=8046,W=8256:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M539950P17113.mail1,S=8017,W=8227:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M563073P17113.mail1,S=7983,W=8193:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M587009P17113.mail1,S=7963,W=8173:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M611652P17113.mail1,S=7994,W=8204:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M65180P17113.mail1,S=8015,W=8225:2,S'
add 'Maildir/.INBOX.years.2020.family/cur/1601986419.M90376P17113.mail1,S=8036,W=8246:2,S'
-----------------------------------

Would you be so kind and confirm that this is all I need to archive and that Dovecot will/can re-create all files ignored by git?

For example based on my research the following files / directories would be ignored:
- new/ (empty dir)
- tmp/ (empty dir)
- maildirfolder/ (dir with maildirfolder file)
- dovecot-uidlist
- dovecot-uidvalidity
- dovecot-uidvalidity.5f7c4f24
- dovecot.index
- dovecot.index.cache
- dovecot.index.log
- dovecot.index.log.2
- dovecot.list.index
- dovecot.list.index.log
- dovecot.list.index.log.2
- dovecot.mailbox.log

In case of my test:
# git status --ignored
-----------------------------------
On branch master

No commits yet

Untracked files:
(use "git add <file>..." to include in what will be committed)

.gitignore
Maildir/

Ignored files:
(use "git add -f <file>..." to include in what will be committed)

Maildir/.INBOX.years.2020.family/dovecot-uidlist
Maildir/.INBOX.years.2020.family/dovecot.index
Maildir/.INBOX.years.2020.family/dovecot.index.cache
Maildir/.INBOX.years.2020.family/dovecot.index.log
Maildir/.INBOX.years.2020.family/dovecot.index.log.2
Maildir/.INBOX.years.2020.family/maildirfolder
Maildir/dovecot-uidlist
Maildir/dovecot-uidvalidity
Maildir/dovecot-uidvalidity.5f7c4f24
Maildir/dovecot.index.log
Maildir/dovecot.list.index
Maildir/dovecot.list.index.log
Maildir/dovecot.list.index.log.2
Maildir/dovecot.mailbox.log
Maildir/subscriptions

nothing added to commit but untracked files present (use "git add" to track)
-----------------------------------

Thank you.

Adam