How to move/reorganise existing e-mails to yearly subfolders

[Robert Schetterer]

Am 20.10.20 um 12:15 schrieb R. Diez:
> 
> First of all, thanks for your answer.
> 
>> What is the problem with having huge online mailboxes? Just choose a > 
>> good european provider that has encryption all the way through to 
>> their> storage platform.
> 
> We already have a European ISP with a standard e-mail server. I wanted 
> to keep our own mail server on premises, so that it is not exposed on 
> the Internet. The current server fetches (and removes) all e-mails from 
> the ISP. That has many advantages, for example, internal e-mail still 
> works in case of an Internet outage.
> 
> If I wanted to change the setup, I would have to start evaluating such 
> an "encryption all the way through to their storage platform" feature. I 
> am not sure that it is worth the effort. In any case, that sounds like a 
> limiting factor when choosing another ISP, in case the current one 
> starts making trouble.
> 
> 
>> I had exactly the same idea about migrating. You have to think twice > 
>> about moving emails around of users. They do not like it ;)
>  > [...]
> 
> I don't really want to do that, that's why I wrote "If I set a mailbox 
> size limit, users will have to delete old mails by themselves".
> 
> I do not know much about the legal aspects, but in case we need to keep 
> all e-mails for legal data retention requirements, I would like to store 
> those e-mails separately, so that if a user deletes it, the original 
> e-mail is still archived somewhere else.
> 
> That is why I mentioned the Postfix's BCC feature. The idea is that you 
> have a separate mailbox where a separate copy of all e-mails to and from 
> all users land. That is the separate mailbox where I wanted to 
> reorganise e-mails by date, in order to archive the e-mails in smaller 
> chunks on a yearly basis. Those e-mails do not need to be online after 
> all. Chances are, they will never be needed anyway.
> 
> 
>> I have created an 'archive' environment on a distributed filesystem, 
>> and it
>> takes me quite a lot of persuading to have people (or allow me) to move
>> messages from common Sent and Inbox mailboxes to the Archive namespace
>  > [...]
> 
> I am actually a newbie in mail service matters, but my guess is that 
> there is no amount of persuasion that could possibly help. You have to 
> set a hard limit per mailbox and let the users deal with it, don't you? 
> Otherwise, sooner or later the server will overload. Or I would need to 
> become a full-time e-mail server admin, which is not an option either!
> 
> I am actually a friend of having 2 backup disks that rotate, where one 
> is always physically off premises, and offline. But I wonder how I could 
> keep the backups encrypted and synchronised with 2 rotating disks. Maybe 
> Veracrypt + rsync.
> 
> I am hoping that the amount of big attachments in all incoming and 
> outgoing mail still fits in normal external USB 3.0 disks. Or at least a 
> few years' worth of it per disk. But I still would not want to have say 
> 1 TB of mail data online. That would make the VM unmanageable for 
> part-time sysadmins like me.
> 
> 
>> [@~]# mailbox-ls.sh testtest size
>  > [...]
>> I would not trust anyone else's programming with my
>> users email, you should also not.
> 
> I am not sure that I would trust my own e-mail server programming 
> abilities either. 8-)
> 
> If you have written such scripts, perhaps you could point me to some 
> example scripts that I could use as a starting point for such e-mail 
> reorganisation tasks?
> 
> 
>  > [...]
>> But when I migrate to mdbox this is not necessary anymore.
> 
> I am not sure that I would trust a file format where the indexes cannot 
> be rebuilt if they become corrupt. If I need an advanced format for 
> search performance reasons, I would probably consider an SQL-based 
> backend then.
> 
> 
>> I do not like the sound of "Postfix BCC feature", I use sendmail and I
>> can duplicate messages with that, without altering anything in them.
>  > [...]
> 
> I am actually not sure yet how to achieve the copying. I am still a 
> little confused anyway. 8-)
> 
> On the incoming side, I may not use Postfix at all, because Dovecot 
> actually needs to download the e-mails from the ISP mail server. I am 
> hoping that I can use a single "catch all" mailbox on the ISP. So I 
> would need to copy the incoming e-mails in another way.
> 
> On the outgoing side, anything sent (actually per SMTP relay) through 
> our internal mail server could be copied somehow with some BCC feature. 
> But if the user connects to the external ISP's SMTP servers directly, 
> then I cannot get a copy so easily. Maybe I need to force the users to 
> always use the internal mail server for sending.
> 
> In any case, let's say that the duplicate mails, stored somewhere else 
> for data retention purposes, get altered in some way, like some header 
> is added or changed in a predictable way. I am thinking of a header like 
> "BCC: dataretention at example.com". Why would that be a problem?
> 
> 
> Best regards,
>    rdiez

why not use archive solution
like

https://blog.sys4.de/mailarchiv-mit-dovecot-und-postfix-sortiert-nach-datum-mailadressen-und-ein-ausgehend-unterordnern-de.html

sort with sieve, its not exact what you search but should be enough to 
go on, you might consider a double fetch with i.e getmail

https://blog.sys4.de/abholdienst-fur-mail-de.html

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein