[Dovecot] keep users from deleting email
I looked on the dovecot website, but didn't find an answer. With dovecot 1.0, is there a way to keep users from deleting their email? So that when they click the delete button on their email client, nothing happens/dovecot refuses to delete email, etc?
On 4/14/2008 9:42 AM, Adam Williams wrote:
I looked on the dovecot website, but didn't find an answer. With dovecot 1.0, is there a way to keep users from deleting their email? So that when they click the delete button on their email client, nothing happens/dovecot refuses to delete email, etc?
I was thinking about a possible plugin - call it maybe 'fake-delete or something - that would move all messages that a user deletes to a hidden folder in their maildir, for admin purposes... you could also use the expire plugin to keep this from growing indefinitely.
But to prevent them from deleting them altogether? Why?
If you just want a read-only maildir, just do that - so only admins can delete messages from protected accounts...
--
Best regards,
Charles
Charles Marcus wrote:
I was thinking about a possible plugin - call it maybe 'fake-delete or something - that would move all messages that a user deletes to a hidden folder in their maildir, for admin purposes... you could also use the expire plugin to keep this from growing indefinitely.
But to prevent them from deleting them altogether? Why?
If you just want a read-only maildir, just do that - so only admins can delete messages from protected accounts... Management wants the users to be able to use the email like normal (to read as normal, to file into mailboxes, etc) but not be able to delete any email, ever, for archival/legal purposes. Also, I'm using the wu-imap mbox format and not the Maildir format for keeping email, because a year ago I migrated away from wu-imap to dovecot.
On 4/14/2008, Adam Williams (awilliam@mdah.state.ms.us) wrote:
Management wants the users to be able to use the email like normal (to read as normal, to file into mailboxes, etc) but not be able to delete any email, ever, for archival/legal purposes.
Then you are using the wrong tool.
For legal purposes, your message archives should be completely separate from your normal mail store.
Set up a parallel delivery system for your archiver.
--
Best regards,
Charles
Charles Marcus wrote:
Then you are using the wrong tool.
For legal purposes, your message archives should be completely separate from your normal mail store.
Set up a parallel delivery system for your archiver.
Do you mean like, Postfix's "always_bcc = awilliam@mdah.state.ms.us" option? But, then I'm doubling my data. One copy is the user's email, and one copy for always_bcc. Then I have twice the data to back up, more CPU cycles to compress it to tape, etc...
At 9:45 AM -0500 4/14/08, Adam Williams imposed structure on a stream of electrons, yielding:
Charles Marcus wrote:
Then you are using the wrong tool.
For legal purposes, your message archives should be completely separate from your normal mail store.
Set up a parallel delivery system for your archiver.
Do you mean like, Postfix's "always_bcc = awilliam@mdah.state.ms.us" option? But, then I'm doubling my data. One copy is the user's email, and one copy for always_bcc. Then I have twice the data to back up, more CPU cycles to compress it to tape, etc...
If you use something like Postfix's recipient_bcc_maps you may not need to back up the live user-facing mailboxes at all, because you can have an archive mailbox for each user instead of the big shared dump you get with always_bcc.
Bill Cole
bill@scconsult.com
On 4/14/2008, Adam Williams (awilliam@mdah.state.ms.us) wrote:
Do you mean like, Postfix's "always_bcc = awilliam@mdah.state.ms.us" option? But, then I'm doubling my data. One copy is the user's email, and one copy for always_bcc. Then I have twice the data to back up, more CPU cycles to compress it to tape, etc...
Yep... but its the only legally recognizable way of insuring you are actually archiving your email.
If you are archiving because you are legally *required* to do so, then I don't think such a scheme would satisfy that requirement - but I could be wrong.
I'm glad I'm not in a regulated industry where we *have* to archive.
--
Best regards,
Charles
Adam Williams wrote:
Charles Marcus wrote:
Then you are using the wrong tool.
For legal purposes, your message archives should be completely separate from your normal mail store.
Set up a parallel delivery system for your archiver.
Do you mean like, Postfix's "always_bcc = awilliam@mdah.state.ms.us" option? But, then I'm doubling my data. One copy is the user's email, and one copy for always_bcc. Then I have twice the data to back up, more CPU cycles to compress it to tape, etc...
If you still want to do it this way, upon source inspection it looks like you can accomplish it by making imap_expunge function (src/imap/imap-expunge.c) a noop, that is, returning always true. If it works for read-only mailboxes it should do for this, but haven't tested myself :) BTW I'm sure it can also be done as a plugin, but I'm not familiar with the plugin api at all.
Bright side is that if expunge is a noop, users stop seeing the message as it gets marked as deleted but it's left there.
Anyway as others have pointed out, I'd look at doing this outside the imap server. It's too easy to get something like this disabled by mistake (deploy an unpatched binary) and do unnoticed.
Regards,
Angel Marin http://anmar.eu.org/
At 8:58 AM -0500 4/14/08, Adam Williams wrote:
Charles Marcus wrote:
I was thinking about a possible plugin - call it maybe 'fake-delete or something - that would move all messages that a user deletes to a hidden folder in their maildir, for admin purposes... you could also use the expire plugin to keep this from growing indefinitely.
But to prevent them from deleting them altogether? Why?
If you just want a read-only maildir, just do that - so only admins can delete messages from protected accounts... Management wants the users to be able to use the email like normal (to read as normal, to file into mailboxes, etc) but not be able to delete any email, ever, for archival/legal purposes.
Just so others don't read that as some sort of generally relevant good idea, it should be noted that as a government agency you have very special archival/legal concerns, and that they may be even more unusual as an agency focused on historical information. Acme's Widgets Inc. would not be well-advised to use the Mississippi Dept. of Archives and History as a role model in email preservation.
I still would think that the particular approach you describe is looking at the archival requirements in the wrong place. Rather than interfering with how users work with their mail, you would probably be better off looking at the delivery process and having it create your permanent archive rather than treating the user-facing mailbox as the archive.
Also, I'm using the wu-imap mbox format and not the Maildir format for keeping email, because a year ago I migrated away from wu-imap to dovecot.
That eliminates the Lazy Expunge option...
Bill Cole
bill@scconsult.com
At 8:42 AM -0500 4/14/08, Adam Williams imposed structure on a stream of electrons, yielding:
I looked on the dovecot website, but didn't find an answer. With dovecot 1.0, is there a way to keep users from deleting their email? So that when they click the delete button on their email client, nothing happens/dovecot refuses to delete email, etc?
Presumably you're users are all using IMAP, since the question doesn't really make sense for POP users, whose view of mail is entirely local to their machines, not the server.
I'd argue that having the sort of in-your-face dysfunction you describe is probably not the best approach unless user antagonism is one of your goals. You would probably be better off making a user's deletion into a server-side hiding/archiving. The "Lazy Expunge" plugin can do that. http://wiki.dovecot.org/Plugins/Lazyexpunge
--
Bill Cole
bill@scconsult.com
On 4/14/2008, Bill Cole (dovecot-20061108@billmail.scconsult.com) wrote:
You would probably be better off making a user's deletion into a server-side hiding/archiving. The "Lazy Expunge" plugin can do that. http://wiki.dovecot.org/Plugins/Lazyexpunge
Oh, right... forgot about that one...
Dovecot 'just' rocks... :)
--
Best regards,
Charles
Bill Cole wrote:
Presumably you're users are all using IMAP, since the question doesn't really make sense for POP users, whose view of mail is entirely local to their machines, not the server.
I'd argue that having the sort of in-your-face dysfunction you describe is probably not the best approach unless user antagonism is one of your goals. You would probably be better off making a user's deletion into a server-side hiding/archiving. The "Lazy Expunge" plugin can do that. http://wiki.dovecot.org/Plugins/Lazyexpunge Thanks, I agree with you. However, I just do what management tells me to do, and we're trying to get Sarbanes-Oxley Act of 2002 compliant with our data. Lazy Expunge only operates on Maildir format, and while dovecot's website has instructions for converting from mbox to Maildir, mbox works pretty good for us, and I don't want to fix what isn't broken.
At 9:36 AM -0500 4/14/08, Adam Williams wrote:
Bill Cole wrote:
Presumably you're users are all using IMAP, since the question doesn't really make sense for POP users, whose view of mail is entirely local to their machines, not the server.
I'd argue that having the sort of in-your-face dysfunction you describe is probably not the best approach unless user antagonism is one of your goals. You would probably be better off making a user's deletion into a server-side hiding/archiving. The "Lazy Expunge" plugin can do that. http://wiki.dovecot.org/Plugins/Lazyexpunge Thanks, I agree with you. However, I just do what management tells me to do, and we're trying to get Sarbanes-Oxley Act of 2002 compliant with our data.
Um, really?
I've had a little experience with SOx, HIPAA, GLBA, and Federal E-Discovery compliance projects, and I've never heard that SOx applied at all to state agencies or that it requires anyone to archive all email forever. In fact, doing so as a matter of normal policy may be a very bad idea under the E-Discovery rules. I'm certainly no lawyer, but your management may want to find better ones than they seem to have...
Lazy Expunge only operates on Maildir format, and while dovecot's website has instructions for converting from mbox to Maildir, mbox works pretty good for us, and I don't want to fix what isn't broken.
You definitely need to be aware of the fact that one of the downsides of mbox is performance and resource demands as the mbox files grow.
Bill Cole
bill@scconsult.com
Bill Cole wrote:
Um, really?
I've had a little experience with SOx, HIPAA, GLBA, and Federal E-Discovery compliance projects, and I've never heard that SOx applied at all to state agencies or that it requires anyone to archive all email forever. In fact, doing so as a matter of normal policy may be a very bad idea under the E-Discovery rules. I'm certainly no lawyer, but your management may want to find better ones than they seem to have...
Well it doesn't, but management wants us to archive all electronic data forever in a fashion that is compliant to that law.
You definitely need to be aware of the fact that one of the downsides of mbox is performance and resource demands as the mbox files grow.
What do you consider a large mbox? We have users with single mbox files of 3G and their mail loads up fine. We are using Seamonkey's email client and it makes a local index file of all the message to/from/subject/date/etc so it loads mail very quickly, pretty much instantaneous. One thing I don't like about Maildir is that it keeps each message as a seperate file, so you'll end up with directories with 20k+ files and run into glob problems, and with ext3, you have all these sub 4kb messages still taking up a 4k block on the disk, wasting disk space.
You definitely need to be aware of the fact that one of the downsides of mbox is performance and resource demands as the mbox files grow. What do you consider a large mbox? We have users with single mbox files of 3G and their mail loads up fine. We are using Seamonkey's email client and it makes a local index file of all the message to/from/subject/date/etc so it loads mail very quickly, pretty much instantaneous. One thing I don't like about Maildir is that it keeps each message as a seperate file, so you'll end up with directories with 20k+ files and run into glob problems, and with ext3, you have all these sub 4kb messages still taking up a 4k block on the disk, wasting disk space.
Indeed, but if your mbox gets corrupted then you loose 3G of email... If you delete the first message in the 3G store then you need to rewrite the whole store to free up the space (Dovecot has some optimisations to simply mark messages dead without removing them from the mbox.)
Maildir has different performance characteristics from Mbox and it's best to be aware of the performance implications of both before choosing one. For ext3 it seems like a good idea to enable dir_index option on the filesystem though. (I have quite a few folders with 60K+ messages on ext3)
It may also be worth paying Timo to look again at single instance storage and see if he can code a solution for you? This way you can store multiple copies of messages without penalty...?
I would have thought that the best option would be to use something like BCC recipient maps into an MBox folder (for the archive), then look at either the gzip plugin for dovecot or simply bzip the folders up one a week/month (etc) and rotate them out for a new folder. Mbox is probably good for a write only archive folder and will zip up nicely and can be burned off to some permanent archive media periodically.
Good luck
Ed W
At 10:28 AM -0500 4/14/08, Adam Williams imposed structure on a stream of electrons, yielding:
Bill Cole wrote:
Um, really?
I've had a little experience with SOx, HIPAA, GLBA, and Federal E-Discovery compliance projects, and I've never heard that SOx applied at all to state agencies or that it requires anyone to archive all email forever. In fact, doing so as a matter of normal policy may be a very bad idea under the E-Discovery rules. I'm certainly no lawyer, but your management may want to find better ones than they seem to have...
Well it doesn't, but management wants us to archive all electronic data forever in a fashion that is compliant to that law.
You definitely need to be aware of the fact that one of the downsides of mbox is performance and resource demands as the mbox files grow.
What do you consider a large mbox? We have users with single mbox files of 3G and their mail loads up fine. We are using Seamonkey's email client and it makes a local index file of all the message to/from/subject/date/etc so it loads mail very quickly, pretty much instantaneous. One thing I don't like about Maildir is that it keeps each message as a seperate file, so you'll end up with directories with 20k+ files and run into glob problems, and with ext3, you have all these sub 4kb messages still taking up a 4k block on the disk, wasting disk space.
Well, I suppose that if you never delete any messages you avoid the worst mbox performance problems, since you never need to re-write the whole file. Between Dovecot's indexing and client-side indexing, mboxes that only ever get written to by adding more messages might not be such a big deal.
The interesting issue would then be the size of the Dovecot index files. I don't know if they are designed to work well when all of your users have tens of thousands of messages in their index. I expect you'll want a lot of RAM...
Bill Cole
bill@scconsult.com
On Apr 17, 2008, at 6:13 PM, Bill Cole wrote:
The interesting issue would then be the size of the Dovecot index
files. I don't know if they are designed to work well when all of
your users have tens of thousands of messages in their index. I
expect you'll want a lot of RAM...
I've some future plans about making them work better, but I don't
think there should be real problems even now. Cache files are the ones
that take most of the disk space (and memory), but clients typically
read only small parts of it.
On Mon, 2008-04-14 at 08:42 -0500, Adam Williams wrote:
I looked on the dovecot website, but didn't find an answer. With dovecot 1.0, is there a way to keep users from deleting their email? So that when they click the delete button on their email client, nothing happens/dovecot refuses to delete email, etc?
For specific mailboxes you can do this with ACL plugin. Or you'd probably also get quite far by creating a .DEFAULT global ACL. It would apply to all root-level mailboxes. I should add support for hierarchical ACL rules some day..
Anyway, http://wiki.dovecot.org/ACL describes how it currently works.
participants (6)
-
Adam Williams
-
Angel Marin
-
Bill Cole
-
Charles Marcus
-
Ed W
-
Timo Sirainen