[Dovecot] Quota handling - opportunity for new Feature?
This initial proposal for a Feature Request is the result of my desire to implement quotas, but not have the attendant headaches that inevitably accompany its implementation.
Ralf Hildebrandt wrote:
I have to face it, my users are retards:
Is there any other kind of user? ;)
<snip>
Thus I need a feature in dovecot that will tell them via email:
Level1: "You ALMOST exceeded your quota, you're at 90% now" Level2: "You're very close to exceededin your quota, you're at 95% now" Level3: "Would you please clean up now? You're at 99% now"
What I'd *really* like to see implemented is something along the lines outlined below - but of course, this will depend entirely on whether or not Timo thinks it is doable - or desirable...
I know this would only be applicable if also using the dovecot LDA, because what I want to do requires cooperation at both the deliver and pop/imap ends...
Have two 'special' user-specific folders (by special, I mean like the Drafts, Sent, Templates folders) that dovecot controls: a) one, that the user has read-only access to and shows up in his folders list, that is used only for system-related messages, like over-quota notices - and maybe even user-specific log-type errors could be delivered here? b) and one (hidden) that the user does *not* have access to, to temporarily hold messages that come in that are unable to be delivered due to an over-quota condition
When user is over quota, have LDA deliver to folder b (yes, accept the message for delivery from the sending mta), and then generate an over-quota message that is delivered to folder a.
Optionally, a bounce could be generated to the sender, informing them that their message is being 'held in queue' or something to that effect, due to the recipient being over-quota.
- Once the user deletes enough mail to come back under quota, dovecot would then move messages from the 'over-quota' folder to his Inbox.
Ok, am willing to hear reasons how/why this is a terrible idea... :)
--
Best regards,
Charles Marcus I.T. Director Media Brokers International 678.514.6200 x224 678.514.6299 fax
On 5/22/07, Charles Marcus <CMarcus@media-brokers.com> wrote:
- Have two 'special' user-specific folders (by special, I mean like the Drafts, Sent, Templates folders) that dovecot controls:
These are just special in terms of the client side of things. The only special folder might be 'Trash' and various '.expunge' folders if you go that route. This can be seen by the large number of people with Sent, Sent-Mail, Drafts, Postponsed, Junk Mail (and all the variants Outlook has created on the phrase 'junk mail' over the years)
a) one, that the user has read-only access to and shows up in his folders list, that is used only for system-related messages, like over-quota notices - and maybe even user-specific log-type errors could be delivered here? b) and one (hidden) that the user does *not* have access to, to temporarily hold messages that come in that are unable to be delivered due to an over-quota condition
Your taking control of my email and not delivering it? In myexperience this would cause both a potential backlash and a people resending mail. Especially since people really only look at their INBOX for new mail or particular folders they have scripted. Using a 'poll all fodlers for new mail' I could see I had new mail in your 'a' folder. But that would be the only way.
- When user is over quota, have LDA deliver to folder b (yes, accept the message for delivery from the sending mta), and then generate an over-quota message that is delivered to folder a.
Optionally, a bounce could be generated to the sender, informing them that their message is being 'held in queue' or something to that effect, due to the recipient being over-quota.
- Once the user deletes enough mail to come back under quota, dovecot would then move messages from the 'over-quota' folder to his Inbox.
Ok, am willing to hear reasons how/why this is a terrible idea... :)
This is how it actually works right now if configured, only the 'B' folder is sender's /var/mail/spool and the 'a' folder is a intelligent client like thunderbird or squirrelmail with a indicator that goes nuts at a certain % full.
Off topic but you could also wrap your MTA (easily do this with exim and I am sure others) to 'deny message = "Achtung! your so over quota man"\ncondition = ${run{gimmequota.pl}}\nhosts = local_domains" if you want that instant 'halt your over quota' experience.
-- Gabriel Millerd
Gabriel Millerd wrote:
On 5/22/07, Charles Marcus <CMarcus@media-brokers.com> wrote:
- Have two 'special' user-specific folders (by special, I mean like the Drafts, Sent, Templates folders) that dovecot controls:
These are just special in terms of the client side of things.
Special, in that the user/client doesn't have full control of them, dovecot does.
This can be seen by the large number of people with Sent, Sent-Mail, Drafts, Postponsed, Junk Mail (and all the variants Outlook has created on the phrase 'junk mail' over the years)
I know - I'd love a way to define in dovecot to clean these kinds of things up too... maybe a list of folders to 'consolidate' into the specifid/preferred folder...
a) one, that the user has read-only access to and shows up in his folders list, that is used only for system-related messages, like over-quota notices - and maybe even user-specific log-type errors could be delivered here? b) and one (hidden) that the user does *not* have access to, to temporarily hold messages that come in that are unable to be delivered due to an over-quota condition
Your taking control of my email and not delivering it?
That is not what I said - it is delivered - the system admin simply prevents the end user from seeing it unless/until they rectify their over-quota condition.
In my experience this would cause both a potential backlash and a people resending mail.
I don't see how that would be a problem... the user will see *something* in their Inbox, and unless they are a *total* moron, they will actually *read* the message that is generated - especially since it will have a subject yelling at them in all caps that they are over quota.
Especially since people really only look at their INBOX for new mail or particular folders they have scripted. Using a 'poll all fodlers for new mail' I could see I had new mail in your 'a' folder. But that would be the only way.
I agree, and actually, I thought of a better way after I sent that...
I'm guessing that dovecot could over-ride the Quota limit to inject small system generated messages like over-quota - so, forget about the folder 'a'...
How about every time a message comes in while the user is over-quota, dovecot injects another notification saying so directly in the users Inbox - ie, 'Message with Subject: "blah" has been received from "blahblah", but you are #MB over quota - no new messages will be delivered to your Inbox until you delete some messages or move them to 'Local Folders'.
Of course, this message should be fully customizable using variables...
This is how it actually works right now if configured, only the 'B' folder is sender's /var/mail/spool
I'm not sure what you mean - most systems I have seen are configured to reject mail for users who are over quota. I know that postfix can be configured to soft-bounce, which is a part of what I'm talking about...
But I'd like a nice, clean, simple way for users to:
Know they are over-quota.
Never have mail rejected if they go over quota
Have a way of informing them when new messages have arrived but are being held pending rectification of their over-quota condition.
Provide a configurable way to send a notification to the sender that the email was accepted for delivery, but it being held pending rectification of an over quota condition.
This way, the user has only themselves to blame, and as long as this works reliably, I should never get a phone call about an over quota situation.
I'd prefer to handle this totally at the server level, and give a consistent experience regardless of the client used.
Off topic but you could also wrap your MTA (easily do this with exim and I am sure others) to 'deny message = "Achtung! your so over quota man"\ncondition = ${run{gimmequota.pl}}\nhosts = local_domains" if you want that instant 'halt your over quota' experience.
See above - I'm not doing this to be mean - I'm just trying to figure out a viable way of managing quotas that will be user friendly for the users, so that they won;t have to call me.
--
Best regards,
Charles
On 5/22/07, Charles Marcus <CMarcus@media-brokers.com> wrote:
Your taking control of my email and not delivering it?
That is not what I said - it is delivered - the system admin simply prevents the end user from seeing it unless/until they rectify their over-quota condition.
Unlikely the user will see the difference in deliver and received.
I am envisioning a situation like this "Sales Droid: sold send me that quote", "techdata rep: sending, hear back from you soon", <server: receipt received && receipt delivered>, mail sent to 'B' folder. At that moment I am getting called on where the mail is and I get to watch an episode of Sales Droid: CSI.
In my experience this would cause both a potential backlash and a people resending mail.
I don't see how that would be a problem... the user will see *something* in their Inbox, and unless they are a *total* moron, they will actually *read* the message that is generated - especially since it will have a subject yelling at them in all caps that they are over quota.
If SalesDroid is on the phone with someone and wants a document and doesn't get the document, but the sender gets receipt for the document temperatures rise, then the SalesDroid heads to gmail and emails themselves a message 'test message to my crappy mail server' and he gets it (because its small and fits under the radar temperatures rise further and the request to 'send it again' or what not occurs and the 'B' folder starts getting packed.
I'm guessing that dovecot could over-ride the Quota limit to inject small system generated messages like over-quota - so, forget about the folder 'a'...
Ummm, it can override the quota if its not a filesystem quota. If its a filesystem quota touching the disk gets tricky, even differences in types of buffered IO gets gross. But yes with a quota like Maildir++ you need a LDA that will honor the quota accounting system and a popmail and imap server that will as well. You could simple alter the dovecot LDA script to drop a small textfile in ~/Maildir/new/ manually. Tweaking it to not be obnoxious which is why I remove the previous alert and replace it with a new one.
How about every time a message comes in while the user is over-quota,
I would assume people would go spastic about the INBOX spam, not to mention quickly adding these messages to a filter even. You really need to delete the previous messages I think unless this is a daily or less infrequent thing I think.
I'm not sure what you mean - most systems I have seen are configured to reject mail for users who are over quota. I know that postfix can be configured to soft-bounce, which is a part of what I'm talking about...
I see, I guess with a Hotmail or what not this might work. I would rather 450 quota issues since they will get resolved soon. Just like greylisting you can just have the sender's email message try again later gracefully.
But I'd like a nice, clean, simple way for users to:
With pop-mail this was easy, because you have bulletins. Lots of issues with imap.
-- Gabriel Millerd
Your taking control of my email and not delivering it?
That is not what I said - it is delivered - the system admin simply prevents the end user from seeing it unless/until they rectify their over-quota condition.
Unlikely the user will see the difference in deliver and received.
Sure they will - since if one is delivered but the user is over-quota, they *will* get a message/notification that the server received their message (even telling them the subject *and* *who* *it* *is* *from* - why on earth would they bother telling the sender they haven't received it?
And if you come back and say - 'Hey, users are stoopid' - well, then allow me to retort - if they are *that* stupid, then they should be fired - pure and simple.
The fact is, if this were implemented, they would be much *less* likely to bother you, because they would have a status message in their Inbox telling them everything - *including* *how* *to* *fix* *it* *with* *your* *own* *custom* *instructions* - whereas, currently, what notification they get depends on the client.
I am envisioning a situation like this "Sales Droid: sold send me that quote", "techdata rep: sending, hear back from you soon", <server: receipt received && receipt delivered>, mail sent to 'B' folder. At that moment I am getting called on where the mail is and I get to watch an episode of Sales Droid: CSI.
No, you wouldn't - because the Sales Droid would have a notification in their Inbox that the message had been received, *but* wouldn't be delivered to their Inbox until they resolved their over-quota situation.
If they call me (as the sys admin), and I ask them the standard questions - did you get a notice in your Inbox about receiving the message but being over quota? yes? Did you deal with the over quota problem? no? well, what the hell are you calling me for then - deal with the problem!
Now, obviously, this would all have to be configurable (I'm guessing this would all live in the Quota Plugin - or maybe it would be an alternate Quota plugin) - off by default, etc...
In my experience this would cause both a potential backlash and a people resending mail.
Not at all - because the recipient *will* *know* they received the message, including the subject, the size, attachments, and who it is from.
I don't see how that would be a problem... the user will see *something* in their Inbox, and unless they are a *total* moron, they will actually *read* the message that is generated - especially since it will have a subject yelling at them in all caps that they are over quota.
If SalesDroid is on the phone with someone and wants a document and doesn't get the document, but the sender gets receipt for the document temperatures rise,
How exactly is my proposed method for dealing with the over quota situation *worse* than what it is currently??
then the SalesDroid heads to gmail and emails themselves a message 'test message to my crappy mail server' and he gets it (because its small and fits under the radar temperatures rise further and the request to 'send it again' or what not occurs and the 'B' folder starts getting packed.
Same question...
Again this can all be dealt with very simply, by the notification message. It can tell them precisely what wasn't delivered, *and* *why* - ie, because it is a large message and would put them over quota - complete with your custom instructions on how to fix the problem and get their message immediately.
I'm guessing that dovecot could over-ride the Quota limit to inject small system generated messages like over-quota - so, forget about the folder 'a'...
Ummm, it can override the quota if its not a filesystem quota.
Ok, so this would only work under certain conditions. Fine... lots of things work like that... you can only use maildir++ quotas with maildir
- fine...
How about every time a message comes in while the user is over-quota,
I would assume people would go spastic about the INBOX spam,
Ridiculous... don't punish me for your poor training/support or poor hiring criteria.
If you really do have such morons (and/or poor training/support) that they cannot learn to trust that if they get a message in their Inbox saying they won't get any new mail delivered until they deal with their quota problem - and then try to filter this system notice, rather than deal with the over quota problem - well, then you have a much worse problems than stupid users - you have an 'incompetent admin' problem.
not to mention quickly adding these messages to a filter even.
Oh, please - that is ridiculous. If they are saavy enough to create a filter to filter them out, they are saavy enough to understand the message *and* *deal* *with* *the* *problem*.
You really need to delete the previous messages I think unless this is a daily or less infrequent thing I think.
A very good idea, thanks - this message - or maybe a better word would be 'dovecot quota status message' - could be very special, and each subsequent one could replace the previous, and contain a list of all of the messages in the 'queue' (for lack of a better word).
Look - if you don't like the idea, fine. Personally, I don't care if you do or not... *I* do - but not even my opinion matters. I'd like to hear what Timo has to say. If he says this is a dumb/bad idea - or very difficult or impossible to code in such a way as to behave reliably - then fine, that will be the end of it.
--
Best regards,
Charles
On 5/22/2007 Charles Marcus (CMarcus@Media-Brokers.com) wrote:
Now, obviously, this would all have to be configurable (I'm guessing this would all live in the Quota Plugin - or maybe it would be an alternate Quota plugin) - off by default, etc...
Also, let me be clear... I am not the 'sys admin from hell'... what I am talking about are corporate situations where corporate policy *requires* quotas, and requires that they be enforced.
Personally I don't like quotas - but, I do understand the reasons for them. I have one client that has users who regularly email themselves 25+MB messages with pictures of their kids, new house, dog licking himself - and end up with 5+GB of email, most of which is crap like this. Multiply this user times 50, or 100, or 1000 - and backups become really unwieldy, just so Jim can keep 5GB of pictures of his dog in his email.
So, of *course* you would have rules in place to deal with the genuine situation where someone needs more storage, and you allow for this.
But, if this FR were implemented (or even implementable), at least the users would have consistent, meaningful instructions that are automatically provided if/when they go over quota. Your over-quota status message could also inform them of the procedure for requesting additional storage.
--
Best regards,
Charles
participants (2)
-
Charles Marcus
-
Gabriel Millerd