dbox format needs at least two tools:
Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir.
Ability to force index resync/rebuild in case it doesn't figure out itself that there's something wrong.
I'm starting to think about building a more generic mailutil tool, maybe something similar to what UW-IMAP has. So two questions:
What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
Is there any functionality you'd like it to do?
- Timo Sirainen <dovecot@dovecot.org>:
dbox format needs at least two tools:
Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir.
Ability to force index resync/rebuild in case it doesn't figure out itself that there's something wrong.
I'm starting to think about building a more generic mailutil tool, maybe something similar to what UW-IMAP has. So two questions:
- What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
Postfix has postconf for configuration issues, postqueue for queue management and postsuper to execute superuser commands.
doveconf? dovequeue? dovesuper? dovestore for storage (mbox, maildir, dbox, ...) handling?
- Is there any functionality you'd like it to do?
- Create mbox|maildir|dbox mailboxes
- Report mailbox size, number of messages, min.|avg.|max. message size
p@rick
-- state of mind Agentur für Kommunikation, Design und Softwareentwicklung
Franziskanerstraße 15 Telefon +49 89 45227227 81669 München Telefax +49 89 45227226
Amtsgericht München Partnerschaftsregister PR 563
On 3/24/2009 2:21 AM, Patrick Ben Koetter wrote:
I'm starting to think about building a more generic mailutil tool, maybe something similar to what UW-IMAP has. So two questions:
- What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
Postfix has postconf for configuration issues, postqueue for queue management and postsuper to execute superuser commands.
doveconf? dovequeue? dovesuper? dovestore for storage (mbox, maildir, dbox, ...) handling?
I like this idea, and had suggested a while back to change the 'dovecot -n' command to 'doveconf -n' to mirror the way postfix does this kind of thing. It makes sense, and why reinvent the wheel when there's already a model that works.
- Is there any functionality you'd like it to do?
- Create mbox|maildir|dbox mailboxes
- Report mailbox size, number of messages, min.|avg.|max. message size
- Be able to go through an *existing*, maybe even very large, mailstore and 'de-duplicate' the entire mail store - once 'Single Instance Storage' is supported of course.
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 24 Mar 2009, Patrick Ben Koetter wrote:
- Create mbox|maildir|dbox mailboxes
Well, I thought this, too, but because of the direct way to call dovecot-imap, most of this could be handled through IMAP interactively or script.
Maybe: to create mailboxes in a particular format in a place, which normally contains other mailboxes of another format.
- Report mailbox size, number of messages, min.|avg.|max. message size
In the end same as above - one could speak IMAP.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBScjE+3WSIuGy1ktrAQLMxggAgoF5xJi6jYGQRDVfQu3PmRM57iWyeFXv wHfVPYCYIC5foSu32nmX8MvZt9s0tSOY6+AmZL2o6yfqcg3KJp35zJLMXPMEaEFh ck0H63UDSyPJSC/Y3sao1FFdTXT62dh4m/CVkCwe7lcwr+tnJweo73LNNkGdO3dT RqKfOaPG7noE5PJKaTIg/tuPvsuQmYvQtCPoTcBBqVwb6qv3GsXbyWE1N0nzij5e Jgll8bM+5w8LXToAxv+W5ucINtTmJFv8KN+3iUdplpM54R6R7aMlYmUMFXOfbrB5 uClgHZ8cCBR6cPqjWS4rUwLoYSHg9Zj+Ci74e4MBzPEp3SLvFye4ew== =wZL/ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 23 Mar 2009, Timo Sirainen wrote:
- Ability to force index resync/rebuild in case it doesn't figure out itself that there's something wrong.
- What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
dovecot-tool
- Is there any functionality you'd like it to do?
- (re-)index selected mailboxes, maybe recursively, based on mailbox and/or possibly corrupted, existing indexes
- remove/sanitize (user-) flags from mailboxes
- acl evaluator / tester / "effective rights of user XYZ"
- hm, if you have a non-common storage, like dbox, you will require maintainance, like sanitize defective files (or duplicate/recover messages into another mailbox, so you can remove the old one)
- transform messages from one storage format into another, keeping all IMAP-related information, e.g. to find duplicate or unwanted messages across mailboxes/users.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBScjD93WSIuGy1ktrAQIkZwf+MNfFPK26K8la0gSGMFWnJNVvZOztaud8 L6mSts4j4AUgBwNIueOgoEBT0bw2dAuiez2IsHA5q33Cj0hlsnhgpxj3Z6Y2S4n7 8T7yFrqmNUbLp4rKCCVrKJENsg4xXqit6AzPVyMFQES5OxKbMSXX/GYafMQm7Lvr KW+NF0/d4f9GC5R/sGKfUKvwwrkMeXG0U2TdipUz8lMIs25pWN7lTpGd0nMPM0a5 VT6SKPgM1Ip7b4I8brY5TtlppME4FWuoK4OCnrAzNs6obGmeBcS4cZlCAoizK/ux JwoDTuNR4tsubG1aUvowtkg6At6pNUcRA34Z8g4uB/Ag/Qh7FizcQA== =bZRn -----END PGP SIGNATURE-----
Steffen Kaiser wrote:
- Is there any functionality you'd like it to do?
- (re-)index selected mailboxes, maybe recursively, based on mailbox and/or possibly corrupted, existing indexes
- remove/sanitize (user-) flags from mailboxes
I was going to suggest this ... something to at least go through and clean up the size flags on Maildir (which seems worthwhile, especially for people using quotas).
- acl evaluator / tester / "effective rights of user XYZ"
Certainly a +1 on that... as I'm about to implement ACLs and it'd be nice to be able to test I've got it right from a simple command line tool.
- transform messages from one storage format into another, keeping all IMAP-related information, e.g. to find duplicate or unwanted messages across mailboxes/users.
Additional to this would be (plugin permitting) to compress Maildir, or recompress+reindex mbox (Am I right in thinking compressed mbox can be appended to, but not modified? In which case recompressing could likely save some extra space.)
-- Curtis Maloney cmaloney@cardgate.net
On Mar 26, 2009, at 6:15 PM, Curtis Maloney wrote:
Steffen Kaiser wrote:
- Is there any functionality you'd like it to do?
- (re-)index selected mailboxes, maybe recursively, based on
mailbox and/or possibly corrupted, existing indexes
- remove/sanitize (user-) flags from mailboxes
I was going to suggest this ... something to at least go through and
clean up the size flags on Maildir (which seems worthwhile,
especially for people using quotas).
They are not flags, they are part of the base maildir filename. If you
change them, the message shows up as new. And in any case they should
never be wrong..
- transform messages from one storage format into another, keeping
all IMAP-related information, e.g. to find duplicate or unwanted
messages across mailboxes/users.Additional to this would be (plugin permitting) to compress Maildir,
or recompress+reindex mbox (Am I right in thinking compressed mbox
can be appended to, but not modified? In which case recompressing
could likely save some extra space.)
There is no compression code in Dovecot right now. But you can use
standard gzip tools to do such things to an existing mbox. Yes, you
can append more messages to it.
Timo Sirainen wrote:
On Mar 26, 2009, at 6:15 PM, Curtis Maloney wrote:
Steffen Kaiser wrote:
- Is there any functionality you'd like it to do?
- (re-)index selected mailboxes, maybe recursively, based on mailbox and/or possibly corrupted, existing indexes
- remove/sanitize (user-) flags from mailboxes
I was going to suggest this ... something to at least go through and clean up the size flags on Maildir (which seems worthwhile, especially for people using quotas).
They are not flags, they are part of the base maildir filename. If you change them, the message shows up as new. And in any case they should never be wrong..
That's what I meant... flags in the filename (sorry if my terminology is a little off.)
I agree they "should" never be wrong, but that's the point of a cleanup tool, isn't it? To fix cases where that "should" assertion fails.
What about someone moving from not having sizes/quotas, to having it? Wouldn't it be nice if they could update all of their files at once -- much like the option to re-index.
- transform messages from one storage format into another, keeping all IMAP-related information, e.g. to find duplicate or unwanted messages across mailboxes/users.
Additional to this would be (plugin permitting) to compress Maildir, or recompress+reindex mbox (Am I right in thinking compressed mbox can be appended to, but not modified? In which case recompressing could likely save some extra space.)
There is no compression code in Dovecot right now. But you can use standard gzip tools to do such things to an existing mbox. Yes, you can append more messages to it.
I seem to be having a lot of trouble communicating of late .. :(
Yes, I realise the compression is a plugin (hence the 'plugin permitting') ... I just thought that having an offline tool to help clean up would be nice. Perhaps some way to hook it into an expunge phase (seems most logical to me, at least)...
Ah well... you asked for input on what people would like in such a "dovetool"... that's mine :)
As for naming, I don't agree with "doveadm" and such, as this appears to more be a tool for managing mail stores, NOT for admining Dovecot itself -- what I've seen so far suggests it could be used on ANY mail store, not just one used by Dovecot.
-- Curtis Maloney cmaloney@cardgate.net
On Mar 26, 2009, at 6:35 PM, Curtis Maloney wrote:
I was going to suggest this ... something to at least go through
and clean up the size flags on Maildir (which seems worthwhile,
especially for people using quotas). They are not flags, they are part of the base maildir filename. If
you change them, the message shows up as new. And in any case they
should never be wrong..That's what I meant... flags in the filename (sorry if my
terminology is a little off.)I agree they "should" never be wrong, but that's the point of a
cleanup tool, isn't it? To fix cases where that "should" assertion
fails.
Maybe. But I've rarely heard of people having this problem, so I don't
really see it worth the trouble.
What about someone moving from not having sizes/quotas, to having
it? Wouldn't it be nice if they could update all of their files at
once -- much like the option to re-index.
No. Like I said, it gives messages new UIDs, which causes IMAP clients
to re-download the mails and POP3 clients that leave mail on server
will receive them as duplicates. Not a good idea.
One thing I could do about this is to make it save the S sizes to
dovecot-uidlist, similar to how W sizes are already saved. That'd just
require fixing also the Maildir++ quota code to use dovecot-uidlist.
Additional to this would be (plugin permitting) to compress
Maildir, or recompress+reindex mbox (Am I right in thinking
compressed mbox can be appended to, but not modified? In which
case recompressing could likely save some extra space.) There is no compression code in Dovecot right now. But you can use
standard gzip tools to do such things to an existing mbox. Yes, you
can append more messages to it.I seem to be having a lot of trouble communicating of late .. :(
Yes, I realise the compression is a plugin (hence the 'plugin
permitting') ... I just thought that having an offline tool to help
clean up would be nice. Perhaps some way to hook it into an expunge
phase (seems most logical to me, at least)...
I meant the zlib plugin can only do uncompression, not compression. So
compression would require all new functionality.
Ah well... you asked for input on what people would like in such a
"dovetool"... that's mine :)
I was hoping slightly less obscure uses :)
As for naming, I don't agree with "doveadm" and such, as this
appears to more be a tool for managing mail stores, NOT for admining
Dovecot itself -- what I've seen so far suggests it could be used on
ANY mail store, not just one used by Dovecot.
It might have some admining functions too. Like someone mentioned
verifying ACLs. Perhaps it might also be used to set them. And who
knows, maybe in future Dovecot has built-in features to create/delete/
manage users themselves too :)
On Mon, 2009-03-23 at 20:25 -0400, Timo Sirainen wrote:
- What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
dovestore, dovemail, doveadm, doveadmin, dovetool are probably usable names.
I'm beginning to think doveadm or doveadmin would be good. Because Cyrus had cyradm :)
On Tue, Mar 24, 2009 at 04:40:41PM -0400, Timo Sirainen wrote:
I'm beginning to think doveadm or doveadmin would be good. Because Cyrus had cyradm :)
I prefer dovecotadm or doveadm. It's an administrative command
and the name fits the purpose.
John-- "I'm sorry but our engineers do not have phones." As stated by a Network Solutions Customer Service representative when asked to be put through to an engineer.
"My other computer is your windows box." Ralf Hildebrandt <sxem> trying to play sturgeon while it's under attack is apparently not fun.
Hello Timo,
Timo Sirainen, 24.03.2009 (d.m.y):
On Mon, 2009-03-23 at 20:25 -0400, Timo Sirainen wrote:
- What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
dovestore, dovemail, doveadm, doveadmin, dovetool are probably usable names.
I'm beginning to think doveadm or doveadmin would be good. Because Cyrus had cyradm :)
I'd favor "dcutil" or "dctool". And what about (even shorter) "dool"? ;-)
Gruss/Regards, Christian Schmidt
-- You will engage in a profitable business activity.
On Mon, 2009-03-23 at 20:25 -0400, Timo Sirainen wrote:
- Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir.
Another important question is what should this functionality be called? UW-IMAP calls it "burping".
Words by Timo Sirainen [Wed, Mar 25, 2009 at 02:33:31PM -0400]:
On Mon, 2009-03-23 at 20:25 -0400, Timo Sirainen wrote:
- Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir.
Another important question is what should this functionality be called? UW-IMAP calls it "burping".
Funny name. It should be a funny name :)
-- Jose Celestino | http://japc.uncovering.org/files/japc-pgpkey.asc
"One man’s theology is another man’s belly laugh." -- Robert A. Heinlein
Timo Sirainen wrote:
dbox format needs at least two tools:
Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir.
Ability to force index resync/rebuild in case it doesn't figure out itself that there's something wrong.
I'm starting to think about building a more generic mailutil tool, maybe something similar to what UW-IMAP has. So two questions:
What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
Is there any functionality you'd like it to do?
Sounds like dbox is what the tool is mainly used for, I know dbox format is only used by dovecot, but still:
dboxutils: 1: dboxclean or dboxexpunge 2: dboxrebuild/dboxresync/dboxrepair
:P
-- Asbjørn Sannes
On 24/03/2009 11:25 AM, Timo Sirainen wrote:
dbox format needs at least two tools:
Expunging only marks the message having refcount=0. The actual disk space is freed only after calling a tool pointing to the user's dbox dir.
Ability to force index resync/rebuild in case it doesn't figure out itself that there's something wrong.
I'm starting to think about building a more generic mailutil tool, maybe something similar to what UW-IMAP has. So two questions:
What should the tool be called? Probably not mailutil to avoid confusion. doveutil? dovemail? dovetool? mailtool?
Is there any functionality you'd like it to do?
Did anything like this end up being implemented? We are in the middle of migrating a bunch of people from local storage (pop) to server side storage using imap. It would be really useful to be able to tell dovecot to index a particular maildir after we have copied it onto the server to save our users opening up their shiny new mail client and clicking on all their folders and loading up the server indexing a hundred maildirs at once.
-- Thanks, Phill Macey (CiSRA IT Services)
participants (10)
-
Asbjørn Sannes
-
Charles Marcus
-
Christian Schmidt
-
Curtis Maloney
-
John R. Dennison
-
Jose Celestino
-
Patrick Ben Koetter
-
Phillip Macey
-
Steffen Kaiser
-
Timo Sirainen