On 25 Jun 2012, at 08:52, Kaya Saman wrote:

I've previously **only** ever worked with Maildir but I was told that there are some benefits to mbox which is why I decided to try to use it here!

I used mbox before Dovecot, but once I found Maildir, I never looked back.

I've not come up with any significant advantages of mbox that count for much in my experiences and installations . . Would be interested to hear of suggested advantages that I might have overlooked or know of reasons why they're not an issue . . not sure how much the list wants to hear, but feel free to email me direct if you want.

J.

On 25 Jun 2012, at 10:44, Charles Marcus wrote:

On 2012-06-25 3:58 AM, J E Lyon role.Dovecot-Readers@JLAssocs.com wrote:

...

I've not come up with any significant advantages of mbox that count for much in my experiences and installations . . Would be interested to hear of suggested advantages that I might have overlooked

One major advantage of mbox (and now mdbox) over maildir is the time it takes to back things up for larger mail stores.

It takes much less time to compare a single mbox file that contains 20,000 messages (and rsync only the changed bits) than it does to compare read/compare 20,000 individual files (maildir)...

I too like maildir, but am seriously considering implementing a solution where older mail is automatically archived to slower/cheaper SATA III based storage using mdbox format.

Very interesting. I use "rdiff-backup" as a cronjob in the wee hours (, so the fact it takes a while to work out what to incrementally save, doesn't matter to me -- but the point is that I know it's an issue and have addressed it that way.

I too have contemplated some automated archiving of older mail . . it's getting closer to needing to be addressed at some point soon I think.

J.

On Mon, Jun 25, 2012 at 8:37 AM, Trever L. Adams trever@middleearth.sapphiresunday.org wrote:

...

On 06/25/2012 01:20 AM, Kaya Saman wrote:

Now what I would like to know is, which is better for "virtual hosting" Maildir or mbox?

Basically my requirement is that I would like to separate users via either individual folders and then put each user's mbox or Maildir in the created directory, or simply name each mbox or Maildir according to the user name.

First up is this possible?

Secondly, how would I go about doing it?

Sorry, I missed this at first. It is quite simple. I don't store it in passwd or any other place, since you are doing vmail, you might find this easiest:

in /etc/dovecot/conf.d/10-mail.conf (where mail_location is):

mail_home = /home/vmail/%Ld/%Ln mail_location = maildir:~/Maildir

in /etc/dovecot/conf.d/10-mail.conf (after paragraph “# System user and group used to access mails...”):

mail_uid=vmail mail_gid=vmail

Obviously, vmail may not be your user for vmail. Also, some of my notes may no longer be accurate for location, just find where it exists and edit.

I hope this helps. Of course, this is a Maildir setup. mbox is probably very similar, but I have had too many mbox style mail queues go south losing all of the mail (or more than one would like), so I do Maildir, even though it isn't necessarily the best use of disk space.

Trever

"I do not fear computers. I fear the lack of them." -- Isaac Asimov Thanks for the responses!

Sorry if I reply to every single one in this email however, I am using Gmail's awful Web UI so I don't really have much control over what I'm doing......

To start with the reason I'm not using LDAP is because I couldn't find enough information on how to set it up! I did post here a couple of times but got no responses...... so I figured it was something that people either didn't know or found trivial. Sorry, I missed these. I have been busy. Tim and others were very kind and helped me figure things out. I do not have any web sites where I can

On 06/25/2012 01:52 AM, Kaya Saman wrote: post things, so I will give an overview here and if you need help, either follow up privately or here.

I'll take a look at the above config for Maildir format as briefly playing around with mbox it seems that folders on the / root (parent) IMAP directory are stored separately. It may be better if everything got stored under the Maildir heading.... I've previously **only** ever worked with Maildir but I was told that there are some benefits to mbox which is why I decided to try to use it here!

Regards,

Kaya

The only draw back I see with maildir is one file per message. This makes it resilient to corruption that mbox sees (if a message gets corrupted, you erase one message and that corruption won't propagate even if you leave it in place). In many setups this also leads to MUCH faster system. On unix systems this doesn't just waste disk space, it could lead to inode (or whatever your *nix of choice calls it) depletion. I haven't yet seen this in my setups. I archive most of my personal mail, so I have at least some of my mail going back to 1998, I think.

Kerberos Directions:

Microsoft suggests on linux systems that you create an account (separate from the HOST account) and add the appropriate user principal names. There are pros and cons to this, their way is a and doing it as part of the host account is b.

a) On S4 dc (replace MAILSERVER_HOST and MAILSERVER_FQDN with host and host.example.org, in lowercase as I use the caps only to help me see what I need to change, respectively as fits your domain - i.e. not example.org and create some long random password and put it wherever you see $RANDOMPASSWORD) :

/usr/local/samba/sbin/samba-tool newuser mail-MAILSERVER_HOST /usr/local/samba/sbin/samba-tool spn add imap/MAILSERVER_FQDN mail-MAILSERVER_HOST /usr/local/samba/sbin/samba-tool spn add smtp/MAILSERVER_FQDN mail-MAILSERVER_HOST /root/samba-master/source4/scripting/bin/ktpass.sh --out /tmp/mail.keytab --princ smtp/MAILSERVER_FQDN --path-to-ldbsearch /usr/local/samba/bin/ --pass $RANDOMPASSWORD /root/samba-master/source4/scripting/bin/ktpass.sh --out /tmp/mail.keytab --princ imap/MAILSERVER_FQDN --path-to-ldbsearch /usr/local/samba/bin/ --pass $RANDOMPASSWORD /root/samba-master/source4/scripting/bin/ktpass.sh --out /tmp/mail.keytab --princ mail-MAILSERVER_HOST --path-to-ldbsearch /usr/local/samba/bin/ --pass $RANDOMPASSWORD

Move the mail.keytab to dovecot's main configuration directory on dovecot server (/etc/dovecot here). Then do the appropriate version of:

chmod 640 /etc/dovecot/mail.keytab chown dovecot.dovenull /etc/dovecot/mail.keytab

b) Make sure your local samba setup is joined to the domain. Make sure it writes an appropriate krb5.keytab (/etc/krb5.keytab in my setup) as part of its password management, etc.

net ads keytab add smtp/mail_server_fqdn net ads keytab add imap/mail_server_fqdn

You may have to edit the sam.ldb on your S4 server as many times S3 doesn't create the principals ( /usr/local/samba/bin/ldbedit -H /usr/local/samba/private/sam.ldb sAMAccountName=mailserverhostname$ should do the trick and add userPrincipalName so that it has imap/MAILSERVER_FQDN and smtp/MAILSERVER_FQDN, each being its own userPrincipalName, this should give the machine account 3 userPrincipalName lines)

then do the following modified for your samba krb5.keytab location (the following is how to set the extended posix ACLs on Linux, I don't know what it would be for FreeBSD, this adds read writes to dovecot user on the file krb5.keytab):

setfacl -m u:dovecot:r krb5.keytab

NOTE: For ldap access dovecot needs access to the krb5.keytab as I haven't figured out why, but Windows (including S4) AD doesn't like the a) method principals acting in some of the ways they need to. So, I just go with method b.

Change the following or insert them into your dovecot setup modifying to fit your setup (/etc/dovecot/conf.d/10-auth.conf for me) :

auth_realms = DOMAIN_FQDN auth_gssapi_hostname = HOST_FQDN auth_krb5_keytab = /etc/dovecot/mail.keytab (this is method a, b would be /etc/krb5.keytab) auth_mechanisms = gssapi gss-spnego login plain

If you are using postfix anywhere, you can use dovecot as the lda (avoid messing with trying to make it deliver to the right directories, etc.) and use dovecot for the auth. Doing the later makes things overlap perfectly for auth too.

LDAP (simple if you used method b, method a always gave me trouble - if people reading this know how to make method a work, I would love to read it myself as it may be more secure according to Microsoft):

Create a userdb setup that reads (I do this in a file called /etc/dovecot/conf.d/auth-vmail.conf.ext):

userdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext }

Then create that dovecot-ldap.conf.ext file (again, make sure your replace DOMAIN_FQDN with example.org, or whatever it is in your setup, replacing example.org as well):

hosts = DOMAIN_FQDN base = dc=example,dc=org ldap_version = 3 user_attrs = userPrincipalName=user user_filter = (&(objectClass=person)(|(mail=%u)(sAMAccountName=%u)(userPrincipalName=%u))) dn = mail-MAIL_HOST@DOMAIN_FQDN sasl_bind = yes sasl_mech = GSSAPI sasl_realm = DOMAIN_FQDN sasl_authz_id = mail-MAIL_HOST@DOMAIN_FQDN (this is for method a, method b you would think would be the machine$ account, but I found it works better without this line with method b) # For using doveadm -A: iterate_attrs = userPrincipalName=user iterate_filter = (objectClass=person)

Finally, you need to do a cronjob that will keep a credential cache for the machine account around for dovecot to use to do ldap:

02 03 */2 * * /usr/bin/kinit -l 10d -k MAIL_HOST$ -c /etc/dovecot/krb5.cc && /bin/chown dovecot:dovecot /etc/dovecot/krb5.cc

03 * * * * /usr/bin/kinit -c /etc/dovecot/krb5.cc -R && /bin/chown dovecot:dovecot /etc/dovecot/krb5.cc

Does the trick for me. The cronjob should be for root, hence the need for the chown. It may work as dovecot. I cannot remember if it does or not. If it does, have the cronjob be for dovecot's user.

One side effect of the above ldap.conf.ext file, the mail entry (this is the mail shown in AD Users and Computers if you edit a user) becomes an alias. So, if you wish to hide users logins or have an additional email in the same domain, use the mail field (you can edit it similar to how you added the userPrincipalName above). to hide user logins, do the client setup so that it uses the alias as the from address and account name stuff while using the real login for all the login stuff.

Unless I missed a step, you just need to tell Outlook, Thunderbird, etc. to do GSSAPI or SPNEGO with GSSAPI (whatever Outlook calls it).

If you use postfix with dovecot lda, the aliasing stuff works. If you don't, you will have to setup some things for postfix to do the aliasing the same way. I have such ldap-users and ldap-alias.cf files. I am not sure they work as I don't remember if I ever tested them before moving to dovecot lda. They should work with method b, so long as you also add the postfix user to the read list. I find dovecot lda with sieve gives me everything I need/want, so I won't test these out. They are available to anyone upon request.

Nothing here interferes with pam_krb5 stuff you mentioned. As I said, I use it myself for devices or setups that can't/don't do krb5.

There you have it. I hope this helps you and others.

Trever

P.S. Yes, I know you said this is not an MTA box, just IMAP. I keep mentioning postfix as in my work, it works best for me and it is nice to have them work very well together.

-- "Noise proves nothing. Often a hen who has merely laid an egg cackles as if she laid an asteroid." -- Mark Twain

On 06/26/2012 07:11 AM, Wojciech Puchar wrote:

...

The only draw back I see with maildir is one file per message. This

it is mostly adventage. Agreed.

...

makes it resilient to corruption that mbox sees (if a message gets corrupted, you erase one message and that corruption won't propagate even if you leave it in place). In many setups this also leads to MUCH faster system. On unix systems this doesn't just waste disk space, it

even with 32kB block/4kB fragment filesystem under FreeBSD which is my common setup, it isn't that a problem. i just checked one of my users folder - 2.3GB in 8500 files. the average is 270 kilobytes per mail.

checked few others and it looks similar.

dovecot's own storage system can do something in between - packing smallest messages by a few in one file.

...

could lead to inode (or whatever your *nix of choice calls it) depletion. you decide how much inode you need while creating filesystem on every unix system, except filesystems where it is allocated on demand. Yes, as I noted, I haven't seen this. But it could be an annoyance depending on how things were created and when. I don't believe all file systems can do allocation on demand. I don't know.

as of latter discussion about what microsoft recommends with linux (being of course expert of everything) - i would keep silent.

The only reason I know what they recommend is it came up on several sites that described how to setup the service principals. I read something recently on Samba lists that explains why this may be their recommendation. The funny thing is, it really isn't any different than on their systems unless they think that because it is their system the keytab is some how miraculously going to stay more secure than it would on other systems.

Sorry if I seemed like I was claiming to be some super expert. I just had a lot of help to pull things together. If he was struggling to find things, I would like to help.

Trever

"Fairy tales are more than true; not because they tell us that dragons exist, but because they tell us that dragons can be beaten." -- G.K. Chesterton

On 06/25/2012 08:37 AM, Trever L. Adams wrote:

On 06/25/2012 01:20 AM, Kaya Saman wrote:

...

Now what I would like to know is, which is better for "virtual hosting" Maildir or mbox?

Basically my requirement is that I would like to separate users via either individual folders and then put each user's mbox or Maildir in the created directory, or simply name each mbox or Maildir according to the user name.

First up is this possible?

Secondly, how would I go about doing it?

Sorry, I missed this at first. It is quite simple. I don't store it in passwd or any other place, since you are doing vmail, you might find this easiest:

in /etc/dovecot/conf.d/10-mail.conf (where mail_location is):

mail_home = /home/vmail/%Ld/%Ln mail_location = maildir:~/Maildir

in /etc/dovecot/conf.d/10-mail.conf (after paragraph “# System user and group used to access mails...”):

mail_uid=vmail mail_gid=vmail Obviously, vmail may not be your user for vmail. Also, some of my notes may no longer be accurate for location, just find where it exists and edit.

I hope this helps. Of course, this is a Maildir setup. mbox is probably very similar, but I have had too many mbox style mail queues go south losing all of the mail (or more than one would like), so I do Maildir, even though it isn't necessarily the best use of disk space.

Trever

"I do not fear computers. I fear the lack of them." -- Isaac Asimov Hi, I'm just responding as the OP to say that the above was what I was looking for!

Thanks Trever :-)

Everything is setup and working fine now.

Though responding quite late and of course having read through the latest messages within the thread I don't feel that my users will notice any difference between mbox or mdbox and Maildir format, speedwise.

The reasoning behind this is that my end users unfortunately are all using M$ Outlook which is absolute garbage! FULL STOP!

Comparing the IMAP capability speeds between Thunderbird and Outlook linking to my server yielded that I was able to get around 150Mbps transfer rate using T-Bird while Outlook only managed a few 100's of kbps. I think it's because 2010 relies heavily on PST's (whatever they are....) and the fact it is ultimately M$ also so it's basically built by nincompoops to be sold at hideous prices and even higher tech-support prices.

In all fairness to Outlook I did manage to get a pathetic ~2Mvbps tops of transfer...... :-S

Luckily I'm the only one using T-Bird or Alpine so am fine :-)

Can't send any mail though as need to go through Exchange - there's no winning in the corporate world :-(

Regards,

Kaya

On Mon, Jun 25, 2012 at 10:31 AM, Charles Marcus CMarcus@media-brokers.com wrote:

On 2012-06-25 3:20 AM, Kaya Saman kayasaman@gmail.com wrote:

...

# cat dovecot.conf # v1.2+: auth_use_winbind = yes

Please always only provide output of doveconf -n, not copy/pastes from the config files.

This proves (to yourself and everyone else) that you are using the config that dovecot is actually using - it this shows you mistakes like typos, certain deprecated/invalid settings, and even if you are editing the wrong config file(s).

--

Best regards,

Charles

Thanks for the tip!

I didn't know of the dovecot -n command so thanks for pointing that out to me......

It's strange as I've been fiddling around with mail servers for some time in test labs at home but I still feel like I'm on the outside looking in; oh well at least this design at work is much better even though it took forever to get the PAM potion for AD sorted out.

Regards,

Kaya