October 2010 - dovecot

[Dovecot] migration 1.2.5 -> 2.0.5 load increased dramatically
by Ralf Hildebrandt 06 Oct '10

06 Oct '10

During norml operation this morning we're seeing a significantly increased load on the IMAP server (factor 5, from 5 to 25). Does dovecot need to rebuild index files or perform other IO intensive operations after the upgrade? We're using Maildir Our monitoring shows increased "system" time. We're runnign fewer processes, the IO rate ist identical to yesterday, memory usage is identical. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 13

[Dovecot] Shared folder again !
by Johan Hendriks 06 Oct '10

06 Oct '10

Hello all, First of all sorry for this, it has been discussed many times, but i can not make it work. I tried for a long time now. I do something wrong, but i do not know what. First of all, i have virtual users, with virtual domains. Mail is stored in /usr/local/virtual/domain name/user So if my domain is domain.com and my e-mail address is johan(a)domain.com the directory is /usr/local/virtual/domain.com/johan i followed the wiki, and my configuration looks like the following. beasty dovecot # dovecot -n # 2.0.5: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.1-STABLE amd64 ufs auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes dict { acl = mysql:/usr/local/etc/dovecot/dovecot-dict-acl-sql.conf quotadict = mysql:/usr/local/etc/dovecot/dovecot-dict-quota.conf } disable_plaintext_auth = no first_valid_gid = 1004 first_valid_uid = 1004 last_valid_gid = 1004 last_valid_uid = 1004 log_timestamp = .%Y-%m-%d %H:%M:%S . mail_access_groups = vmail mail_debug = yes mail_gid = 1004 mail_location = maildir:/usr/local/virtual/%d/%n mail_uid = 1004 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { inbox = no list = children location = maildir:/usr/local/virtual/%%d/%%n/shared/:INDEX=/shared/%%n prefix = shared/%%d/%%u/ separator = / subscriptions = no type = shared } passdb { args = /usr/local/etc/dovecot/dovecot-mysql.conf driver = sql } plugin { acl = vfile acl_anyone = allow acl_shared_dict = proxy::acl autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Templates autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Templates quota = dict:::proxy::quotadict quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2 = storage=80%% /usr/local/bin/quota-warning.sh 80 } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 192.168.50.200 port = 143 } } service pop3-login { inet_listener pop3 { address = 192.168.50.200 port = 110 } } ssl = no userdb { args = /usr/local/etc/dovecot/dovecot-mysql.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep imap_idle_notify_interval = 2 mins imap_logout_format = bytes=%i/%o imap_max_line_length = 64 k mail_plugins = quota imap_quota autocreate acl imap_acl mail_log notify } protocol pop3 { mail_plugins = quota trash pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota acl mail_log notify postmaster_address = postmaster(a)test.com sendmail_path = /usr/local/sbin/sendmail } That is my config file. My dovecot-dict-acl-sql.conf looks like the following connect = host=localhost dbname=postfix user=postfix password=password map { pattern = shared/shared-boxes/$to/$from table = user_shares value_field = dummy fields { from_user = $from to_user = $to } } That is slightly different than the wiki says, if i use that , it gives me an error. the pattern on the wiki looks like this pattern = shared/shared-boxes/users/$to/$from if i login as user johan and set ACL on my INBOX i do it this way. i telnet to the server and issue the following beasty dovecot # telnet 192.168.50.200 143 Trying 192.168.50.200... Connected to beasty.localdomain.local. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN AUTH=LOGIN] imap, pop ready a login johan(a)domain.com mypass a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in b SETACL INBOX hans(a)domain.com lr b OK Setacl complete. c GETACL INBOX * ACL "INBOX" "hans(a)domain.com" lr "johan(a)domain.com" lrwstipekxacd c OK Getacl completed. This looks good i guess, also my database is filled, with the from and to ! But i can not see the mailbox if i login as user hans. beasty dovecot # telnet 192.168.50.200 143 Trying 192.168.50.200... Connected to beasty.localdomain.local. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN AUTH=LOGIN] imap, pop ready a login hans(a)domain.com hanspass a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in b list "" * * LIST (\HasNoChildren) "/" "Trash" * LIST (\HasNoChildren) "/" "Junk" * LIST (\HasNoChildren) "/" "Sent" * LIST (\HasNoChildren) "/" "Drafts" * LIST (\HasNoChildren) "/" "Templates" * LIST (\HasNoChildren) "/" "INBOX" b OK List completed. c list "" /shared/* c OK List completed. If been trying this for a very long time now, but can not see what i am missing. It must be something i ovelook, but i really can not find out what. I also tried some other configs from the mailing list, but still the same, no go. Thank you for your time to help another one with this issue. i am a little ashamed :D Regards, Johan Hendriks * *

2 1

[Dovecot] Imap Folder activity detect
by Antonio Perez-Aranda 06 Oct '10

06 Oct '10

Hi all, We are migrating a mail service with 80.000 users, and 40.000 of them are using IMAP. At present, this service is handled by Courier, with some custom patches. And we are migrating to Dovecot 1.2.15. One of these patches is used to interact with Spamassassin. If Courier detects any move/copy operation to the "IsSPAM" or "IsntSPAM" folders, a specific bash script is executed like IsSPAM.sh and IsntSPAM.sh These scripts do the following: IsSPAM.sh 1. Takes user as parameter USER. 2. Retrieves the homeDirectory of USER from ldap, using ldapsearch. 3. Executes /usr/bin/sa-learn --spam ... with IsSPAM folder IsntSPAM.sh 1. Takes user as parameter USER. 2. Retrieves the homeDirectory of USER from ldap, using ldapsearch. 3. Executes /usr/bin/sa-learn --ham ... with IsntSPAM folder 4. Moves all content from IsntSPAM to INBOX I think that it shouldn't be very hard to implement a plugin to detect MOVE/COPY events to specified folders, and to execute specified scripts on detection. Is there already such a plugin for dovecot 1.2.15?

2 2

[Dovecot] 2.0.5: doveadm reload results in dovecot/auth crashing
by Ralf Hildebrandt 06 Oct '10

06 Oct '10

It's not THAT bad, but: GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Core was generated by `dovecot/auth'. Program terminated with signal 6, Aborted. #0 0xb7729430 in __kernel_vsyscall () #0 0xb7729430 in __kernel_vsyscall () No symbol table info available. #1 0xb7551751 in ?? () No symbol table info available. (gdb) quit I think my automatic debugger script needs to be fixed. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

1 1

[Dovecot] How to add the missing maildir folders
by Egbert Jan van den Bussche 06 Oct '10

06 Oct '10

Hi fellow list members, What would be the best way to add the .Sent,.Draft, etc. folders to the maildir tree? It seems not Dovecots task to do that but of the MUA. I have seen several options. Use of skel but that works only for new system users. I have only virtual users preloaded in a database. I've discoverred that ONLY Roundcube add the folders (if configured to do that) but I've not been able to have the folders created by Thunderbird. Yes, you can manage the subscribtions to folders but there is no way to create them. I did not bother to try Outlook... MY users use pop3, imap and web (Roundcube) but I cannot ask them to use Roundcube first, just to create the folders. I have no such thing as maildrop, which seems to be able to create folders. Should I run some script during login? How? TIA Egbert Jan (NL)

2 2

[Dovecot] Feature request for maildir style boxes
by David Ford 06 Oct '10

06 Oct '10

greetings, i'd like to ask for a certain feature request. dovecot:maildir_uidlist_recreate() to set the gid of new files based on the parent directory group ownership and normally that's desired, an appropriate security method. on our server, we use directory permissions to more stringently isolate access between users and services. we have three group ids used for this, and i'll use my name as example. Oct 05 13:44:30 imap(david): Error: fchown(/home/david/.maildir/dovecot-uidlist.tmp, -1, 497(net-mail)) failed: Operation not permitted (egid=1234(david), group based on /home/david/.maildir) Colt log # ls -ld /home/ /home/david /home/david/public_html/ /home/david/.maildir /home/david/.maildir/cur|awk '{printf "%s %5s %9s %s\n", $1,$3,$4,$9}' drwxr-xr-x root root /home/ drwx--x--- david net-all /home/david drwxr-x--- david net-mail /home/david/.maildir drwx------ david david /home/david/.maildir/cur drwxr-x--- david net-web /home/david/public_html/ the purpose of this is to prevent undesired access to personal files. users cannot 'cd' or 'ls' in other user's home directories, mail stores, or web files. apache, procmail, dovecot et cetera, are in the appropriate groups and therefore have access needed to do file ops. however, they're limited to their appropriate stores. as mentioned at the beginning, dovecot tries to match the gid of the parent directory for new files and normally, that's desired and expected behavior, but in our case. dovecot creates the file as uid/gid of the user, so the knob can either ignore the failure to set gid per the parent and not log it, or not attempt to set the gid per parent in the first place. src/lib-storage/index/maildir/maildir-uidlist.c 1412: if (box->file_create_gid != (gid_t)-1 && fchown(fd, (uid_t)-1, box->file_create_gid) < 0) { if (errno == EPERM) { mail_storage_set_critical(box->storage, "%s", eperm_error_get_chgrp("fchown", temp_path, box->file_create_gid, box->file_create_gid_origin)); } else { mail_storage_set_critical(box->storage, "fchown(%s) failed: %m", temp_path); } } to something like: ( "new_files_inherit_parent_gid = true" ) if (box->file_create_gid != -1 && --> box->new_files_inherit_parent_gid) { fchown(fd, -1, box->file_create_gid) ... } bool new_files_inherit_parent_gid [default true] could be added the following for example: src/lib-storage/mailbox-list-private.h:struct mailbox_list src/lib-storage/mail-storage-private.h:struct mailbox this block of code appears in similar instances for a number of other occasions (and could be made a more global function), but not all files in ~/.maildir/* appear to use a function like this. the uidvalidity functions are a little different for example. == for a busy mail server, that's a lot of excess logging and pollution when trying to review nightly logs for issues :) thank you for the consideration, -david

3 10

[Dovecot] Startup error dovecot-2.0.5
by Ralf Hildebrandt 06 Oct '10

06 Oct '10

When starting 2.0.5, I'm getting: # /usr/dovecot-2/sbin/dovecot -F doveconf: Warning: service auth { client_limit=4096 } is lower than required under max. load (12288) doveconf: Fatal: execvp(/usr/dovecot-2/sbin/dovecot) failed: Argument list too long Argument list too long? WTF? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

4 20

[Dovecot] (no subject)
by fakessh 06 Oct '10

06 Oct '10

hi folks hi timo , is a good day for you I use a double deliverance root and user i use lda deliver dovecot authentification and since the switch to 2.0.4 I encounter the problem that the box Maildir root for the name of the INBOX after passing sieve is the type root(a)nsXXXX.ovh.net how to correct the problem Timo your lights -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x092164A7 gpg --keyserver pgp.mit.edu --recv-key 092164A7

3 4

[Dovecot] Force index 2.04
by Henrique Fernandes 06 Oct '10

06 Oct '10

Hey, i am gonna migrate 240 GB of emails. To a new server with dovecot 2.0.4 and i am gonna also change the format from mbox to maildir. Everything is pretty ready. But i now need two things. First is, is there any command to rebuild index files in dovecot ? And i need to force a initial squat build, need comand for it also. It is about 5 thousands accounts. Its prefere have this built in dovecot than run a script that log in every sing account to make dovecot index the files, and after it make a small search to make initial squat also. Any one can help ? []'sf.rique

3 10

Re: [Dovecot] How to add the missing maildir folders
by interfaSys sàrl 06 Oct '10

06 Oct '10

Looks like it's also possible to use doveadm http://wiki2.dovecot.org/Tools/Doveadm/Mailbox

1 0