June 2011 - dovecot - dovecot.org

[Dovecot] dovecot, mdbox and indexes
by Angel L. Mateo 30 Jun '11

30 Jun '11

Hello everybody, We are updating our old dovecot 1.1 servers to 2.0. We have a farm os servers, with users in nfs filesystems and indexes in local fs. Mail is stored in maildir format. We don't warrant that a user is always directed to the same server, although our load balancer sends all request from one IP to the same server (during a session time with a inactivity timeout). This is the second scenario described at http://wiki2.dovecot.org/NFS. We are now trying to migrate to the third scenario and using mdbox. As an intermediate step, we have configured this second scenario, but with indexes in NFS filesystems. The configuration is: * mmap_disable = yes * dotlock_use_excl = no * mail_nfs_storage = yes * mail_nfs_index = yes * lock_method = fcntl (we have also tried with dotlock) This configuration works without any problems with few users, but with high load, we have a lot of lock timeouts error like: Jun 29 10:24:04 myotis34 dovecot: imap(mpm): Error: Timeout (180s) while waiting for lock for transaction log file /home/pas/01/014601/maildir_indexes/.INBOX/dovecot.index.log These errors cause a corrupted index and we have to manually delete the user's index so he could read his mail again. So we are planning to configure the third scenario. In this configuration, we don't have any problem during our tests if we use maildir format. But if we use mdbox one, we have problems when we have to change the final server for the server (We allways send the user to the same final server, but sometimes, if the server is down, for example, this is not possible). The tests we have done (with POP) are: * I connect to "my" server. There's no problem. I have my mail. I delete a mail. * I connect then to a server that have no index for me. The only problem is that the previously deleted mail appears again. * I connect then to a server with an outdated index: this is the problem. I have only the messages in that outdated index. The mail received since then has disappeared (although I could receive new mail). How could I solve this problem? Is this really a problem? Should I use a shared storage for indexes? What is the best configuration for a high availability service? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información _o) y las Comunicaciones Aplicadas (ATICA) / \\ http://www.um.es/atica _(___V Tfo: 868887590 Fax: 868888337

2 2

Re: [Dovecot] [Urgent] Email Retrieval from remote servers doesn't use Dovecot
by William Blunn 30 Jun '11

30 Jun '11

On 30/06/2011 14:08, Shazia Javed wrote: > Thank you for the reply. Now I have relatively better understanding of Dovecot and email service components (MUA, MTA, MRA, etc). However, I am still struggling with the relevant conceptual understanding, and need your help: > > I have installed 'getmail' as I read that it offers ease-of-use and is relatively more secure. So far I have managed to download emails from Inbox and Sent folder. All the downloaded emails go to single folder 'new' from 'Maildir' directory. Please tell me if there is a way (configuration) to download emails in respective folders. "getmail" appears to support delivery to an arbitrary Maildir folder. See "getmail" documentation. You might want to read up about Maildir to see how it works. In particular you need to know that mail is delivered to a directory "new" (the delivery process involves the directory "tmp", but the message isn't considered delivered until it reaches the directory "new"), but gets moved to a folder "cur" when the Maildir is looked at. So mail could be in either directory "new" or "cur". (The design of Maildir attempts to address various problems, and only partially succeeds, and in other cases makes things worse than they would have been otherwise. But despite its flaws, it still has some strengths, such as reliability, (relative) simplicity, wide support, and the ease by which a message store can be queried/interrogated.) > Also, is there any need for Dovecot now? As I see it, I have managed to download emails from remote servers in maildir format using getmail. How and why would Dovecot fit in? Will Dovecot (via deliver and sieve) assist in segregated email storage (i.e. INBOX in 'new', DRAFTS in '.draft/new' etc). It might be handy to have Dovecot installed as it would mean you could "see into" the mail folders using an IMAP client such as Thunderbird. This will probably make it easier to get a handle on which messages are in which folders than trying to look in the Maildir directories directly. IMAP has a special folder called "INBOX" which appears as the Inbox in most IMAP clients. Under Maildir, in the default location, INBOX can be found at ~/Maildir/{new,cur,tmp} Other folders are mapped into directories, for example the mail folder "Sent" will appear at ~/Maildir/.Sent/{new,cur,tmp} Nested mail folders are flattened into a single directory level in the filesystem, with the folder levels separated by dots, so the mail folder "Projects/PJ1" will appear at ~/Maildir/.Projects.PJ1/{new,cur,tmp} (The above is the default way for Maildir to work. However, Dovecot can be configured to map IMAP folders into Maildir directories in other ways.) Bill

1 0

[Dovecot] Problem accessing Dovecot mailing lists via IMAP
by William Blunn 30 Jun '11

30 Jun '11

I recently tried to look at the Dovecot mailing lists through an already-configured IMAP profile, which had worked fine in the past. But it failed saying that the authentication method was not supported. http://www.dovecot.org/mailinglists.html says: "You can get access to the IMAP archives by using *dovecot.org* as the hostname, *anonymous* as the username and an empty password. It's also possible to use ANONYMOUS authentication mechanism if your client supports it (few do)." I noticed that my e-mail client was configured to use the server "imap.dovecot.org", so I changed it to "dovecot.org". But it still complains that the authentication method is not supported. I am using Thunderbird 3.1 over Windows 7 64-bit. Thunderbird was configured for "Password, Transmitted insecurely" (and this option was not working). I have tried the other authentication options available in Thunderbird. "Encrypted password": Does not work; same problem "Kerberos / GSSAPI": Does not work; same problem "NTLM": Does not work; same problem Any ideas? Bill

3 3

[Dovecot] what to expect from changing index location
by Davide Vaghetti 30 Jun '11

30 Jun '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello everybody, I have one thousand virtual users with mdbox mailbox format and 10 GByte quota. I have noticed some performance problem related to I/O (the mailbox disk is a 6TB raid1+0 on ISCSI), so I want to put the index files on a different disk. My actual mail_location is: mail_location = mdbox:/var/vmail/%-1.1u/%u/mdbox and I want to switch to mail_location = mdbox:/var/vmail/%-1.1u/%u/mdbox:INDEX=/var/indexes/%-1.1u/%u/ But I cannot figure out a pair of things: - - do the switch trigger the rebuilding of the index files? - - can I get rid of all the old index files? - - how much the index files (no fts squat) can grow? thanks in advance bye davide - -- Dott. Davide Vaghetti Centro Servizi Informatici Facolta' di Ingegneria Universita' di Pisa PGP: http://keys.keysigning.org:11371/pks/lookup?op=get&search=0x7A1B3BA18C4E0A4D -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4J/Z4ACgkQehs7oYxOCk3QagCfd+yVvR2Ps6hbjZOCumn2Wx2W 6cgAnjOvQDfe1O9jDOAbp4k0deEmytd3 =ZJFQ -----END PGP SIGNATURE-----

4 8

[Dovecot] Mbox corruption - Inbox beginning with 'FFrom'
by Uwe Klaus 30 Jun '11

30 Jun '11

I have two accounts where the inboxes sometimes get corrupted. In that case they begin with "FFrom". I saw similar reports in July/August last year. So I applied the patch http://dovecot.org/pipermail/dovecot/attachments/20100720/e87b32ae/attachme… Yesterday a mbox corruption happened again. Here are the log lines. Jun 29 18:08:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:08:12 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=0/0, del=0/76, size=32800751 Jun 29 18:09:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:09:12 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=0/0, del=0/76, size=32800751 Jun 29 18:09:48 mail dovecot: deliver(xxxxxx): sieve: msgid=<D0FBDD35-A53D-4940-A770-DDA06AD0E70B(a)hgb-leipzig.de>: stored mail into mailbox 'INBOX' Jun 29 18:09:48 mail sm-mta[27493]: p5TFl24L014636: to=xxxxxx, delay=00:22:46, xdelay=00:00:00, mailer=local, pri=17899944, dsn=2.0.0, stat=Sent Jun 29 18:10:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:10:13 mail dovecot: POP3(xxxxxx): mbox got corrupted, state=1 Jun 29 18:10:13 mail dovecot: POP3(xxxxxx): mbox got corrupted, state=2 Jun 29 18:10:13 mail dovecot: POP3(xxxxxx): mbox got corrupted, state=4 Jun 29 18:10:13 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=1/2066, del=6/77, size=32802800 Jun 29 18:11:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:11:12 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=0/0, del=0/71, size=32777356 Jun 29 18:12:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:12:12 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=0/0, del=0/71, size=32777356 Jun 29 18:19:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:19:12 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=0/0, del=0/71, size=32777356 Jun 29 18:19:56 mail dovecot: deliver(xxxxxx): mbox got corrupted, state=0 Jun 29 18:19:56 mail dovecot: deliver(xxxxxx): sieve: msgid=<20110629175547.33981yt5jzia7f9f(a)mail.hgb-leipzig.de>: failed to store into mailbox 'INBOX': Mailbox isn't a valid mbox file Jun 29 18:19:56 mail dovecot: deliver(xxxxxx): sieve: script /home/xxxxxx/.dovecot.sieve failed with unsuccessful implicit keep Jun 29 18:19:57 mail sm-mta[12213]: p5TFvI0W029582: to=xxxxxx, delay=00:22:39, xdelay=00:00:01, mailer=local, pri=17900128, dsn=2.0.0, stat=Sent Jun 29 18:20:12 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=92.229.xxx.xxx, lip=194.94.211.50, TLS Jun 29 18:20:12 mail dovecot: POP3(xxxxxx): mbox got corrupted, state=0 Jun 29 18:20:12 mail dovecot: POP3(xxxxxx): Couldn't init INBOX: Mailbox isn't a valid mbox file Jun 29 18:20:12 mail dovecot: POP3(xxxxxx): Mailbox init failed top=0/0, retr=0/0, del=0/0, size=0 After removing the first "F": Jun 29 19:06:11 mail dovecot: pop3-login: Login: user=<xxxxxx>, method=PLAIN, rip=77.5.xxx.xxx, lip=194.94.211.50, TLS Jun 29 19:06:13 mail dovecot: POP3(xxxxxx): Disconnected: Logged out top=0/0, retr=2/4502, del=0/73, size=32781824 # dovecot --version 1.2.17 # dovecot -n # 1.2.17: /usr/pkg/etc/dovecot.conf Warning: fd limit 128 is lower than what Dovecot can use under full load (more than 768). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: NetBSD 5.1_STABLE amd64 protocols: imap imaps pop3 pop3s ssl_key_file: /etc/openssl/certs/dovecot.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/pkg/libexec/dovecot/imap-login login_executable(imap): /usr/pkg/libexec/dovecot/imap-login login_executable(pop3): /usr/pkg/libexec/dovecot/pop3-login first_valid_uid: 400 last_valid_uid: 10000 mail_location: mbox:~/mail:INBOX=/var/mail/%u:INDEX=/var/indexes/%u mail_executable(default): /usr/pkg/libexec/dovecot/imap mail_executable(imap): /usr/pkg/libexec/dovecot/imap mail_executable(pop3): /usr/pkg/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/pkg/lib/dovecot/imap mail_plugin_dir(imap): /usr/pkg/lib/dovecot/imap mail_plugin_dir(pop3): /usr/pkg/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu lda: postmaster_address: postmaster(a)hgb-leipzig.de mail_plugins: sieve auth default: username_format: %Lu verbose: yes passdb: driver: passwd userdb: driver: passwd plugin: sieve: ~/.dovecot.sieve sieve_global_path: /var/lib/dovecot/sieve/default.sieve sieve_dir: ~/sieve sieve_global_dir: /var/lib/dovecot/sieve/global/ Regards, Uwe

1 0

[Dovecot] Migrating shared folders from courier-imap to dovecot
by Tom Clark 30 Jun '11

30 Jun '11

Hi, I'm hoping you'll be able to help. I'm working on a project to migrate our users from a courier-imap (4.3.0) based mail server to a dovecot (2.0.8) mail server. I have got all the users migrated and working. However, I can't see how to migrate the shared folders from courier-imap. I have tried to find extra information all the migration information seems to centre around a user mailbox rather than shared mailboxes. Is it simply a case of setting up the shared folders on dovecot, then copying the messages in, finally running the dovecot migration script across them? Any advice would be greatly appreciated. Thanks, Tom

2 4

[Dovecot] LMTP returncode 450?
by Peter Mogensen 30 Jun '11

30 Jun '11

Hi, How do I get the LMTP-server to know which mailbox's are locally hosted and return SMTP code 450 if delivery is attempted to a non local user? I can see that a lookup in the userdb is done, but now matter what I return (1/111) from my checkpassword script I just get: 451 4.3.0 <lp(a)domain.tld> Internal error occurred. Refer to server log for more information. /Peter

2 3

[Dovecot] lda root vs lda postfix
by ml＠smtp.fakessh.eu 30 Jun '11

30 Jun '11

hello folks hello Timo I am facing a new problem I'm using dovecot LDA to deliver the emails now root of the system emails are delivered in the box to the letter of the postfix directory /vap/spool/postfix I do not know what to do to redirect mail from root to root -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x092164A7 gpg --keyserver pgp.mit.edu --recv-key 092164A7

2 2

[Dovecot] Denying user won't work as expected
by Frank Bonnet 30 Jun '11

30 Jun '11

Hello I tried to use the "deny user" feature but it does not work as expected ... ( dovecot 2.0.12 ) we use LDAP as authentication source I modified the 10-auth.conf file as the following # <doc/wiki/UserDatabase.txt> !include auth-deny.conf.ext #!include auth-master.conf.ext #!include auth-system.conf.ext #!include auth-sql.conf.ext !include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext #!include auth-static.conf.ext I modified the auth-deny.conf.ext file as the following passdb { driver = ldap deny = yes # File contains a list of usernames, one per line args = /usr/local/etc/dovecot/deny-users } I have tested writting one user (me) in the /usr/local/etc/dovecot/deny-users and the result is the login is denied for ALL users ... If I do not uncomment the !include auth-deny.conf.ext line in 10-auth.conf file everything works well I really need this feature Thanks for any help

2 3

Re: [Dovecot] Question about "Storage keeps breaking" in logs
by Maria Arrea 30 Jun '11

30 Jun '11

This dovecot box (RHEL 5.6 x64, dovecot 2.0.13, ext4) is a VM over vmware vsphere. The remote SAN disks are presented via iSCSI to the ESX hosts, and the ESX host show them as "local storage" /dev/sdX We have several VM sharing that datastore, no I/O contention detected. I will try with imaptest but this a production enviroment, I will take a couple of weeks to get a "mainteneance window" for this. Any other idea about this? Maybe this problem is load-related? Regards Maria ----- Original Message ----- From: Timo Sirainen Sent: 06/28/11 01:33 AM To: Maria Arrea Subject: Re: [Dovecot] Question about "Storage keeps breaking" in logs On Wed, 2011-06-22 at 19:32 +0000, Maria Arrea wrote: > Jun 6 17:03:23 buzon dovecot: imap(juanan): Warning: mdbox /buzones/18/66/juanan/storage: rebuilding indexes Problem 1: It doesn't log the reason for this rebuild. I can find only one place in code why that could happen, fixed: http://hg.dovecot.org/dovecot-2.0/rev/028b953e7040 > Jun 6 17:03:26 buzon dovecot: imap(juanan): Error: mdbox /buzones/18/66/juanan/mailboxes/P7.Bielorrusia/cd-eslpod-enviado/dbox-Mails: Storage keeps breaking Problem 2: It apparently can't fix that error. But in my tests it gets fixed fine.. > Jun 6 17:03:27 buzon dovecot: imap(juanan): Error: Recent flags state corrupted for mailbox Deleted Messages Probably related to the index rebuild. So this is a local disk with ext4 filesystem, nothing special? It shouldn't be breaking.. How often do these happen? Can you reproduce these by running imaptest against a test account? http://imapwiki.org/ImapTest

2 1