December 2011 - dovecot

[Dovecot] Trouble with proxy_maybe and auth_default_realm
by Gedalya 01 Jan '12

01 Jan '12

Hello, I'm using proxy_maybe and auth_default_realm. It seems that when a user logs in without the domain name, relying on auth_default_realm, and the "host" field points to the local server, I get the Proxying loops to itself error. It does work as expected - log on to the local server without proxying, if the user does include the domain name in the login. (IP's and domain name masked below) No domain: Dec 29 11:49:07 imap01 dovecot: pop3-login: Error: Proxying loops to itself: user=<jedi(a)----.com>, method=PLAIN, rip=00.00.52.18, lip=00.00.241.140 Dec 29 11:49:27 imap01 dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<jedi(a)----.com>, method=PLAIN, rip=00.00.52.18, lip=00.00.241.140 With domain: Dec 29 11:52:13 imap01 dovecot: pop3-login: Login: user=<jedi(a)---.com>, method=PLAIN, rip=00.00.52.18, lip=00.00.241.140, mpid=19969 Dec 29 11:52:18 imap01 dovecot: pop3(jedi(a)---.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/1, size=731 Otherwise, e.g. when the proxy host is indeed another host, auth_default_domain works fine, including or not including the domain seems to make no difference, and everything works. I'm using mysql, and I'm able to get around this problem including the following in the password query: IF(host='' or host='00.00.241.140', NULL, 'Y') as proxy_maybe # dovecot --version 2.0.15 # dovecot -n # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 auth_default_realm = ----.com auth_mechanisms = plain login cram-md5 ntlm auth_username_format = %Lu auth_verbose = yes auth_verbose_passwords = plain dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no login_greeting = How can I help you? mail_gid = vmail mail_uid = vmail passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 lmtp service lmtp { inet_listener lmtp { address = 0.0.0.0 port = 7025 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes ----- dovecot-sql.conf.ext ---- driver = mysql connect = host=localhost dbname=email user=email default_pass_scheme = PLAIN password_query = SELECT password, \ IF('%s' = 'pop3', host_pop3, host) as host, \ IF(host='' or host='00.00.241.140', NULL, 'Y') as proxy_maybe, \ concat(userid, '@', domain) as destuser, \ password as pass, \ '/stor/mail/domains/%d/%n' AS userdb_home, \ 'maildir:/stor/mail/domains/%d/%n/Maildir' as userdb_mail, \ concat('*:storage=', quota_mb, 'M') as userdb_quota_rule, \ 'vmail' AS userdb_uid, 'vmail' AS userdb_gid \ FROM email WHERE userid = '%n' AND domain = '%d' user_query = SELECT '/stor/mail/domains/%d/%n' AS home, \ 'maildir:/stor/mail/domains/%d/%n/Maildir' as mail, \ concat('*:storage=', quota_mb, 'M') as quota_rule, \ 'vmail' AS uid, 'vmail' AS gid \ FROM email WHERE userid = '%n' AND domain = '%d'

1 1

[Dovecot] imap process limits problem
by Calvin Cochran 31 Dec '11

31 Dec '11

I am having a problem with the number of current processes that I cannot seem to diagnose adequately, and is a possible bug. This will be a bit long, but usually more info is better. I am running dovecot 2.0.16 on a CentOS 5 x86_64 server with the mailstore on gfs (output from dovecot -n at bottom). This is an imap issue. This is mostly to do with one client, but none of my tests indicate an issue with the client side. We have mail_max_userip_connections = 10 for imap, but they are not hitting the limit. We also have verbose_proctitle = yes to help in diagnosing the situation. Most of our clients, including this one, use SSL or TLS and connect on 993. As I understand it, that should have an imap-login process and an imap process per authenticated session. Based on some other diagnosis the client seems to have a PC using Outlook 2010 and an i-device (phone or pad, not sure), both on the office network, and both with imap connections to the server. Based on my analysis it seems like the client is connecting, authenticating, and then closing the session, but the imap-login process does not drop until it times out server side (I don't know a way to tell which device, the PC or i???). One odd thing is that the tcp sessions time out at 2 hours and 11 minutes (this is where the possible bug aspect comes in). I have put a strace on the process, and there does not appear to be any traffic, so I don't understand why the 30 min timeout isn't happening. Based on netstat and verbose_proctitle, at this moment there are 99 connections from the IP in question, all of which show in ps output as: dovecot/imap-login [1 connections (1 TLS)] My understanding is that means they have successfully authenticated, and that there should be line with dovecot/imap [username ip TLS] in ps output, but there isn't, so I am taking that to mean the client closed the imap session. The client ip address puts them on comcast (tcp resets?) and we do have a load balancer in front of two servers, just to add a little challenge to the diagnosis fun. The short term fix has been to increase the process limits. However, it is clearly not a workable solution to increase the limits by 100 every time someone starts accessing the server with their new i??? device. I appreciate your thoughts on this, and I am happy to provide additional useful debug info if I have missed something. Thanks, Calvin output from dovecot -n auth_mechanisms = plain apop cram-md5 auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 400 default_process_limit = 400 disable_plaintext_auth = no first_valid_uid = 89 last_valid_uid = 89 login_greeting = EMCS ready. mail_fsync = never maildir_very_dirty_syncs = yes mmap_disable = yes passdb { driver = vpopmail } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = no } protocols = imap pop3 shutdown_clients = no ssl_cert = </var/qmail/control/servercert.pem ssl_key = </var/qmail/control/servercert.pem userdb { driver = vpopmail } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 10 } protocol pop3 { mail_max_userip_connections = 3 pop3_no_flag_updates = yes pop3_uidl_format = %f }

4 4

[Dovecot] fts_solr break-imap-search 2.1
by Daniel L. Miller 31 Dec '11

31 Dec '11

Is "break-imap-search" still required using 2.1 with Solr to index TEXT & BODY searches? -- Daniel

2 1

[Dovecot] crashes on 2.0.16
by Cor Bosman 31 Dec '11

31 Dec '11

Hey all, I upgraded some servers today from Debian Lenny to Debian Squeeze, and after the upgrade I started getting dovecot crashes. I was on 2.0.13 but got these there as well, and hoped upgrading to 2.0.16 would help. It didn't. Anyone have an idea? Cor Dec 18 23:32:21 userimap1 kernel: [263492.245530] __ratelimit: 122 callbacks suppressed Dec 18 23:32:21 userimap1 kernel: [263492.245536] imap[19665]: segfault at 7f108efb7f60 ip 00007f108ecd37e0 sp 00007fff73c3d658 error 5 in libc-2.11.2.so[7f108ec5d000+158000] Dec 18 23:32:21 userimap1 kernel: [263492.245663] BUG: Bad page map in process imap pte:00000001 pmd:09767067 Dec 18 23:32:21 userimap1 kernel: [263492.252563] page:ffffea0000000000 flags:(null) count:-84 mapcount:-84 mapping:(null) index:0 Dec 18 23:32:21 userimap1 kernel: [263492.261186] addr:00007f108ef46000 vm_flags:08000070 anon_vma:(null) mapping:ffff88012b6e5238 index:2e9 Dec 18 23:32:21 userimap1 kernel: [263492.270804] vma->vm_ops->fault: filemap_fault+0x0/0x460 Dec 18 23:32:21 userimap1 kernel: [263492.276172] vma->vm_file->f_op->mmap: generic_file_mmap+0x0/0x60 Dec 18 23:32:21 userimap1 kernel: [263492.282424] Pid: 19665, comm: imap Tainted: G B 2.6.32.36-xsserver #1 Dec 18 23:32:21 userimap1 kernel: [263492.282429] Call Trace: Dec 18 23:32:21 userimap1 kernel: [263492.282441] [<ffffffff81103ee5>] print_bad_pte+0x1d5/0x280 Dec 18 23:32:21 userimap1 kernel: [263492.282448] [<ffffffff8110568a>] unmap_vmas+0xa1a/0xa20 Dec 18 23:32:21 userimap1 kernel: [263492.282458] [<ffffffff8137e501>] ? sch_direct_xmit+0x81/0x1d0 Dec 18 23:32:21 userimap1 kernel: [263492.282465] [<ffffffff8110b60d>] exit_mmap+0xbd/0x190 Dec 18 23:32:21 userimap1 kernel: [263492.282473] [<ffffffff81059cbd>] mmput+0x4d/0x120 Dec 18 23:32:21 userimap1 kernel: [263492.282481] [<ffffffff8105f282>] exit_mm+0x112/0x150 Dec 18 23:32:21 userimap1 kernel: [263492.282488] [<ffffffff8106104d>] do_exit+0x13d/0x800 Dec 18 23:32:21 userimap1 kernel: [263492.282496] [<ffffffff81072f31>] ? __dequeue_signal+0xe1/0x200 Dec 18 23:32:21 userimap1 kernel: [263492.282503] [<ffffffff81061765>] do_group_exit+0x55/0xd0 Dec 18 23:32:21 userimap1 kernel: [263492.282510] [<ffffffff81075251>] get_signal_to_deliver+0x1e1/0x3e0 Dec 18 23:32:21 userimap1 kernel: [263492.282517] [<ffffffff8100b315>] do_signal+0x75/0x7d0 Dec 18 23:32:21 userimap1 kernel: [263492.282523] [<ffffffff81103203>] ? print_vma_addr+0xb3/0x120 Dec 18 23:32:21 userimap1 kernel: [263492.282533] [<ffffffff8143856e>] ? printk+0x41/0x43 Dec 18 23:32:21 userimap1 kernel: [263492.282541] [<ffffffff81036436>] ? __bad_area_nosemaphore+0xf6/0x1e0 Dec 18 23:32:21 userimap1 kernel: [263492.282547] [<ffffffff810365ee>] ? bad_area_access_error+0x4e/0x60 Dec 18 23:32:21 userimap1 kernel: [263492.282553] [<ffffffff8100bac7>] do_notify_resume+0x57/0x60 Dec 18 23:32:21 userimap1 kernel: [263492.282559] [<ffffffff8100cadc>] retint_signal+0x48/0x8c # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32.36-xsserver x86_64 Debian 6.0.3 auth_cache_size = 1 M auth_verbose = yes base_dir = /var/run/dovecot/ default_vsz_limit = 2 G disable_plaintext_auth = no first_valid_uid = 20 lock_method = dotlock login_greeting = User-IMAP ready. login_trusted_networks = 194.109.26.128/27 mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mailbox_idle_check_interval = 1 mins maildir_stat_dirs = yes mmap_disable = yes namespace { inbox = yes location = maildir:%Nu:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u:INBOX=%Nu:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/INBOX prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot.masterusers driver = passwd-file master = yes } passdb { args = cache_key=%u%r dovecot driver = pam } plugin { quota = fs:User quota } protocols = imap service auth { client_limit = 5000 } service imap-login { process_min_avail = 8 service_count = 0 vsz_limit = 2 G } service imap { process_limit = 4096 vsz_limit = 2 G } shutdown_clients = no ssl_cert = </etc/ssl/certs/xx ssl_key = </etc/ssl/private/xx userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS imap_client_workarounds = delay-newmail mail_max_userip_connections = 24 mail_plugins = " quota imap_quota" }

16 30

[Dovecot] Upgrade/Migration
by sottilette＠rfx.it 30 Dec '11

30 Dec '11

Current box: CentOS 4.x (32 bit), dovecot 1.02, mbox format New box (supposed): CentOS 6.x (32/64 bit ? ), dovecot 2.04, mbox format The server have about 3.000 active users (unix users); /var/mail is about 100 GB; /home/Users about 150 GB I read the docs and searched on the net, but still request hints for planning the migration. 32/64 bit? Tricks to avoid users get all messages ...? Better first update the old server from 1.02 to 2.0x or ..? Any hints/comment appreciated ;-) Thanks, P. 1.02 dovecot.conf: ----------------------- protocols = pop3 pop3s imap imaps listen = [::] log_path = /var/log/mail/dovecot.log info_log_path = /var/log/mail/dovecot.info.log log_timestamp = "%b %d %H:%M:%S " default_mail_env = mbox:~/:INBOX=/var/mail/%u mail_full_filesystem_access = yes first_valid_uid = 100 mbox_read_locks = dotlock fcntl mbox_write_locks = dotlock fcntl auth default { mechanisms = plain passdb pam { } userdb passwd { } user = root } -----------------------

2 2

[Dovecot] mbox, hierarchical folders/folders and IMAP
by Alexander Chekalin 30 Dec '11

30 Dec '11

Hello, I'm now dealing with funny problem. I try to use Dovecot (2.1.rc1, actually) and mbox format to store old mailing, to keep logic of "one folder = one file". If I'd choose Maildir, I'd be able to create hierarchical folders (like "OldMail -> Maillists -> Dovecot"), and in fact on disk such a folders would be stored as real folders with dots in their names. In PHP it is easy to create with "imap_createmailbox()" function, I'd simple fire it 3 times (for "OldMail", for "OldMail.Maillists" and for "OldMail.Maillists.Dovecot"). But when I do the same with mbox, in fact it creates separate files (indeed, named "OldMail", "OldMail.Maillists" and "OldMail.Maillists.Dovecot") instead of creating filesystem folders ("OldMail/Maillists/Dovecot"). PHP is not guilty, Thunderbird won't create such folders as well. I can only create folders myself (say, with mkdir in the shell), chown it, and the it perfectly works in IMAP. Now I just wanted to know, are there any way to create IMAP-folders and subfolders using standard IMAP functionality? Thank you, Alexander

4 8

[Dovecot] Using Amazon S3 storage for Maildir ?
by Forrest Aldrich 30 Dec '11

30 Dec '11

I wonder if anyone has tried using an IMAP server with Amazon S3 as a backend. There would be many factors to consider (speed, etc). However, moreso, I don't know that Maildir(++) would support multiple folders that might be linked into the cloud. For example, higher-I/O local incoming mail is on disk, some folders archived on S3. Could be easily done by simply creating another account and storing its Maildir in S3 -- Curious....

3 2

[Dovecot] How to serve a subset of IMAP folders for hand held devices.
by David Pottage 30 Dec '11

30 Dec '11

Hello I am currently running dovecot 1.2.15 on my home server. I have over 200 folders in personal email account, and I have found that my smart-phone does not work well trying to display that many folders. I solved the problem by creating a second instance of dovecot running on a non standard port, and configured it to serve only a subset my email folders, so that way the email client on my smartphone works well and I can read the emails I am interested in on the move. The process of configuring two instances of dovecot on the same machine serving different views of the same mailbox was faily complex, so I think it would be worth documenting the process in an FAQ or suchlike for other users. I thought a good place to write up my method would be in the HowTo section of the Dovecot Wiki. Is that a good idea? Can I request an account to edit the Wiki, or should I send a draft text to an editor? Are there any Dovecot experts here who can review my methods and suggest improvements? -- David Pottage

8 10

[Dovecot] /usr/lib/dovecot/imap ignores log settings?
by Asheesh Laroia 29 Dec '11

29 Dec '11

Hi Dovecot peeps, I'm enjoying running /usr/lib/dovecot/imap directly as a PREAUTH IMAP pipe. I find that it outputs log messages whose severity is INFO to stdout. I'm running it as follows: /usr/lib/dovecot/imap -c /tmp/dovecot.conf and I see the following messages output to stdout: * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as paulproteus imap(paulproteus): Info: Connection closed bytes=0/305 Naturally, the first line is great. My /tmp/dovecot.conf is: <cut here> # single-dbox mail_location = sdbox:~/Maildir/dbox log_path = syslog info_log_path = /dev/null </cut> I'm also attaching strace output, created by running this command: echo | strace /usr/lib/dovecot/imap -c /tmp/dovecot.conf > /tmp/straced 2>&1 (The point of the "echo | " is to provide a newline and then EOF to /usr/lib/dovecot/imap.) You can see in the strace output that /usr/lib/dovecot/imap outputs the "Connection closed" message to file descriptor 2. I'd like that message to actually not be printed, which is why I set log_path to syslog. -- Asheesh.

2 2

[Dovecot] Dsync Error Can't Delete Mailbox
by Asai 29 Dec '11

29 Dec '11

Greetings, I'm getting this error in the dsync logs: Warning: Destination mailbox INBOX has been modified, need to recreate it before we can continue syncing dsync(foo(a)bar.org): Error: Can't delete mailbox INBOX: INBOX can't be deleted. In this situation, what should I do to remedy the problem? Just delete the backup and rebuild? -- --asai

2 2