- dovecot - dovecot.org

[Dovecot] Multiply mailboxes vs one huge
by Alexander Chekalin 10 Apr '12

10 Apr '12

Hello, as I need to store a lot of messages on my IMAP server (order of 900K-1000K; this is an archive for some time, maybe a year or so), I see some "slowness" in dealing with such a huge amount. I mainly need to do searches like "get all messages from user1(a)domain1.com to user2(a)domain2.tld recieved between date1 and date2". So I really interested will it be wise to a) split all messages into several smaller mailboxes (per-month, or per-day, or create 2-level-structure like "month/day/") b) use dbox (vs currently used mbox) storage scheme (I'm afraid of mdbox as I still not sure I'll be able to parse it by scripts later "just in case") Dovecot is the latest one (2.1.3). No compression Dovecot-side, but it mails are in zfs volume with compression on. I ask this mainly due to my not fully understand how Dovecot indexes are working. I also test another approach: to use my own index somewhere outside Dovecot which will store reference between emails and UIDs, and dates and UIDs, so I'll simple query my index for things I need. But then, that's exactly what IMAP index can do, so I simple slow my search down, isn't it? The only reason I think about my own index is I won't use 'all header' as search scope, I need to deal only with From:, To:, Cc:, Bcc: (if any), Recieved (if nowehere else I see the from/to info), and date field(s) - I doubt IMAP will care for that for me. Yours, Alexander

2 3

[Dovecot] v2.1.4 released
by Timo Sirainen 10 Apr '12

10 Apr '12

http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.4.tar.gz.sig + Added mail_temp_scan_interval setting and changed its default value from 8 hours to 1 week. + Added pop3-migration plugin for easily doing a transparent IMAP+POP3 migration to Dovecot: http://wiki2.dovecot.org/Migration/Dsync + doveadm user: Added -m parameter to show some of the mail settings. - Proxying SSL connections crashed in v2.1.[23] - fts-solr: Indexing mail bodies was broken. - director: Several changes to significantly improve error handling - doveadm import didn't import messages' flags - mail_full_filesystem_access=yes was broken - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors.

3 3

[Dovecot] mount
by Luigi Rosa 09 Apr '12

09 Apr '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have a Dovecot installation on CentOS 5 where I sometimes mount external fs in /mnt Every Dovecot data is in local / file system, nothing is mounted elswhere After upgrading to 1.2.4 I rebooted the system for other reasons and at startup I got this on Dovecot log: master: Warning: /mnt is no longer mounted. If this is intentional, remove it with doveadm mount No /mnt entry in /etc/fstab and nothing pmounted under /mnt I THINK that the last time I used /mnt to mount something was few weeks ago to update VMware tools. Is there a way toi tell Dovecot to ignore /mnt ? Ciao, luigi - -- / +--[Luigi Rosa]-- \ $100 invested at 7% interest for 100 years will become $100,000, at which time it will be worth absolutely nothing. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+C58gACgkQ3kWu7Tfl6ZRc0wCgl0Z4OtblYbfYwzvOp1/vUifV PqYAoIvfltvmq3cijvDbOEKV2Tai2rpu =hyrI -----END PGP SIGNATURE-----

2 2

[Dovecot] Director pop-login and imap-login processes exiting on signal 11
by Andy Dills 09 Apr '12

09 Apr '12

We recently upgraded our cluster to 2.1.3, to enable director proxying. Everything appears to be working fine for the most part; the only odd thing is that I'm seeing a lot of entries in the logs like this: Apr 7 02:18:05 mail-out06 dovecot: pop3-login: Fatal: master: service(pop3-login): child 75029 killed with signal 11 (core not dumped - set service pop3-login { drop_priv_before_exec=yes }) This is on the proxy side, not that backend side. When I try to get a dump out of it, and add drop_prive_before_exec and chroot= to the pop3-login statement on the proxy, I keep running into permissions errors with the various service sockets. Any suggestions? Thanks, Andy --- Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972 ---

2 2

[Dovecot] v2.0.20 released
by Timo Sirainen 09 Apr '12

09 Apr '12

http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.20.tar.gz.sig + doveadm user: Added -m parameter to show some of the mail settings. - doveadm import didn't import messages' flags - Make sure IMAP clients can't create directories when accessing nonexistent users' mailboxes via shared namespace. - Dovecot auth clients authenticating via TCP socket could have failed with bogus "PID already in use" errors.

1 0

[Dovecot] Director simplification?
by Timo Sirainen 09 Apr '12

09 Apr '12

An idea I just had: Director basically works by assigning the backend IP address by: ip = vhosts[ md5(username) mod vhosts_count ].ip The rest of director is about what happens when vhosts[] or vhosts_count changes. What about instead doing this on IP address level? ip = ip_pool[ md5(username) mod ip_pool_size ] When a backend dies, you'll reassign the backend's IPs to other backends. Each backend should have many IPs. The main restriction here is that the IP pool cannot change without stopping the entire Dovecot. But if you initially allocate enough IPs, that shouldn't be a problem. And the advantage of this over the current director? To guarantee that one director can't break others, because they don't need to communicate with each others. The disadvantage of course is that it's a little less flexible and requires more planning ahead. The IP address reassignment would also need some distro-specific scripts. This could be implemented as an alternative director-lite or something. The doveadm director status-related commands could still work with it.

1 0

[Dovecot] Outlook (2010) -> Dovecot (IMAP) >10x slower with high network load and many folders
by Thomas von Eyben 09 Apr '12

09 Apr '12

Hi, I am seeing a >10x as slow performance when trying to complete a "send/receive" from an Outlook 2010 client to Dovecot via IMAP, but only when the LAN is fully loaded with other traffic, EG file copying. It seems the problem is when outlook is trying to identify folders that have changed since last "send/receive" thus traversing the hierachy. Observations: - Apple's Mail.app does not have problems when exposed to the same environment. - Eliminating Outlook is not an option. - The tests have been performed during the easter vacation, thus almost no other clients are using the network/mailserver. Under normal load there are 2-300 users using the network. Description of the environment: Server sw: Mac OS X Server 10.6.8 running dovecot: 1.1.20apple0.5 (OS = fully updated 10.6.x) Server hw: Xserve Quadcore intel Xeon 2.26 GHz 12 GB RAM - Never having a cpu load > 20%, mail is stored on a Promise vtrack RAID connected via Fibre Channel Client: Windows 7, Outlook 2010 The client have maybe 50 folders (and each folder about a handfull of subfolders) the size of the mailbox is around 3-5 GB. Test results: CLIENT-1 is having the problems when CLIENT-2 is using all the (100Mbps) bandwidth eg. copying files to MAIL-SRV. If I move CLIENT-1 to CLIENT-3 then almost all the delay is gone. NB.: I have not (yet) tested if the problem also exists when CLIENT-2 generates traffic to MAIL-SRV as opposed to OTHER-SRV (but I am expecting the same problems). When dumping the traffic on CLIENT-1 (with Wireshark) one thing is catching my eyes for the 'bad' case: There is a very long delay after each "REQUEST IDLE" until the next "REQUEST DONE" ca. 0,3 seconds. Comparing to the 'good' setup the pause at the same place in the communication is less than 1 ms ! So why this delay? Where/how shall I continue my debugging? - Run dtruss/dtrace scripts on the server? - Get dovecot to output more debug info (I guess it's doing it's best so no problems will be seen there…)? Are there any dovecot configurations that can be altered to workaround [what I think is a limitation in Outlook's IMAP implementation]? The current dovecot configuration is Apple's defaults (+POP3 disabled)) I am seing one warning from dovecotd -n though: - But I see the same warning on a clean installed not-yet-configured OS X Server so I guess it's not 'that' bad: Warning: fd limit 256 is lower than what Dovecot can use under full load (more than 456). Either grow the limit or change login_max_processes_count and max_mail_processes settings Physical setup: +---------------+ +------------------------------+ +-----------------+ | CLIENT-1 | | CLIENT-2 | | CLIENT-3 | | Outlook "bad" | | Traffic generator, eg. Samba | | Outlook "good" | +---------------+ +------------------------------+ +-----------------+ | | | +-------------------------------------------------+ | | 100Mbps switch (clients) | | +-------------------------------------------------+ | | | +------------------------------------------------------------------------+ | 1Gbps switch (for servers) | +------------------------------------------------------------------------+ | | +-----------------------------+ +-----------+ | MAIL-SRV: Dovecot, AFP, SMB | | OTHER-SRV | +-----------------------------+ +-----------+ Wireshark dump - IMAP communication: |Time | 10.211.55.3 | | | | 10.0.0.10 | |9.851 | Response: gatq OK L |IMAP: Response: gatq OK Logged in. | |(49433) <------------------ (143) | |9.852 | Request: o47u SELEC |IMAP: Request: o47u SELECT "1_GROUPS" | |(49433) ------------------> (143) | |9.853 | Response: * FLAGS ( |IMAP: Response: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) | |(49433) <------------------ (143) | |9.854 | Request: 3y4b NOOP |IMAP: Request: 3y4b NOOP | |(49433) ------------------> (143) | |9.854 | Response: 3y4b OK N |IMAP: Response: 3y4b OK NOOP completed. | |(49433) <------------------ (143) | |9.855 | Request: 4vlj IDLE |IMAP: Request: 4vlj IDLE | |(49433) ------------------> (143) | |9.856 | Response: + idling |IMAP: Response: + idling | |(49433) <------------------ (143) | |10.108 | Request: DONE |IMAP: Request: DONE | |(49433) ------------------> (143) | |10.108 | Response: 4vlj OK I |IMAP: Response: 4vlj OK Idle completed. | |(49433) <------------------ (143) | |10.108 | Request: wh89 SELEC |IMAP: Request: wh89 SELECT "1_GROUPS.Adm" BR Thomas von Eyben

4 4

[Dovecot] Setting ACL for master user after login
by PL MB 09 Apr '12

09 Apr '12

Hello, I'd like to log in to normal user accounts as a master user but retain the normal users' ACLs. The Master Users page on the Dovecot 1.x wiki (1) says that I can set the master user's ACLs in a postlogin script. The documentation for master users on the 2.x wiki (2) no longer has any statements about master user ACLs. Has something important in this regard changed? Can I no longer override the ACLs in a postlogin script? Thanks in advance. (1) http://wiki.dovecot.org/Authentication/MasterUsers (2) http://wiki2.dovecot.org/Authentication/MasterUsers

2 1

[Dovecot] POP3 dele to Trash?
by Kelsey Cummings 09 Apr '12

09 Apr '12

To simplify recovery of accidentally deleted messages we'd like to send all messages that are deleted by pop clients to their Trash folder. (Which is auto expired already.) This allows a POP client who deletes an important email to restore it themselves by logging into our webmail client and move the lost message from Trash to the Inbox where it would be fetched by their client again. Has anyone already done this? Should this be possible via a plugin? I see the deleted-to-trash imap plugin. We are using Maildir if it makes a difference. -- Kelsey Cummings - kgc(a)corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407

2 2

[Dovecot] distributed mdbox
by James Devine 09 Apr '12

09 Apr '12

Anyone know how to setup dovecot with mdbox so that it can be used through shared storage from multiple hosts? I've setup a gluster volume and am sharing it between 2 test clients. I'm using postfix/dovecot LDA for delivery and I'm using postal to send mail between 40 users. In doing this, I'm seeing these errors in the logs Mar 21 09:36:29 test-gluster-client2 dovecot: lda(testuser34): Error: Fixed index file /mnt/testuser34/mdbox/storage/dovecot.map.index: messages_count 272 -> 271 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: Log synchronization error at seq=4,offset=3768 for /mnt/testuser28/mdbox/storage/dovecot.map.index: Append with UID 516, but next_uid = 517 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: Log synchronization error at seq=4,offset=4220 for /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update for invalid uid=517 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Error: Log synchronization error at seq=4,offset=5088 for /mnt/testuser28/mdbox/storage/dovecot.map.index: Extension record update for invalid uid=517 Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser28): Warning: fscking index file /mnt/testuser28/mdbox/storage/dovecot.map.index Mar 21 09:36:30 test-gluster-client2 dovecot: lda(testuser34): Warning: fscking index file /mnt/testuser34/mdbox/storage/dovecot.map.index This is my dovecot config currently: jdevine@test-gluster-client2:~> dovecot -n # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-13-server x86_64 Ubuntu 11.10 lock_method = dotlock mail_fsync = always mail_location = mdbox:~/mdbox mail_nfs_index = yes mail_nfs_storage = yes mmap_disable = yes passdb { driver = pam } protocols = " imap" ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd }

5 19