- dovecot - dovecot.org

[Dovecot] error bad file number with compressed mbox files
by Jürgen Obermann 04 Jan '12

04 Jan '12

Hello, can dsync convert from compressed mbox to compressed mdbox format? When I use compressed mbox files, either with gzip or with bzip2, I can read the mails as usual, but I find the following errors in dovecots log file: imap(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.gz, F_RDLCK) failed: Bad file number imap(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.bz2, F_RDLCK) failed: Bad file number These errors also appear when I use dsync to convert the compressed mbox to mdbox format on a second dovecot server: /opt/local/bin/dsync -v -u userxy backup mdbox:/sanpool/mail/home/hrz/userxy/mdbox dsync(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.gz, F_RDLCK) failed: Bad file number But now dovecot does not find the mails in the folder mymbox.gz on the second dovecot server in mdbox format! The relevant part of the dovcot configuration is: # 2.0.16: /opt/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4v mail_fsync = always mail_location = mbox:~/Mail:INBOX=/var/mail/%u mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = mail_log notify zlib mmap_disable = yes Thank you, -- Jürgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universität Gießen Heinrich-Buff-Ring 44 Tel. 0641-9913054

2 2

Re: [Dovecot] GPFS for mail-storage (Was: Re: Compressing existing maildirs)
by list＠airstreamcomm.net 04 Jan '12

04 Jan '12

Great information, thank you. Could you remark on GPFS services hosting mail storage over a WAN between two geographically separated data centers? ----- Reply message ----- From: "Jan-Frode Myklebust" <janfrode(a)tanso.net> To: "Stan Hoeppner" <stan(a)hardwarefreak.com> Cc: "Timo Sirainen" <tss(a)iki.fi>, <dovecot(a)dovecot.org> Subject: [Dovecot] GPFS for mail-storage (Was: Re: Compressing existing maildirs) Date: Tue, Jan 3, 2012 2:14 am On Sat, Dec 31, 2011 at 01:54:32AM -0600, Stan Hoeppner wrote: > Nice setup. I've mentioned GPFS for cluster use on this list before, > but I think you're the only operator to confirm using it. I'm sure > others would be interested in hearing of your first hand experience: > pros, cons, performance, etc. And a ball park figure on the licensing > costs, whether one can only use GPFS on IBM storage or if storage from > others vendors is allowed in the GPFS pool. I used to work for IBM, so I've been a bit uneasy about pushing GPFS too hard publicly, for risk of being accused of being biased. But I changed job in November, so now I'm only a satisfied customer :-) Pros: Extremely simple to configure and manage. Assuming root on all nodes can ssh freely, and port 1191/tcp is open between the nodes, these are the commands to create the cluster, create a NSD (network shared disks), and create a filesystem: # echo hostname1:manager-quorum > NodeFile # "manager" means this node can be selected as filesystem manager # echo hostname2:manager-quorum >> NodeFile # "quorum" means this node has a vote in the quorum selection # echo hostname3:manager-quorum >> NodeFile # all my nodes are usually the same, so they all have same roles. # mmcrcluster -n NodeFile -p $(hostname) -A ### sdb1 is either a local disk on hostname1 (in which case the other nodes will access it over tcp to ### hostname1), or a SAN-disk that they can access directly over FC/iSCSI. # echo sdb1:hostname1::dataAndMetadata:: > DescFile # This disk can be used for both data and metadata # mmcrnsd -F DescFile # mmstartup -A # starts GPFS services on all nodes # mmcrfs /gpfs1 gpfs1 -F DescFile # mount /gpfs1 You can add and remove disks from the filesystem, and change most settings without downtime. You can scale out your workload by adding more nodes (SAN attached or not), and scale out your disk performance by adding more disks on the fly. (IBM uses GPFS to create scale-out NAS solutions http://www-03.ibm.com/systems/storage/network/sonas/ , which highlights a few of the features available with GPFS) There's no problem running GPFS on other vendors disk systems. I've used Nexsan SATAboy earlier, for a HPC cluster. One can easily move from one disksystem to another without downtime. Cons: It has it's own page cache, staticly configured. So you don't get the "all available memory used for page caching" behaviour as you normally do on linux. There is a kernel module that needs to be rebuilt on every upgrade. It's a simple process, but it needs to be done and means we can't just run "yum update ; reboot" to upgrade. % export SHARKCLONEROOT=/usr/lpp/mmfs/src % cp /usr/lpp/mmfs/src/config/site.mcr.proto /usr/lpp/mmfs/src/config/site.mcr % vi /usr/lpp/mmfs/src/config/site.mcr # correct GPFS_ARCH, LINUX_DISTRIBUTION and LINUX_KERNEL_VERSION % cd /usr/lpp/mmfs/src/ ; make clean ; make World % su - root # export SHARKCLONEROOT=/usr/lpp/mmfs/src # cd /usr/lpp/mmfs/src/ ; make InstallImages > > To this point IIRC everyone here doing clusters is using NFS, GFS, or > OCFS. Each has its downsides, mostly because everyone is using maildir. > NFS has locking issues with shared dovecot index files. GFS and OCFS > have filesystem metadata performance issues. How does GPFS perform with > your maildir workload? Maildir is likely a worst case type workload for filesystems. Millions of tiny-tiny files, making all IO random, and getting minimal controller read cache utilized (unless you can cache all active files). So I've concluded that our performance issues are mostly design errors (and the fact that there were no better mail storage formats than maildir at the time these servers were implemented). I expect moving to mdbox will fix all our performance issues. I *think* GPFS is as good as it gets for maildir storage on clusterfs, but have no number to back that up ... Would be very interesting if we could somehow compare numbers for a few clusterfs'. I believe our main limitation in this setup is the iops we can get from the backend storage system. It's hard to balance the IO over enough RAID arrays (the fs is spread over 11 RAID5 arrays of 5 disks each), and we're always having hotspots. Right now two arrays are doing <100 iops, while others are doing 4-500 iops. Would very much like to replace it by something smarter where we can utilize SSDs for active data and something slower for stale data. GPFS can manage this by itself trough it's ILM interface, but we don't have the very fast storage to put in as tier-1. -jf

2 1

[Dovecot] Problem with huge IMAP Archive after Courier migration
by Preacher 04 Jan '12

04 Jan '12

I have a mail server running Debian 6.0 with Courier IMAP to store project related mail. Currently the maildir of the archive (one user) contains about 37GB of data. Our staff is acessing the archive via Outlook 2007 where they drag their Exchange inbox or sent files to it. The problem with courier is that is sometimes mixes up headers with message bodies, so I wanted to migrate to dovecot. I tried this on my proxy running Debian 7.0 with some test data and this worked fine (OK, spent some hours to get the config files done - Dovecot without authentication). Dovecot version here is 2.0.15. Tried it with our productive system today, but got Dovecot 1.2.15 installed on Debian 6.0 Config files and parameters I took from my test system were not compatible and I didn't get it to work. So I forced to install the Debisn 7.0 packages with 2.0.15 and finally got the server running, I also restarted the whole machine to empty caches. But the problem I got was that in the huge folder hierarchy the downloaded headers in the individual folders disappeared, some folders showed a few very old messages, some none. Also some subfolders disappeared. I checked this with Outlook and Thunderbird. The difference was, that Thunderbird shows more messages (but not all) than Outlook in some folders, but also none in some others. Outlook brought up a message in some cases, that the connection timed out, although I set the timeout to 60s. After being frustrated uninstalled dovecot, went back to Courier and folder contents are displayed correctly again. Anyone a clue what's wrong here? Finally some config information: proxy-server:~# dovecot -n # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.1.0-1-686-pae i686 Debian wheezy/sid auth_debug_passwords = yes auth_mechanisms = plain login disable_plaintext_auth = no namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap ssl = no ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd }

5 7

[Dovecot] Maildir migration and uids
by David Jonas 03 Jan '12

03 Jan '12

I'm in the process of migrating a large number of maildirs to a 3rd party dovecot server (from a dovecot server). Tests have shown that using imap to sync the accounts doesn't preserve the uidl for pop3 access. My current attempt is to convert the maildir to mbox and add an X-UIDL header in the process. Run a second dovecot that serves the converted mbox. But dovecot's docs say, "None of these headers are sent to IMAP/POP3 clients when they read the mail". Is there any way to sync these maildirs to the new server and maintain the uids? The real goal is keep customers email clients happy when they are pointed at the new server. Am I just wishing?

2 2

[Dovecot] Deliver all addresses to the same mdbox:?
by Ralf Hildebrandt 03 Jan '12

03 Jan '12

For archiving purposes I'm delivering all addresses to the same mdbox: like this: passdb { driver = passwd-file args = username_format=%u /etc/dovecot/passwd } userdb { driver = static args = uid=1000 gid=1000 home=/home/copymail allow_all_users=yes } Yet I'm getting this: Jan 3 19:03:27 mail postfix/lmtp[29378]: 3THjg02wfWzFvmL: to=<firstname.lastname@charite.de@backup.invalid>, relay=mail.charite.de[private/dovecot-lmtp], conn_use=20, delay=323, delays=323/0/0/0, dsn=4.1.1, status=SOFTBOUNCE (host mail.charite.de[private/dovecot-lmtp] said: 550 5.1.1 <"firstname.lastname(a)charite.de"@backup.invalid> User doesn't exist: "firstname.lastname(a)charite.de"@backup.invalid (in reply to RCPT TO command)) (using soft_bounce = yes here in Postfix) In short: backup.invalid is delivered to dovecot by means of LMTP (local socket). I thought my static mapping in userdb would enable the lmtp listener to accept ALL recipients and map their $home to /home/copymail - why is that not working? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

2 2

[Dovecot] Multiple Maildirs per Virtual User
by Ruslan Nabioullin 03 Jan '12

03 Jan '12

How would it be possible to configure dovecot (2.0.16) in such a way that it would serve several maildirs (e.g., INBOX, INBOX.Drafts, INBOX.Sent, forum_email, [Gmail].Trash, etc.) per virtual user? I am only able to specify a single maildir, but I want all maildirs in /home/my-username/mail/account1/ to be served. e.g., /etc/dovecot/passwd: my-username_account1:{PLAIN}password:my-username:my-group::::userdb_mail=maildir:/home/my-username/mail/account1/INBOX Thanks in advance, Ruslan -- Ruslan Nabioullin rnabioullin(a)gmail.com

2 3

[Dovecot] What is normal CPU usage of dovecot imap?
by Mikko Lampikoski 03 Jan '12

03 Jan '12

I got Dual Core Intel Xeon CPU 3.00GHz, over 1000 mailbox and almost 1 dovecot login / second (peak time). Server stats says that load is continually over 2 and cpu usage is 60%. top says that imap is making this load. virtual users are in mysql database and mysqld is running on another server (this server is ok). Do I need better CPU or is there something going on that I do not understand? # dovecot -n # 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-4-pve i686 Ubuntu 9.10 nfs log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s listen: *, [::] ssl_ca_file: /etc/ssl/**********.crt ssl_cert_file: /etc/ssl/**********.crt ssl_key_file: /etc/ssl/**********.key ssl_key_password: ********** disable_plaintext_auth: no verbose_ssl: yes shutdown_clients: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_process_size: 128 login_processes_count: 10 login_max_processes_count: 2048 mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 first_valid_uid: 99 last_valid_uid: 100 mail_privileged_group: mail mail_location: maildir:/var/vmail/%d/%n:INDEX=/var/indexes/%d/%n fsync_disable: yes mail_nfs_storage: yes mbox_write_locks: fcntl mbox_min_index_size: 4 mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 2048 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle imap_client_workarounds(imap): outlook-idle imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls auth default: mechanisms: plain login cram-md5 cache_size: 1024 passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=99 gid=99 home=/var/vmail/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail

2 3

[Dovecot] Compressing existing maildirs
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

I've just enabled zlib for our users, and am looking at how to compress the existing files. The routine for doing this at http://wiki2.dovecot.org/Plugins/Zlib seems a bit complicated. What do you think about simply doing: find /var/vmail -type f -name "*,S=*" -mtime +1 -exec gzip -S Z -6 '{}' + I.e. find all maildir-files: - with size in the name ("*,S=*") - modified before I enabled zlib plugin - compress them - add the Z suffix - keep timestamps (gzip does that by default) It's of course racy without the maildirlock, but are there any other problems with this approach ? -jf

4 14

[Dovecot] Dsync fails on second sync for folders with dot in the name
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

I'm in the processes of running our first dsync backup of all users (from maildir to mdbox on remote server), and one problem I'm hitting that dsync will work fine on first run for some users, and then reliably fail whenever I try a new run: $ sudo dsync -u janfrode(a)example.net backup ssh -q mailbackup(a)repo1.example.net dsync -u janfrode(a)example.net $ sudo dsync -u janfrode(a)example.net backup ssh -q mailbackup(a)repo1.example.net dsync -u janfrode(a)example.net dsync-remote(janfrode(a)example.net): Error: Can't delete mailbox directory INBOX/a: Mailbox has children, delete them first The problem here seems to be that this user has a maildir named ".a.b". On the backup side I see this as "a/b/". So dsync doesn't quite seem to agree with itself for how to handle folders with dot in the name. -jf

3 7

[Dovecot] lmtp-postlogin ?
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

We have last-login tracking for imap and pop, and I intend to use this for deciding which users to backup daily. But, it would also be nice to backup users who has only received messages, but not logged in lately.. So is it possible to implement "last-login" tracking for lmtp ? I naively tried copying the settings from imap, but it didn't work: service lmtp-postlogin { executable = script-login /usr/local/sbin/lmtp-postlogin.sh unix_listener lmtp-postlogin { } } service lmtp { executable = lmtp lmtp-postlogin <snip> -jf

2 3

[Dovecot] Newbie: LDA Isn't Logging
by Michael Papet 03 Jan '12

03 Jan '12

Hi, I'm a newbie having some trouble getting deliver to log anything. Related to this, there are no return values unless the -d is missing. I'm using LDAP to store virtual domain and user account information. Test #1: /usr/lib/dovecot/deliver -e -f mpapet(a)yahoo.com -d zed(a)mailswansong.dom < bad.mail Expected result: supposed to fail, there's no zed account via ldap lookup and supposed to get a return code per the wiki at http://wiki2.dovecot.org/LDA. Supposed to log too. Actual result: nothing gets delivered, no return code, nothing is logged. Test #2: /usr/lib/dovecot/deliver -f mpapet(a)yahoo.com -d dude(a)mailswansong.dom < good.mail Expected result: deliver to dude and return 0. Actual result: delivers, but no return code. Nothing logged. The wiki is vague about the difficulties of getting deliver LDA to log, but I thought I had it covered in my config. I even opened permissions up wide (777) on my log files specified below. Nothing gets logged. The ONLY thing changed in 15-lda.conf is as follows. protocol lda { # Space separated list of plugins to load (default is global mail_plugins). #mail_plugins = $mail_plugins log_path = /var/log/dovecot/lda.log info_log_path = /var/log/dovecot/lda-info.log service auth { unix_listener auth-client { mode = 0600 user = vmail } } I'm running plain Debian Testing and used dovecot from Debian's repository. The end-goal is to write a qpsmtpd queue plugin, but I need to figure out what's the matter first. Thanks in advance. mpapet

2 1

[Dovecot] dsync / separator / namespace config-problem
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

Guess I don't quite get namespaces, but we have the following namespace configured: $ doveconf namespace namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = subscriptions = yes type = private } But when I try running dsync it complains: $ sudo -u atmail dsync -u janfrode(a)tanso.net mirror mdbox:/tmp/janfrodeboks dsync(atmail): Error: user janfrode(a)tanso.net: Initialization failed: namespace configuration error: list=yes requires prefix=INBOX. to end with separator dsync(atmail): Fatal: User init failed I can fix this by adding "separator = ." to the namespace and dsync is happy, but is changing this now safe, or will this affect existing clients? -jf

2 5

[Dovecot] Trouble with proxy_maybe and auth_default_realm
by Gedalya 01 Jan '12

01 Jan '12

Hello, I'm using proxy_maybe and auth_default_realm. It seems that when a user logs in without the domain name, relying on auth_default_realm, and the "host" field points to the local server, I get the Proxying loops to itself error. It does work as expected - log on to the local server without proxying, if the user does include the domain name in the login. (IP's and domain name masked below) No domain: Dec 29 11:49:07 imap01 dovecot: pop3-login: Error: Proxying loops to itself: user=<jedi(a)----.com>, method=PLAIN, rip=00.00.52.18, lip=00.00.241.140 Dec 29 11:49:27 imap01 dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=<jedi(a)----.com>, method=PLAIN, rip=00.00.52.18, lip=00.00.241.140 With domain: Dec 29 11:52:13 imap01 dovecot: pop3-login: Login: user=<jedi(a)---.com>, method=PLAIN, rip=00.00.52.18, lip=00.00.241.140, mpid=19969 Dec 29 11:52:18 imap01 dovecot: pop3(jedi(a)---.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/1, size=731 Otherwise, e.g. when the proxy host is indeed another host, auth_default_domain works fine, including or not including the domain seems to make no difference, and everything works. I'm using mysql, and I'm able to get around this problem including the following in the password query: IF(host='' or host='00.00.241.140', NULL, 'Y') as proxy_maybe # dovecot --version 2.0.15 # dovecot -n # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 auth_default_realm = ----.com auth_mechanisms = plain login cram-md5 ntlm auth_username_format = %Lu auth_verbose = yes auth_verbose_passwords = plain dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no login_greeting = How can I help you? mail_gid = vmail mail_uid = vmail passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 lmtp service lmtp { inet_listener lmtp { address = 0.0.0.0 port = 7025 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes ----- dovecot-sql.conf.ext ---- driver = mysql connect = host=localhost dbname=email user=email default_pass_scheme = PLAIN password_query = SELECT password, \ IF('%s' = 'pop3', host_pop3, host) as host, \ IF(host='' or host='00.00.241.140', NULL, 'Y') as proxy_maybe, \ concat(userid, '@', domain) as destuser, \ password as pass, \ '/stor/mail/domains/%d/%n' AS userdb_home, \ 'maildir:/stor/mail/domains/%d/%n/Maildir' as userdb_mail, \ concat('*:storage=', quota_mb, 'M') as userdb_quota_rule, \ 'vmail' AS userdb_uid, 'vmail' AS userdb_gid \ FROM email WHERE userid = '%n' AND domain = '%d' user_query = SELECT '/stor/mail/domains/%d/%n' AS home, \ 'maildir:/stor/mail/domains/%d/%n/Maildir' as mail, \ concat('*:storage=', quota_mb, 'M') as quota_rule, \ 'vmail' AS uid, 'vmail' AS gid \ FROM email WHERE userid = '%n' AND domain = '%d'

1 1

[Dovecot] imap process limits problem
by Calvin Cochran 01 Jan '12

01 Jan '12

I am having a problem with the number of current processes that I cannot seem to diagnose adequately, and is a possible bug. This will be a bit long, but usually more info is better. I am running dovecot 2.0.16 on a CentOS 5 x86_64 server with the mailstore on gfs (output from dovecot -n at bottom). This is an imap issue. This is mostly to do with one client, but none of my tests indicate an issue with the client side. We have mail_max_userip_connections = 10 for imap, but they are not hitting the limit. We also have verbose_proctitle = yes to help in diagnosing the situation. Most of our clients, including this one, use SSL or TLS and connect on 993. As I understand it, that should have an imap-login process and an imap process per authenticated session. Based on some other diagnosis the client seems to have a PC using Outlook 2010 and an i-device (phone or pad, not sure), both on the office network, and both with imap connections to the server. Based on my analysis it seems like the client is connecting, authenticating, and then closing the session, but the imap-login process does not drop until it times out server side (I don't know a way to tell which device, the PC or i???). One odd thing is that the tcp sessions time out at 2 hours and 11 minutes (this is where the possible bug aspect comes in). I have put a strace on the process, and there does not appear to be any traffic, so I don't understand why the 30 min timeout isn't happening. Based on netstat and verbose_proctitle, at this moment there are 99 connections from the IP in question, all of which show in ps output as: dovecot/imap-login [1 connections (1 TLS)] My understanding is that means they have successfully authenticated, and that there should be line with dovecot/imap [username ip TLS] in ps output, but there isn't, so I am taking that to mean the client closed the imap session. The client ip address puts them on comcast (tcp resets?) and we do have a load balancer in front of two servers, just to add a little challenge to the diagnosis fun. The short term fix has been to increase the process limits. However, it is clearly not a workable solution to increase the limits by 100 every time someone starts accessing the server with their new i??? device. I appreciate your thoughts on this, and I am happy to provide additional useful debug info if I have missed something. Thanks, Calvin output from dovecot -n auth_mechanisms = plain apop cram-md5 auth_verbose = yes auth_verbose_passwords = plain auth_worker_max_count = 400 default_process_limit = 400 disable_plaintext_auth = no first_valid_uid = 89 last_valid_uid = 89 login_greeting = EMCS ready. mail_fsync = never maildir_very_dirty_syncs = yes mmap_disable = yes passdb { driver = vpopmail } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_log_group_events = no } protocols = imap pop3 shutdown_clients = no ssl_cert = </var/qmail/control/servercert.pem ssl_key = </var/qmail/control/servercert.pem userdb { driver = vpopmail } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 10 } protocol pop3 { mail_max_userip_connections = 3 pop3_no_flag_updates = yes pop3_uidl_format = %f }

4 4

[Dovecot] fts_solr break-imap-search 2.1
by Daniel L. Miller 31 Dec '11

31 Dec '11

Is "break-imap-search" still required using 2.1 with Solr to index TEXT & BODY searches? -- Daniel

2 1

[Dovecot] crashes on 2.0.16
by Cor Bosman 31 Dec '11

31 Dec '11

Hey all, I upgraded some servers today from Debian Lenny to Debian Squeeze, and after the upgrade I started getting dovecot crashes. I was on 2.0.13 but got these there as well, and hoped upgrading to 2.0.16 would help. It didn't. Anyone have an idea? Cor Dec 18 23:32:21 userimap1 kernel: [263492.245530] __ratelimit: 122 callbacks suppressed Dec 18 23:32:21 userimap1 kernel: [263492.245536] imap[19665]: segfault at 7f108efb7f60 ip 00007f108ecd37e0 sp 00007fff73c3d658 error 5 in libc-2.11.2.so[7f108ec5d000+158000] Dec 18 23:32:21 userimap1 kernel: [263492.245663] BUG: Bad page map in process imap pte:00000001 pmd:09767067 Dec 18 23:32:21 userimap1 kernel: [263492.252563] page:ffffea0000000000 flags:(null) count:-84 mapcount:-84 mapping:(null) index:0 Dec 18 23:32:21 userimap1 kernel: [263492.261186] addr:00007f108ef46000 vm_flags:08000070 anon_vma:(null) mapping:ffff88012b6e5238 index:2e9 Dec 18 23:32:21 userimap1 kernel: [263492.270804] vma->vm_ops->fault: filemap_fault+0x0/0x460 Dec 18 23:32:21 userimap1 kernel: [263492.276172] vma->vm_file->f_op->mmap: generic_file_mmap+0x0/0x60 Dec 18 23:32:21 userimap1 kernel: [263492.282424] Pid: 19665, comm: imap Tainted: G B 2.6.32.36-xsserver #1 Dec 18 23:32:21 userimap1 kernel: [263492.282429] Call Trace: Dec 18 23:32:21 userimap1 kernel: [263492.282441] [<ffffffff81103ee5>] print_bad_pte+0x1d5/0x280 Dec 18 23:32:21 userimap1 kernel: [263492.282448] [<ffffffff8110568a>] unmap_vmas+0xa1a/0xa20 Dec 18 23:32:21 userimap1 kernel: [263492.282458] [<ffffffff8137e501>] ? sch_direct_xmit+0x81/0x1d0 Dec 18 23:32:21 userimap1 kernel: [263492.282465] [<ffffffff8110b60d>] exit_mmap+0xbd/0x190 Dec 18 23:32:21 userimap1 kernel: [263492.282473] [<ffffffff81059cbd>] mmput+0x4d/0x120 Dec 18 23:32:21 userimap1 kernel: [263492.282481] [<ffffffff8105f282>] exit_mm+0x112/0x150 Dec 18 23:32:21 userimap1 kernel: [263492.282488] [<ffffffff8106104d>] do_exit+0x13d/0x800 Dec 18 23:32:21 userimap1 kernel: [263492.282496] [<ffffffff81072f31>] ? __dequeue_signal+0xe1/0x200 Dec 18 23:32:21 userimap1 kernel: [263492.282503] [<ffffffff81061765>] do_group_exit+0x55/0xd0 Dec 18 23:32:21 userimap1 kernel: [263492.282510] [<ffffffff81075251>] get_signal_to_deliver+0x1e1/0x3e0 Dec 18 23:32:21 userimap1 kernel: [263492.282517] [<ffffffff8100b315>] do_signal+0x75/0x7d0 Dec 18 23:32:21 userimap1 kernel: [263492.282523] [<ffffffff81103203>] ? print_vma_addr+0xb3/0x120 Dec 18 23:32:21 userimap1 kernel: [263492.282533] [<ffffffff8143856e>] ? printk+0x41/0x43 Dec 18 23:32:21 userimap1 kernel: [263492.282541] [<ffffffff81036436>] ? __bad_area_nosemaphore+0xf6/0x1e0 Dec 18 23:32:21 userimap1 kernel: [263492.282547] [<ffffffff810365ee>] ? bad_area_access_error+0x4e/0x60 Dec 18 23:32:21 userimap1 kernel: [263492.282553] [<ffffffff8100bac7>] do_notify_resume+0x57/0x60 Dec 18 23:32:21 userimap1 kernel: [263492.282559] [<ffffffff8100cadc>] retint_signal+0x48/0x8c # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32.36-xsserver x86_64 Debian 6.0.3 auth_cache_size = 1 M auth_verbose = yes base_dir = /var/run/dovecot/ default_vsz_limit = 2 G disable_plaintext_auth = no first_valid_uid = 20 lock_method = dotlock login_greeting = User-IMAP ready. login_trusted_networks = 194.109.26.128/27 mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mailbox_idle_check_interval = 1 mins maildir_stat_dirs = yes mmap_disable = yes namespace { inbox = yes location = maildir:%Nu:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u:INBOX=%Nu:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/INBOX prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot.masterusers driver = passwd-file master = yes } passdb { args = cache_key=%u%r dovecot driver = pam } plugin { quota = fs:User quota } protocols = imap service auth { client_limit = 5000 } service imap-login { process_min_avail = 8 service_count = 0 vsz_limit = 2 G } service imap { process_limit = 4096 vsz_limit = 2 G } shutdown_clients = no ssl_cert = </etc/ssl/certs/xx ssl_key = </etc/ssl/private/xx userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS imap_client_workarounds = delay-newmail mail_max_userip_connections = 24 mail_plugins = " quota imap_quota" }

16 30

[Dovecot] Upgrade/Migration
by sottilette＠rfx.it 30 Dec '11

30 Dec '11

Current box: CentOS 4.x (32 bit), dovecot 1.02, mbox format New box (supposed): CentOS 6.x (32/64 bit ? ), dovecot 2.04, mbox format The server have about 3.000 active users (unix users); /var/mail is about 100 GB; /home/Users about 150 GB I read the docs and searched on the net, but still request hints for planning the migration. 32/64 bit? Tricks to avoid users get all messages ...? Better first update the old server from 1.02 to 2.0x or ..? Any hints/comment appreciated ;-) Thanks, P. 1.02 dovecot.conf: ----------------------- protocols = pop3 pop3s imap imaps listen = [::] log_path = /var/log/mail/dovecot.log info_log_path = /var/log/mail/dovecot.info.log log_timestamp = "%b %d %H:%M:%S " default_mail_env = mbox:~/:INBOX=/var/mail/%u mail_full_filesystem_access = yes first_valid_uid = 100 mbox_read_locks = dotlock fcntl mbox_write_locks = dotlock fcntl auth default { mechanisms = plain passdb pam { } userdb passwd { } user = root } -----------------------

2 2

[Dovecot] mbox, hierarchical folders/folders and IMAP
by Alexander Chekalin 30 Dec '11

30 Dec '11

Hello, I'm now dealing with funny problem. I try to use Dovecot (2.1.rc1, actually) and mbox format to store old mailing, to keep logic of "one folder = one file". If I'd choose Maildir, I'd be able to create hierarchical folders (like "OldMail -> Maillists -> Dovecot"), and in fact on disk such a folders would be stored as real folders with dots in their names. In PHP it is easy to create with "imap_createmailbox()" function, I'd simple fire it 3 times (for "OldMail", for "OldMail.Maillists" and for "OldMail.Maillists.Dovecot"). But when I do the same with mbox, in fact it creates separate files (indeed, named "OldMail", "OldMail.Maillists" and "OldMail.Maillists.Dovecot") instead of creating filesystem folders ("OldMail/Maillists/Dovecot"). PHP is not guilty, Thunderbird won't create such folders as well. I can only create folders myself (say, with mkdir in the shell), chown it, and the it perfectly works in IMAP. Now I just wanted to know, are there any way to create IMAP-folders and subfolders using standard IMAP functionality? Thank you, Alexander

4 8

[Dovecot] Using Amazon S3 storage for Maildir ?
by Forrest Aldrich 30 Dec '11

30 Dec '11

I wonder if anyone has tried using an IMAP server with Amazon S3 as a backend. There would be many factors to consider (speed, etc). However, moreso, I don't know that Maildir(++) would support multiple folders that might be linked into the cloud. For example, higher-I/O local incoming mail is on disk, some folders archived on S3. Could be easily done by simply creating another account and storing its Maildir in S3 -- Curious....

3 2

[Dovecot] How to serve a subset of IMAP folders for hand held devices.
by David Pottage 30 Dec '11

30 Dec '11

Hello I am currently running dovecot 1.2.15 on my home server. I have over 200 folders in personal email account, and I have found that my smart-phone does not work well trying to display that many folders. I solved the problem by creating a second instance of dovecot running on a non standard port, and configured it to serve only a subset my email folders, so that way the email client on my smartphone works well and I can read the emails I am interested in on the move. The process of configuring two instances of dovecot on the same machine serving different views of the same mailbox was faily complex, so I think it would be worth documenting the process in an FAQ or suchlike for other users. I thought a good place to write up my method would be in the HowTo section of the Dovecot Wiki. Is that a good idea? Can I request an account to edit the Wiki, or should I send a draft text to an editor? Are there any Dovecot experts here who can review my methods and suggest improvements? -- David Pottage

8 10

[Dovecot] /usr/lib/dovecot/imap ignores log settings?
by Asheesh Laroia 29 Dec '11

29 Dec '11

Hi Dovecot peeps, I'm enjoying running /usr/lib/dovecot/imap directly as a PREAUTH IMAP pipe. I find that it outputs log messages whose severity is INFO to stdout. I'm running it as follows: /usr/lib/dovecot/imap -c /tmp/dovecot.conf and I see the following messages output to stdout: * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as paulproteus imap(paulproteus): Info: Connection closed bytes=0/305 Naturally, the first line is great. My /tmp/dovecot.conf is: <cut here> # single-dbox mail_location = sdbox:~/Maildir/dbox log_path = syslog info_log_path = /dev/null </cut> I'm also attaching strace output, created by running this command: echo | strace /usr/lib/dovecot/imap -c /tmp/dovecot.conf > /tmp/straced 2>&1 (The point of the "echo | " is to provide a newline and then EOF to /usr/lib/dovecot/imap.) You can see in the strace output that /usr/lib/dovecot/imap outputs the "Connection closed" message to file descriptor 2. I'd like that message to actually not be printed, which is why I set log_path to syslog. -- Asheesh.

2 2

[Dovecot] Dsync Error Can't Delete Mailbox
by Asai 29 Dec '11

29 Dec '11

Greetings, I'm getting this error in the dsync logs: Warning: Destination mailbox INBOX has been modified, need to recreate it before we can continue syncing dsync(foo(a)bar.org): Error: Can't delete mailbox INBOX: INBOX can't be deleted. In this situation, what should I do to remedy the problem? Just delete the backup and rebuild? -- --asai

2 2

[Dovecot] Received date no longer email date?
by Brian L. Matthews 29 Dec '11

29 Dec '11

I just upgrade from dovecot 1.2 to dovecot 2.0.16. I use dovecot as a local IMAP server to archive email from my corporate email account (on Exchange). I don't have any automated archiving, I just periodically move messages from my Exchange inbox to the IMAP inbox. I'm on Mac OS X 10.6.8 using Entourage as my mail client. In the previous version of dovecot, the Received date of the email was preserved when I moved a message, but it's not in 2.0.16, it's set to the date and time I move the message. If I open a message, the date displayed is the date on the message as expected. I converted my config from 1.2 with $ doveconf -n -c dovecot.conf > dovecot-2.conf It spit out various warnings but, once I'd created a dovenull user, seemed to work fine. Here's my configuration info: $ dovecot --version 2.0.16 $ dovecot -n # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Darwin 10.8.0 i386 disable_plaintext_auth = no listen = 127.0.0.1 log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir passdb { args = login driver = pam } service auth { user = root } service imap-login { process_limit = 16 } service imap { process_limit = 64 } ssl = no userdb { driver = passwd } I'd like to get it back to preserving the Received date, any help you can provide will be appreciated. Brian

2 1

[Dovecot] proxying, SSL, and client certificate
by Mike Abbott 29 Dec '11

29 Dec '11

How do I configure dovecot-2.0.x to present a client SSL certificate when proxying? If dovecot on server1.example.com has: passdb { driver = static args = proxy=y host=server2.example.com nopassword=y ssl=yes } and dovecot on server2.example.com has: ssl_verify_client_cert = yes auth_ssl_require_client_cert = yes then when a client connects to server1 and authenticates, a connection is established to server2 but the SSL handshake fails because server1 doesn't present a client certificate. I don't see where ssl_client_ctx is tied to a client certificate in ssl-proxy-openssl.c. Thanks.

2 1

[Dovecot] Lucene vs Solr with 2.1 single server
by Daniel L. Miller 29 Dec '11

29 Dec '11

For a single server, is the Lucene backend now "better" than running a Solr server? -- Daniel

2 1