- dovecot - dovecot.org

[Dovecot] doveadm move from one user's mailbox to another user's mailbox?
by Ralf Hildebrandt 04 Jan '12

04 Jan '12

Is something along the lines: doveadm move -u sourceuser destinationuser:/inbox search_query possible with 2.0.16? I want to move mails from a backup mailbox (which has no valid password assigned) to a "restore" mailbox (which *HAS* a password assigned to it). -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

2 2

[Dovecot] migrate dovecot files 1.2.16 to 2.0.13 (OpenBSD 5.0)
by Wesley M. 04 Jan '12

04 Jan '12

Hi, I have a mailserver(Postfix+MySql) on OpenBSD 4.9 with Dovecot 1.2.16, all works fine. Now i want to do the same but on OpenBSD 5.0. I meet problems using dovecot 2.0.13 on OpenBSD 5.0. Some tests (on the box): telnet 127.0.0.1 110 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. Connection closed by foreign host. telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. Connection closed by foreign host. Seems that pop3/imap doesn't work 'netstat -anf inet' tcp 0 0 *.993 *.* LISTEN tcp 0 0 *.143 *.* LISTEN tcp 0 0 *.995 *.* LISTEN tcp 0 0 *.110 *.* LISTEN Therefore, ports are open. When i use Roundcube webmail, i have errors : error imap connection If someone can help me on. Thank you very much. Files to migrate (already tried to modify them) : dovecot.conf / dovecot-sql.conf / and 'dovecot -n ' ###############::::::::dovecot.conf:::::::::::################################# base_dir = /var/dovecot/ protocols = imap pop3 ssl_cert = /etc/ssl/dovecotcert.pem ssl_key = /etc/ssl/private/dovecot.pem ssl_cipher_list = HIGH:MEDIUM:+TLSv1:!SSLv2:+SSLv3 disable_plaintext_auth = yes default_login_user = _dovecot default_internal_user = _dovecot login_process_per_connection = no login_process_size = 64 mail_location = maildir:/var/mailserv/mail/%d/%n first_valid_uid = 1000 mmap_disable = yes protocol imap { mail_plugins = quota imap_quota autocreate imap_client_workarounds = delay-newmail } protocol pop3 { pop3_uidl_format = %08Xv%08Xu mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol lda { mail_plugins = sieve quota postmaster_address = postmaster(a)mailr130.localdomain sendmail_path = /usr/sbin/sendmail auth_socket_path = /var/run/dovecot-auth-master } auth default { mechanisms = plain login digest-md5 cram-md5 apop passdb { driver=sql args = /etc/dovecot/dovecot-sql.conf } userdb { driver=sql args = /etc/dovecot/dovecot-sql.conf } user = root socket listen { client { path = /var/spool/postfix/private/auth mode = 0660 user = _postfix group = _postfix } master { path = /var/run/dovecot-auth-master mode = 0600 user = _dovecot # User running Dovecot LDA group = _dovecot # Or alternatively mode 0660 + LDA user in this group } } } plugin { sieve=~/.dovecot.sieve sieve_storage=~/sieve } plugin { quota = maildir quota_rule = *:storage=5G quota_rule2 = Trash:storage=100M quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2 = storage=80%% /usr/local/bin/quota-warning.sh 80 } plugin { autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts } plugin { antispam_signature = X-Spam-Flag antispam_signature_missing = move # move silently without training antispam_trash = trash;Trash;Deleted Items; Deleted Messages antispam_spam = SPAM;Spam;spam;Junk;junk antispam_mail_sendmail = /usr/local/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_notspam = --ham antispam_mail_tmpdir = /tmp } ###############::::::::dovecot-sql.conf:::::::################################## driver = mysql connect = host=localhost dbname=mail user=postfix password=postfix default_pass_scheme = PLAIN password_query = SELECT email as user, password FROM users WHERE email = '%u' user_query = SELECT id as uid, id as gid, home, concat('*:storage=', quota, 'M') AS quota_rule FROM users WHERE email = '%u' ################### dovecot -n######################################## # 2.0.13: /etc/dovecot/dovecot.conf # OS: OpenBSD 5.0 i386 ffs auth_mechanisms = plain login digest-md5 cram-md5 apop base_dir = /var/dovecot/ default_internal_user = _dovecot default_login_user = _dovecot first_valid_uid = 1000 mail_location = maildir:/var/mailserv/mail/%d/%n mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/local/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items; Deleted Messages autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir quota_rule = *:storage=5G quota_rule2 = Trash:storage=100M quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2 = storage=80%% /usr/local/bin/quota-warning.sh 80 sieve = ~/.dovecot.sieve sieve_storage = ~/sieve } protocols = imap pop3 service auth { unix_listener /var/run/dovecot-auth-master { group = _dovecot mode = 0600 user = _dovecot } unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } user = root } service imap-login { service_count = 0 vsz_limit = 64 M } service pop3-login { service_count = 0 vsz_limit = 64 M } ssl_cert = /etc/ssl/dovecotcert.pem ssl_cipher_list = HIGH:MEDIUM:+TLSv1:!SSLv2:+SSLv3 ssl_key = /etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota autocreate } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xv%08Xu } protocol lda { auth_socket_path = /var/run/dovecot-auth-master mail_plugins = sieve quota postmaster_address = postmaster(a)mailr130.localdomain sendmail_path = /usr/sbin/sendmail } Cheers, Wesley. M www.mouedine.net

1 0

[Dovecot] error bad file number with compressed mbox files
by Jürgen Obermann 04 Jan '12

04 Jan '12

Hello, can dsync convert from compressed mbox to compressed mdbox format? When I use compressed mbox files, either with gzip or with bzip2, I can read the mails as usual, but I find the following errors in dovecots log file: imap(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.gz, F_RDLCK) failed: Bad file number imap(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.bz2, F_RDLCK) failed: Bad file number These errors also appear when I use dsync to convert the compressed mbox to mdbox format on a second dovecot server: /opt/local/bin/dsync -v -u userxy backup mdbox:/sanpool/mail/home/hrz/userxy/mdbox dsync(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.gz, F_RDLCK) failed: Bad file number But now dovecot does not find the mails in the folder mymbox.gz on the second dovecot server in mdbox format! The relevant part of the dovcot configuration is: # 2.0.16: /opt/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4v mail_fsync = always mail_location = mbox:~/Mail:INBOX=/var/mail/%u mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = mail_log notify zlib mmap_disable = yes Thank you, -- Jürgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universität Gießen Heinrich-Buff-Ring 44 Tel. 0641-9913054

2 2

Re: [Dovecot] GPFS for mail-storage (Was: Re: Compressing existing maildirs)
by list＠airstreamcomm.net 04 Jan '12

04 Jan '12

Great information, thank you. Could you remark on GPFS services hosting mail storage over a WAN between two geographically separated data centers? ----- Reply message ----- From: "Jan-Frode Myklebust" <janfrode(a)tanso.net> To: "Stan Hoeppner" <stan(a)hardwarefreak.com> Cc: "Timo Sirainen" <tss(a)iki.fi>, <dovecot(a)dovecot.org> Subject: [Dovecot] GPFS for mail-storage (Was: Re: Compressing existing maildirs) Date: Tue, Jan 3, 2012 2:14 am On Sat, Dec 31, 2011 at 01:54:32AM -0600, Stan Hoeppner wrote: > Nice setup. I've mentioned GPFS for cluster use on this list before, > but I think you're the only operator to confirm using it. I'm sure > others would be interested in hearing of your first hand experience: > pros, cons, performance, etc. And a ball park figure on the licensing > costs, whether one can only use GPFS on IBM storage or if storage from > others vendors is allowed in the GPFS pool. I used to work for IBM, so I've been a bit uneasy about pushing GPFS too hard publicly, for risk of being accused of being biased. But I changed job in November, so now I'm only a satisfied customer :-) Pros: Extremely simple to configure and manage. Assuming root on all nodes can ssh freely, and port 1191/tcp is open between the nodes, these are the commands to create the cluster, create a NSD (network shared disks), and create a filesystem: # echo hostname1:manager-quorum > NodeFile # "manager" means this node can be selected as filesystem manager # echo hostname2:manager-quorum >> NodeFile # "quorum" means this node has a vote in the quorum selection # echo hostname3:manager-quorum >> NodeFile # all my nodes are usually the same, so they all have same roles. # mmcrcluster -n NodeFile -p $(hostname) -A ### sdb1 is either a local disk on hostname1 (in which case the other nodes will access it over tcp to ### hostname1), or a SAN-disk that they can access directly over FC/iSCSI. # echo sdb1:hostname1::dataAndMetadata:: > DescFile # This disk can be used for both data and metadata # mmcrnsd -F DescFile # mmstartup -A # starts GPFS services on all nodes # mmcrfs /gpfs1 gpfs1 -F DescFile # mount /gpfs1 You can add and remove disks from the filesystem, and change most settings without downtime. You can scale out your workload by adding more nodes (SAN attached or not), and scale out your disk performance by adding more disks on the fly. (IBM uses GPFS to create scale-out NAS solutions http://www-03.ibm.com/systems/storage/network/sonas/ , which highlights a few of the features available with GPFS) There's no problem running GPFS on other vendors disk systems. I've used Nexsan SATAboy earlier, for a HPC cluster. One can easily move from one disksystem to another without downtime. Cons: It has it's own page cache, staticly configured. So you don't get the "all available memory used for page caching" behaviour as you normally do on linux. There is a kernel module that needs to be rebuilt on every upgrade. It's a simple process, but it needs to be done and means we can't just run "yum update ; reboot" to upgrade. % export SHARKCLONEROOT=/usr/lpp/mmfs/src % cp /usr/lpp/mmfs/src/config/site.mcr.proto /usr/lpp/mmfs/src/config/site.mcr % vi /usr/lpp/mmfs/src/config/site.mcr # correct GPFS_ARCH, LINUX_DISTRIBUTION and LINUX_KERNEL_VERSION % cd /usr/lpp/mmfs/src/ ; make clean ; make World % su - root # export SHARKCLONEROOT=/usr/lpp/mmfs/src # cd /usr/lpp/mmfs/src/ ; make InstallImages > > To this point IIRC everyone here doing clusters is using NFS, GFS, or > OCFS. Each has its downsides, mostly because everyone is using maildir. > NFS has locking issues with shared dovecot index files. GFS and OCFS > have filesystem metadata performance issues. How does GPFS perform with > your maildir workload? Maildir is likely a worst case type workload for filesystems. Millions of tiny-tiny files, making all IO random, and getting minimal controller read cache utilized (unless you can cache all active files). So I've concluded that our performance issues are mostly design errors (and the fact that there were no better mail storage formats than maildir at the time these servers were implemented). I expect moving to mdbox will fix all our performance issues. I *think* GPFS is as good as it gets for maildir storage on clusterfs, but have no number to back that up ... Would be very interesting if we could somehow compare numbers for a few clusterfs'. I believe our main limitation in this setup is the iops we can get from the backend storage system. It's hard to balance the IO over enough RAID arrays (the fs is spread over 11 RAID5 arrays of 5 disks each), and we're always having hotspots. Right now two arrays are doing <100 iops, while others are doing 4-500 iops. Would very much like to replace it by something smarter where we can utilize SSDs for active data and something slower for stale data. GPFS can manage this by itself trough it's ILM interface, but we don't have the very fast storage to put in as tier-1. -jf

2 1

[Dovecot] Problem with huge IMAP Archive after Courier migration
by Preacher 04 Jan '12

04 Jan '12

I have a mail server running Debian 6.0 with Courier IMAP to store project related mail. Currently the maildir of the archive (one user) contains about 37GB of data. Our staff is acessing the archive via Outlook 2007 where they drag their Exchange inbox or sent files to it. The problem with courier is that is sometimes mixes up headers with message bodies, so I wanted to migrate to dovecot. I tried this on my proxy running Debian 7.0 with some test data and this worked fine (OK, spent some hours to get the config files done - Dovecot without authentication). Dovecot version here is 2.0.15. Tried it with our productive system today, but got Dovecot 1.2.15 installed on Debian 6.0 Config files and parameters I took from my test system were not compatible and I didn't get it to work. So I forced to install the Debisn 7.0 packages with 2.0.15 and finally got the server running, I also restarted the whole machine to empty caches. But the problem I got was that in the huge folder hierarchy the downloaded headers in the individual folders disappeared, some folders showed a few very old messages, some none. Also some subfolders disappeared. I checked this with Outlook and Thunderbird. The difference was, that Thunderbird shows more messages (but not all) than Outlook in some folders, but also none in some others. Outlook brought up a message in some cases, that the connection timed out, although I set the timeout to 60s. After being frustrated uninstalled dovecot, went back to Courier and folder contents are displayed correctly again. Anyone a clue what's wrong here? Finally some config information: proxy-server:~# dovecot -n # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.1.0-1-686-pae i686 Debian wheezy/sid auth_debug_passwords = yes auth_mechanisms = plain login disable_plaintext_auth = no namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap ssl = no ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd }

5 7

[Dovecot] Maildir migration and uids
by David Jonas 03 Jan '12

03 Jan '12

I'm in the process of migrating a large number of maildirs to a 3rd party dovecot server (from a dovecot server). Tests have shown that using imap to sync the accounts doesn't preserve the uidl for pop3 access. My current attempt is to convert the maildir to mbox and add an X-UIDL header in the process. Run a second dovecot that serves the converted mbox. But dovecot's docs say, "None of these headers are sent to IMAP/POP3 clients when they read the mail". Is there any way to sync these maildirs to the new server and maintain the uids? The real goal is keep customers email clients happy when they are pointed at the new server. Am I just wishing?

2 2

[Dovecot] Deliver all addresses to the same mdbox:?
by Ralf Hildebrandt 03 Jan '12

03 Jan '12

For archiving purposes I'm delivering all addresses to the same mdbox: like this: passdb { driver = passwd-file args = username_format=%u /etc/dovecot/passwd } userdb { driver = static args = uid=1000 gid=1000 home=/home/copymail allow_all_users=yes } Yet I'm getting this: Jan 3 19:03:27 mail postfix/lmtp[29378]: 3THjg02wfWzFvmL: to=<firstname.lastname@charite.de@backup.invalid>, relay=mail.charite.de[private/dovecot-lmtp], conn_use=20, delay=323, delays=323/0/0/0, dsn=4.1.1, status=SOFTBOUNCE (host mail.charite.de[private/dovecot-lmtp] said: 550 5.1.1 <"firstname.lastname(a)charite.de"@backup.invalid> User doesn't exist: "firstname.lastname(a)charite.de"@backup.invalid (in reply to RCPT TO command)) (using soft_bounce = yes here in Postfix) In short: backup.invalid is delivered to dovecot by means of LMTP (local socket). I thought my static mapping in userdb would enable the lmtp listener to accept ALL recipients and map their $home to /home/copymail - why is that not working? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

2 2

[Dovecot] Multiple Maildirs per Virtual User
by Ruslan Nabioullin 03 Jan '12

03 Jan '12

How would it be possible to configure dovecot (2.0.16) in such a way that it would serve several maildirs (e.g., INBOX, INBOX.Drafts, INBOX.Sent, forum_email, [Gmail].Trash, etc.) per virtual user? I am only able to specify a single maildir, but I want all maildirs in /home/my-username/mail/account1/ to be served. e.g., /etc/dovecot/passwd: my-username_account1:{PLAIN}password:my-username:my-group::::userdb_mail=maildir:/home/my-username/mail/account1/INBOX Thanks in advance, Ruslan -- Ruslan Nabioullin rnabioullin(a)gmail.com

2 3

[Dovecot] What is normal CPU usage of dovecot imap?
by Mikko Lampikoski 03 Jan '12

03 Jan '12

I got Dual Core Intel Xeon CPU 3.00GHz, over 1000 mailbox and almost 1 dovecot login / second (peak time). Server stats says that load is continually over 2 and cpu usage is 60%. top says that imap is making this load. virtual users are in mysql database and mysqld is running on another server (this server is ok). Do I need better CPU or is there something going on that I do not understand? # dovecot -n # 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-4-pve i686 Ubuntu 9.10 nfs log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s listen: *, [::] ssl_ca_file: /etc/ssl/**********.crt ssl_cert_file: /etc/ssl/**********.crt ssl_key_file: /etc/ssl/**********.key ssl_key_password: ********** disable_plaintext_auth: no verbose_ssl: yes shutdown_clients: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_process_size: 128 login_processes_count: 10 login_max_processes_count: 2048 mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 first_valid_uid: 99 last_valid_uid: 100 mail_privileged_group: mail mail_location: maildir:/var/vmail/%d/%n:INDEX=/var/indexes/%d/%n fsync_disable: yes mail_nfs_storage: yes mbox_write_locks: fcntl mbox_min_index_size: 4 mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 2048 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle imap_client_workarounds(imap): outlook-idle imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls auth default: mechanisms: plain login cram-md5 cache_size: 1024 passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=99 gid=99 home=/var/vmail/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail

2 3

[Dovecot] Compressing existing maildirs
by Jan-Frode Myklebust 03 Jan '12

03 Jan '12

I've just enabled zlib for our users, and am looking at how to compress the existing files. The routine for doing this at http://wiki2.dovecot.org/Plugins/Zlib seems a bit complicated. What do you think about simply doing: find /var/vmail -type f -name "*,S=*" -mtime +1 -exec gzip -S Z -6 '{}' + I.e. find all maildir-files: - with size in the name ("*,S=*") - modified before I enabled zlib plugin - compress them - add the Z suffix - keep timestamps (gzip does that by default) It's of course racy without the maildirlock, but are there any other problems with this approach ? -jf

4 14