- dovecot - dovecot.org

[Dovecot] Using dsync+imapc for migration from other IMAP servers to Dovecot
by Timo Sirainen 24 Jan '11

24 Jan '11

Dovecot v2.1 hg (http://hg.dovecot.org/dovecot-2.1) now supports doing e.g.: dsync -o plugin/imapc_user=remote.user(a)example.com -o plugin/pass=remote-password mirror imapc:imap.example.com dsync will preserve message UIDs and UIDVALIDITY/UIDNEXT values for mailboxes, just like with all other storage backends. A couple of bad things: 1) It currently fetches one message at a time from server. Very slow.. Will be fixed some day. 2) You can't do incremental sync, because dsync relies on message GUIDs, which aren't available in IMAP protocol. This could be fixed some day. Also a support for Dovecot extension could be added some day to make this work for Dovecot as remote IMAP server.

1 0

[Dovecot] Does dsync handle client-side deletions?
by Patrick Schoenfeld 21 Jan '11

21 Jan '11

Hi there, I'm currently evaluating the idea of a multi-master setup where each node shall hold a full copy of the mailboxes. Basically the idea is to use NFS and dsync to keep those copies in sync. So I did some tests with dsync and ran into a problem. Consider the following scenario: 1) Location1 and Location2 are in sync 2) A mail gets deleted on Location1 (via IMAP) 3) dsync mirror run to sync the two locations Expected behaviour: dsync notices that the mail was deleted on Location1 and also deletes it on Location2 to get the locations in sync. What I experience, however, is: dsync notices that the mail is missing on Location2 and copies it from Location1 to get the locations in sync. (At least) In debug mode it will spit a warning: dsync(test2): Info: INBOX: highest_modseq changed: 8 != 11 dsync(test2): Info: INBOX: Couldn't keep all uids dsync(test2): Info: INBOX: Ignored 1 modseq changes dsync(test2): Warning: Mailbox changes caused a desync. You may want to run dsync again. Now the question is: Doesn't dsync handle deletions or is there something I missed? Dovecot version is 2.0.9. Thanks in advance and best Regards, Patrick

3 2

[Dovecot] utility to update indexes ?
by Jan-Frode Myklebust 21 Jan '11

21 Jan '11

We have both dovecot and a webmail application that are both modifying our users maildirs, so dovecot indexes can be out of sync when the webmail has been messing with the maildirs. We also have a webservice that report how many unread messages a user has in his inbox, which is simply counting files in the maildirs (plus caching the result, and present old values if the directory timestamps hasn´t changed). I think this checking of unread messages webservice would be much nicer on the filesystem if it instead used the dovecot indexes, so I created a small utility based on utils/idxview.c that do that: http://blag.tanso.net/code/unread.c But this woun´t work if the maildir has been modified outside of dovecot (i.e. webmail usage). Are there any simple interface I can use in this short snippet for noticing that the index is out of sync, and update it ? -jf

3 6

[Dovecot] LMTP & home, chroot, mail userdb fields.
by Lev Serebryakov 21 Jan '11

21 Jan '11

Hello, Dovecot. I'm using postfix + dovecot with pure virtual users. postfix uses standard virtual transport, and dovecot fetches such fields from userdb: chroot: "/usr/home/hosted/v-mail/%d/%n" home: "/" mail: "maildir:." Everything works Ok -- dovecot founds users' mail. Now, after upgrade to dovecot2, I want to use it LMTP server as virtual_transport in postifx. I've changed virtual_transport setting to "lmtp:unix:/var/run/dovecot/lmtp". dovecot's LMTP can not deliver messages, because it seems that it uses userdb fields in some OTHER way. Errors look like this: Jan 20 12:19:25 lmtp(38939): Info: Connect from local Jan 20 12:19:25 auth: Info: mysql: Connected to /tmp/mysql.sock (mailhost) Jan 20 12:19:25 lmtp(38939, lev(a)domain.com) Error: mkdir(./cur) in directory /var/run/dovecot failed: Permission denied (euid=30000(v-mail) egid=30000(v-mail) missing +w perm: ., euid is not dir owner) Jan 20 12:19:25 lmtp(38939, lev(a)domain.com) Error: Opening INBOX failed: Mailbox doesn't exist: INBOX Jan 20 12:19:25 lmtp(38939, lev(a)domain.com) Error: mkdir(./cur) in directory /var/run/dovecot failed: Permission denied (euid=30000(v-mail) egid=30000(v-mail) missing +w perm: ., euid is not dir owner) Jan 20 12:19:25 lmtp(38939, lev(a)domain.com) Info: XXVtE00oOE0bmAAAWL5c8Q: msgid=unspecified: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2011-01-20 12:19:25] Jan 20 12:19:25 lmtp(38939, lev(a)domain.com) Error: BUG: Saving failed to unknown storage Jan 20 12:19:25 lmtp(38939): Info: Disconnect from local: Client quit How should I change my userdb output to make both POP/IMAP and LMTP processes happy? -- // Black Lion AKA Lev Serebryakov <lev(a)serebryakov.spb.ru>

3 8

[Dovecot] Feature request - statistics for the COMPRESS imap protocol
by Ed W 21 Jan '11

21 Jan '11

Unless I have failed to read the manual for the second time in two days (?), it doesn't seem to be possible to get COMPRESS statistics at logout time from the IMAP protocol? Assuming not, then can I make a feature request that this might become available in some future version (pretty please?) Can I also just check that the meaning (as of today) of the "imap logout" statistics "%i/%o", presumably they refer to bytes over the wire, ie they are the compressed data sizes? Thanks - Dovecot 2 adds a fantastic bag of new features! Ed W

2 12

[Dovecot] Delivered-To header without +extension ?
by Per Jessen 21 Jan '11

21 Jan '11

I'm running postfix 2.6.2 and dovecot 2.0.9, and I've just switched to delivering mails to dovecot via lmtp. It looks like the Delivered-To: header is slightly different - it doesn't seem to include the +extension? Is this intentional or does the +extension somehow get in the way? /Per Jessen, Zürich

5 17

[Dovecot] Fwd: Re: Dotlock dovecot-uidlist errors / NFS / High Load
by list＠airstreamcomm.net 21 Jan '11

21 Jan '11

Stan, Thanks for the reply. In our case we have actually already done most of the work you suggested to no avail. We had rebuilt two new ntp servers that sync against two stratum 1 sources, and all our nfs clients, regardless of using dovecot, sync to those two machines. You bring up the difference between bare metal and hypervisor, and we are running these machines on vmware 4.0. All the vmware knowledge base articles tend to push us towards ntp, and since we are using centos 5.5 there are no kernel modifications that need to be made regarding timing from what we can find. I will give the ntpdate option a try and see what happens. I was also hoping to understand why the uidlist file is the only file that uses dotlock, or if there was plans to give it the option to use other locking mechanisms in the future. Thanks again! Michael -------- Original Message -------- Subject: Re: [Dovecot] Dotlock dovecot-uidlist errors / NFS / High Load Date: Thu, 20 Jan 2011 10:57:24 -0600 From: Stan Hoeppner <stan(a)hardwarefreak.com> To: dovecot(a)dovecot.org list(a)airstreamcomm.net put forth on 1/20/2011 8:32 AM: > Secondly we thought the issues were due to NTP as the time stamps vary so > widely, so we rebuilt our NTP servers and found closer stratum 1 source > clocks to synchronize to hoping it would alleviate the problem but the > dotlock errors returned after about 12 hours. We have fcntl locking set in > our configuration file, but it is our understanding from look at the source > code that this file is locked with dotlock. > > Any help troubleshooting is appreciated. >From your description it sounds as if you're ntpd syncing each of the 4 servers against an external time source, first stratum 2/3 sources, then stratum 1 sources in an attempt to cure this problem. In a clustered server environment, _always_ run a local physical box/router ntpd server (preferably two) that queries a set of external sources, and services your internal machine queries. With RTTs all on your LAN, and using the same internal time sources for every query, this clock drift issue should be eliminated. Obviously, when you first set this up, stop ntpd and run ntpdate to get an initial time sync for each cluster host. If after setting this up, and we're dealing with bare metal cluster member servers, then I'd guess you've got a failed/defective clock chip on one host. If this is Linux, you can work around that by changing the local time source. There are something like 5 options. Google for "Linux time" or similar. Or, simply replace the hardware--RTC chip, mobo, etc. If any of these cluster members are virtual machines, regardless of hypervisor, I'd recommend disabling using ntpd, and cron'ing ntpdate to run once every 5 minutes, or once a a minute, whatever it takes to get the times to remain synced, against your local ntpd server mentioned above. I got to the point with VMWare ESX that I could make any Linux distro VM of 2.4 or 2.6 stay within one minute a month before needing a manual ntdate against our local time source. The time required to get to that point is a total waste. Cron'ing ntpdate as I mentioned is the quick, reliable way to solve this issue, if you're using VMs. -- Stan

3 3

Re: [Dovecot] Problems with Upgrade from Courier
by Aaron Pettitt 21 Jan '11

21 Jan '11

I was looking at my dovecot.deliver log and it's showing that it's delivering it to the Inbox: deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: auth input: home=/home/vmail/mybridemail.com/samantha.freeze deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: maildir: data=/home/vmail/mybridemail.com/samantha.freeze/ deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: maildir++: root=/home/vmail/mybridemail.com/samantha.freeze, index=, control=, inbox=/home/vmail/mybridemail.com/samantha.freeze deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: msgid=<001a01cbb8b8$eabe89c0$c03b9d40$@net>: saved mail to INBOX However, if I login as her through Telnet, it shows that she has no mail: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1295474980] UIDs valid * OK [UIDNEXT 1] Predicted next UID b OK [READ-WRITE] Select completed. If I look in the new folder under her folder, it shows the last emails I sent this morning: -rw------- 1 vmail vmail 3564 Jan 20 10:21 1295536875.M679042P20187.mybridemail.com,W=3672 -rw------- 1 vmail vmail 3540 Jan 20 10:27 1295537272.M522196P26548.mybridemail.com,W=3649 -rw------- 1 vmail vmail 3554 Jan 20 10:39 1295537952.M462095P9353.mybridemail.com,W=3662 -rw------- 1 vmail vmail 3540 Jan 20 10:44 1295538267.M893549P15392.mybridemail.com,W=3649 It's really strange why dovecot can deliver the mail to the inbox but cannot see the inbox when trying to retrieve the mail???? From: Aaron Pettitt [mailto:apettitt@comcast.net] Sent: Thursday, January 20, 2011 10:02 AM To: 'dovecot(a)dovecot.org' Subject: Problems with Upgrade from Courier I inherited a server from a previous employee. The server crashed so it was time to move everything over to another server. We have a web mail site and I installed everything running dovecot, postfix and roundcube. After I installed it, everything worked great when I created a new user. The new user could send and receive emails with no issues. I then copied the home directory over from the other server and ran the courier-dovecot migration script. It created the subscription files and the dovecot-uidlist files in each user (about 1000 total users). When I login as one of the existing users, it says that there is no mail in the mailbox. However, if I look at the user's cur and new folders, there is mail in those folders. If I send a new mail to the user, it does not show up in their inbox. If I look in their new folder, the new mail that I sent was delivered to that folder but it does not show up in their inbox. I've tried going to dovecot directly through telnet with the same results. I've been stuck for 2 days now so any help is greatly appreciated. Below is my dovecot.conf with all the comments removed. Thanks all! protocols = imap imaps disable_plaintext_auth = no log_path = '/var/log/dovecot/error.log' info_log_path = '/var/log/dovecot/info.log' log_timestamp = "%m/%d/%Y %H:%M:%S " #mail_location = maildir:~/ mail_location = maildir:/home/vmail/%d/%n/ mail_privileged_group = mail mail_debug = yes protocol imap { } protocol pop3 { } protocol managesieve { sieve_storage=~/sieve } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster(a)mybridemal.com mail_plugins = cmusieve global_script_path = /home/vmail/globalsieverc } auth_verbose = yes auth_debug = no auth_debug_passwords = no auth default { passdb sql { args = /etc/dovecot/dovecot-sql.conf } args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } user = root path = /var/run/dovecot/auth-master mode = 0600 user = vmail #group = } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } }

2 4

[Dovecot] Populating mailbox dir
by Mauricio Tavares 20 Jan '11

20 Jan '11

Let's say you created a (virtual) mail user but have not created the user mail directory. When you first try to access it, the mail directory is created and populated: ls -a /var/spool/vmail/domain.com/raub/ . dovecot.index.log dovecot-uidvalidity.4d375a39 tmp .. dovecot-uidlist new .Trash cur dovecot-uidvalidity subscriptions Who populates/creates the initial files and folders in the user mailbox? Dovecot? The mail client accessing it through imap? Elvis? If dovecot, is there a way to customize it a bit, so a few extra folders will also be thrown there?

3 2

[Dovecot] Imap Error
by Jason Liedtke 20 Jan '11

20 Jan '11

This morning I have a Outlook 2007 user who getting the error and I am unsure how to fix it. Cannot open this item. The server responded: "Error in IMAP command UID FETCH: Invalid uidset' DoveCot v 1.2.9 # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-27-generic-pae i686 Ubuntu 10.04.1 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/ mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(pop3): imap_client_workarounds(managesieve): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): managesieve_logout_format(default): bytes=%i/%o managesieve_logout_format(imap): bytes=%i/%o managesieve_logout_format(pop3): bytes=%i/%o managesieve_logout_format(managesieve): bytes ( in=%i : out=%o ) namespace: type: private separator: / inbox: yes list: yes subscriptions: yes lda: postmaster_address: hostname: auth_socket_path: /var/run/dovecot/auth-master mail_plugins: quota sieve log_path: info_log_path: syslog_facility: mail auth default: username_format: %Lu passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir quota_rule: *:bytes=20M sieve: ~/sieve/.dovecot.sieve sieve_dir: ~/sieve sieve_global_path: /var/mail/default.sieve sieve_before: /var/mail/sieve/global sieve_extensions: +imapflags

2 1