- dovecot - dovecot.org

[Dovecot] cyrus2dovecot script converts mailboxes to bigger sizes
by Igor Zinovik 13 Sep '12

13 Sep '12

Hello, list. I'm asking question not directly connected with dovecot. I'm planning a migration from cyrus-imapd to dovecot and i'm using cyrus2dovecot script (by Freie Universität Berlin) to convert mailboxes of users, but i'm wondering why size of mailbox in Maildir++ is so much bigger than mailbox in cyrus format after conversion: linux-a9qw:~/ # du -sh /mnt/imap/z/user/zinovik /srv/vmail/petrsu.ru/z/zinovik/Maildir 238M /mnt/imap/z/user/zinovik 1.2G /srv/vmail/mydom.ru/z/zinovik/Maildir I was planning to implement quota for mailboxes about 1 gigabyte, but after conversion I would not be able to receive messages to my own box, because i'm overquota. I think the only way would be to set quota up to 15 GB since some of users have mailbox with 12GB after conversion, but I have no such disk capacity to allow more than 1500 users have 15GB quota. Would it be practical solution to compress all files after conversion and enable zlib support in dovecot?

1 1

[Dovecot] doveadm director map : User listing returned failure
by Ramón Frontera 13 Sep '12

13 Sep '12

Hello we are running Dovecot 2.1.9. We have 2 proxies with director and 4 mail servers. We have the user information in a LDAP. I try to do: # doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed user mail server ip expire time <unknown> x.x.x.x 2012-09-12 10:52:17 <unknown> x.x.x.x 2012-09-12 10:55:53 in the dovecot log i have found: dovecot: auth: Error: Trying to iterate users, but userdbs don't support it What can I do to solve this problem? Thanks in advance. -- ------------------------------------------ Ramon Frontera Gallardo Universitat de les Illes Balears

2 1

[Dovecot] Dovecot 1.x POP3 to Dovecot 2.x POP3 mbox to Maildir best transfer program/method
by Tim E. 12 Sep '12

12 Sep '12

I have a box running Dovecot 1.x POP3 only server using mbox format. I want to port the mail over to a Centos 6.3 box running a Dovecot 2.x POP3 only server using Maildir format. The main thing I want to do is support XUID/read/unread status so the transfer is transparent for the end user. I have been able to achieve transfers from the mbox format to Maildir but loose the XUID/read/unread status so that the Thunderbird client always re-downloads all the mail as if it were new. Ive googled and I have tried the following: dsync : dovecot 1.x server is too old and doesnt support the command? mb2md.pl: transfers the mbox to maildir and creates file dovecot-uidlist but the Thunderbird client redownloads the files virtualmin copy-mailbox: transfers the mailbox to Maildir but doesnt reproduce the dovecot indexes. Any thoughts would be appreciated. I would like to produce a howto of the process when I am done.

2 6

[Dovecot] TIMO HELP! director ring wont stay connected
by Kelsey Cummings 12 Sep '12

12 Sep '12

I've had 2x director ring up and running with production load on 2.1.8 with around 10,000 active connections for two weeks and everything has been working great - until this morning. There isn't anything obvious in the logs beyond the fact that the director connections started bouncing. It was not resolved by reloads or restarts or an upgrade to 2.1.9 (only the directors.) I've dropped one of the servers out of the ring to prevent suffering but this is a less than ideal situation. Any idea what is going on? Load today is consistent with low weekend load since it is a major US holiday so this wouldn't appear to be a load related issue. auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 10000 director_mail_servers = 1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4 director_servers = a.director.foo b.director.foo disable_plaintext_auth = no doveadm_proxy_port = 1842 login_trusted_networks = 10.0.0.1 mbox_write_locks = fcntl passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 20103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_ca = </etc/dovecot/ssl/gd_bundle.crt ssl_cert = </etc/dovecot/ssl/imap.sonic.net.crt ssl_key = </etc/dovecot/ssl/imap.sonic.net.key ssl_parameters_regenerate = 1 days syslog_facility = local0 protocol imap { ssl_ca = </etc/dovecot/ssl/gd_bundle.crt ssl_cert = </etc/dovecot/ssl/imap.sonic.net.crt ssl_key = </etc/dovecot/ssl/imap.sonic.net.key } protocol pop3 { ssl_ca = </etc/dovecot/ssl/pop.sonic.net.ca-bundle ssl_cert = </etc/dovecot/ssl/pop.sonic.net.crt ssl_key = </etc/dovecot/ssl/pop.sonic.net.key } >Sep 3 09:22:42 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:22:45 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:49 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:53 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:22:54 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/left disconnected >Sep 3 09:22:59 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/left disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:02 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:23:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User foo host lookup failed: Timeout - queued for 47 secs (Ring synced for 30 secs, weak user, user refreshed 47 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bar host lookup failed: Timeout - queued for 38 secs (Ring synced for 30 secs, weak user, user refreshed 38 secs ago) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: director: User bla2 host lookup failed: Timeout - queued for 30 secs (Ring synced for 30 secs) >Sep 3 09:23:32 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:23:32 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:23:32 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:02 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/out disconnected before handshake finished >Sep 3 09:24:02 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:05 a.director. a dovecot: director: Warning: Delaying new user requests until ring is synced >Sep 3 09:24:32 a.director. a dovecot: director: Warning: Ring is synced, continuing delayed requests >Sep 3 09:24:41 b.director. b dovecot: director: Error: Director 10.10.10.71:9321/right disconnected >Sep 3 09:24:41 b.director. b dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:24:41 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:24:41 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director left >Sep 3 09:25:11 b.director. b dovecot: director: Error: User hash 2285697953 is being redirected to two hosts: 10.10.10.39 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689467) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 623192092 is being redirected to two hosts: 10.10.10.76 and 10.10.10.39 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 b.director. b dovecot: director: Error: User hash 1683990717 is being redirected to two hosts: 10.10.10.43 and 10.10.10.76 (old_ts=1346689481,handshaking,recv_ts=1346689468) >Sep 3 09:25:12 a.director. a dovecot: director: Error: Director 10.10.10.37:9321/right disconnected >Sep 3 09:25:12 a.director. a dovecot: director: Warning: director: Couldn't connect to right side, we must be the only director lef -- Kelsey Cummings - kgc(a)corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407

3 10

[Dovecot] Expire plugin with multiple mail servers
by Ramón Frontera 12 Sep '12

12 Sep '12

Hello we are running Dovecot 2.1.9 and I want to configure expire plugin. We have 2 proxies with director and 4 mail servers. I have configured expire plugin on a single mail server and it works Ok. I don't konw how to configure with 4 mail servers and 2 proxies. My first idea is create on each mailserver one sqlite database and configure the expire plugin on proxy, but I don't know if this is a good idea. I don't know if the database should be in one proxy, and I don't know which server has to do the "doveadm expunge" command Do you have any idea how is the best way to configure expire plugin on this scenario? Thanks in advance. -- Ramon Frontera Gallardo Universitat de les Illes Balears

3 3

[Dovecot] dovecot/log not running
by Mark Hinch 12 Sep '12

12 Sep '12

I'm running 2.1.7 as downloaded/installed by Debian (and then appropriately setup for my config, ...). It almost works. I can send/receive email from thunderbird, squirrelmail, ... However there are two anomalies: 1. dovecot/log is not running. After restarting dovecot (and waiting 30 seconds), I have this in the log files: Sep 6 17:42:16 hinch4 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) Sep 6 17:42:56 hinch4 dovecot: master: Error: service(anvil): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(ssl-params): Initial status notification not received in 30 seconds, killing the process Sep 6 17:42:56 hinch4 dovecot: master: Error: service(log): child 31286 killed with signal 9 After 30 seconds, a "ps -ef | grep dovecot" yields: dovenull 1858 20223 0 18:56 ? 00:00:00 dovecot/imap-login root 1859 20223 0 18:56 ? 00:00:00 dovecot/auth -w mhinch 1860 20223 0 18:56 ? 00:00:00 dovecot/imap root 1862 14520 0 18:56 pts/3 00:00:00 grep dovecot root 20223 1 0 18:54 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovenull 25144 20223 0 18:55 ? 00:00:00 dovecot/imap-login dovecot 25147 20223 0 18:55 ? 00:00:00 dovecot/anvil root 25148 20223 0 18:55 ? 00:00:00 dovecot/config dovecot 25161 20223 0 18:55 ? 00:00:00 dovecot/auth root 25174 20223 0 18:55 ? 00:00:00 dovecot/ssl-params mhinch 25208 20223 0 18:55 ? 00:00:00 dovecot/imap Seems like everything is there except dovecot/log. ssl-params and anvil came back (?!?). A "doveadm log find" dumps this in the log: Sep 6 18:52:24 hinch4 dovecot: doveadm: Debug: This is Dovecot's debug log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: This is Dovecot's info log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Warning: This is Dovecot's warning log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Error: This is Dovecot's error log (1346975543) Sep 6 18:52:24 hinch4 dovecot: doveadm: Fatal: This is Dovecot's fatal log (1346975543) However, further debugging that I try to do (see next problem) does not result in ANY further input to the log files, even though I have all known logging turned on. So I *think* that logging is not actually running. If I can get that started, then I'll take a crack at the next problem myself ... 2. However, as long as I have your attention, my next problem is that it seems like dovecot/imap-login hangs(?). After a few hours, I have something like: # ps -ef | grep dovecot dovecot 530 31186 0 18:49 ? 00:00:00 dovecot/auth root 13277 31186 0 17:47 ? 00:00:00 dovecot/config dovenull 13664 31186 0 18:44 ? 00:00:00 dovecot/imap-login root 20207 14520 0 18:54 pts/3 00:00:00 grep dovecot dovenull 27599 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27601 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27603 31186 0 18:45 ? 00:00:00 dovecot/imap-login dovenull 27607 31186 0 18:45 ? 00:00:00 dovecot/imap-login root 31186 1 0 17:42 ? 00:00:00 /usr/sbin/dovecot -c /etc/dovecot/dovecot.conf dovecot 31289 31186 0 17:42 ? 00:00:00 dovecot/anvil with exactly 100 imap-login processes. I'm the only meaningful user on the machine (unless somebody is imap DoS-ing me), so this does not make much sense. Obviously thunderbird is polling the machine and each time is leaving around a imap-log process. As far as I can tell, thunderbird successfully logs in (i.e., I get mail) though I can't be sure whether these are successful logins or not (no logging!). What hangs imap-log processes for hours at a time? After a while of course, dovecot will not start any new imap-login processes so Thunderbird cannot get any more mail until I restart dovecot. Then everything is OK for another 12 hours or so. Ideas on either problem? -Mark

3 3

[Dovecot] Found bug on doing IMAP search with UTF8 chars
by Weinreich, Nico 12 Sep '12

12 Sep '12

Hi! I'm using Roundcube 0.9git as mail frontend and have installed Dovecot 2.1.7 on Debian Squeeze. When searching for mail via Roundcube and the keyword contains "special" chars (like german umlauts), then Dovecot seems to die when the folder does not contain a message which contains this keyword, i.e. folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht" => Roundcube shows that messages folder contains message /w subject "Dies ist eine Nachricht" => search for "Nachricht123" => Roundcube shows error that no messages where found folder contains message /w subject "Auflösung der Frage" => search for "Lösung" => Roundcube shows that messages folder contains message /w subject "Auflösung der Frage" => search for "Lösung123" => Roundcube shows error saying: "Connection to storage server failed" Looking into /var/log/mail.log shows: Sep 5 11:43:32 mail dovecot: imap(abc(a)def.de): Panic: search key not utf8: L.sung123 dsr_sub_allocation Sep 5 11:43:32 mail dovecot: imap(abc(a)def.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x4478a) [0 x7f065ba8e78a] -> /usr/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7f065ba8e7d6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f065ba655ef ] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a3cb) [0x7f065bd6f3cb] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a8aa) [0x7f065bd6f8 aa] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x78641) [0x7f065bd4d641] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_fore ach+0x3c) [0x7f065bd4d7bc] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header+0x29) [0x7f065ba78eb9] -> /usr/lib/dovecot/libdovecot-st orage.so.0(+0x9a0cf) [0x7f065bd6f0cf] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9b160) [0x7f065bd70160] -> /usr/lib/dovecot/libdovecot- storage.so.0(index_storage_search_next_nonblock+0x48) [0x7f065bd706e8] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonbl ock+0x27) [0x7f065bd51a07] -> dovecot/imap() [0x41668f] -> dovecot/imap(imap_search_start+0xd8) [0x416ad8] -> dovecot/imap(cmd_sort+0x1e6) [ 0x40e636] -> dovecot/imap(command_exec+0x3d) [0x41157d] -> dovecot/imap() [0x4104fe] -> dovecot/imap(client_handle_input+0x135) [0x410805] - > dovecot/imap(client_input+0x5f) [0x41112f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f065ba9a926] -> /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f065ba9b95f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f065ba9a8c8] -> /usr/li b/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f065ba87053] -> dovecot/imap(main+0x2a4) [0x419354] -> /lib/libc.so.6(__libc_start_ma in+0xfd) [0x7f065b706c8d] -> dovecot/imap() [0x408e79] Huh? This error only occurs when I search for a keyword, that contains probably non-ASCII-US chars (like umlauts, euro sign EUR ...) and no message was found in that folder. When there is any message which contains this keyword, then Roundcube is fine and there error above cannot be found in mail.log. Can anybody confirm this behaviour? Regards

3 10

[Dovecot] Resynchronization of indexes (or Warning: Maildir Scanning <dir> took <X> seconds)
by Angel L. Mateo 12 Sep '12

12 Sep '12

Hello, We have a dovecot servers farm with storage in NFS and indexes in local files. Under heavy load conditions we a log of messages in the form: Sep 3 12:35:49 myotis32 dovecot: imap(<imapuser>): Warning: Maildir: Scanning <maildir cur directory> took 62 seconds (9516 readdir()s, 0 rename()s to cur/, why=0x80) We have check that messages are directed with lmtp to the same server the user (scenario 3 at http://wiki2.dovecot.org/NFS with director in front of backend servers). So my question is why is dovecot using these readdirs operations? why it is scanning the whole cur directory of the user? I guess this is an index resynchronization, but I don't know could it be the cause. PS: I have attached my dovecot -n output

2 4

[Dovecot] dsync issue: "Server sent invalid input: Error parsing input: Unexpected ')'"
by Gábor Lénárt 12 Sep '12

12 Sep '12

Hi, I'm trying to use dsync (dovecot version 2.1.9) for IMAP only per-user migration from other IMAP servers to my shiny new one. One worked quite well, but there is a problem with an old IMAP server: dsync(migtest@office): Debug: imapc(imap-sunw-old.servers.intra:143): Looking up IP address dsync(migtest@office): Debug: imapc(imap-sunw-old.servers.intra:143): Connecting to 192.168.10.100:143 dsync(migtest@office): Error: imapc(imap-sunw-old.servers.intra:143): Server sent invalid input: Error parsing input: Unexpected ')' dsync(migtest@office): Debug: imapc(imap-sunw-old.servers.intra:143): Disconnected dsync(migtest@office): Error: imapc: Command failed: Disconnected from server dsync(migtest@office): Error: user migtest@office: Initialization failed: Initializing mail storage from mail_location setting failed: imapc: LIST failed: Internal error occurred. Refer to server log for more information. [2012-09-10 11:49:02] dsync(migtest@office): Fatal: User init failed What can cause this? "By hand" test with telnet'ing says: dovecot-test:~$ telnet imap-sunw-old.servers.intra 143 Trying 192.168.10.100... Connected to imap-sunw-old.servers.intra. Escape character is '^]'. * OK [CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS CHILDREN BINARY UNSELECT SORT LANGUAGE XSENDER X-NETSCAPE XSERVERINFO X-SUN-SORT X-SUN-IMAP X-ANNOTATEMORE X-UNAUTHENTICATE XUM1 AUTH=PLAIN] imap-sunw-old.servers.intra IMAP4 service (Sun Java(tm) System Messaging Server 6.3-5.02 (built Oct 12 2007; 32bit)) (this is one line of course) Is it possible that the problem about ')' is caused by this line? Or is it tottaly different issue? I just think this, because the the major difference between this and the working server on migration is to have ')' and '(' at the imap "greeting", the working server does not have those, and the error message is about something with ')'. The command was the following (no configuration in dovecot for this, only this command is given at shell level - currently mainly for testing if it works or not); doveadm -D -v -o imapc_features=rfc822.size \ -o imapc_host=192.168.10.100 \ -o imapc_user=migtest@office -o imapc_password=SECRET backup -R -f \ -u migtest@office imapc: This was OK with a dovecot v1 as the "source" (as I've written) but has the problem I've desribed with that "old" sun messaging server as the source. I've also used tcpdump then wireshark's "follow TCP stream" on it: the communication seems to be just that "greeting" line, then my (target) dovecot sends FIN. That "old" server works otherwise since years with various IMAP clients. The log of my server does not contain too much just the db lookups, which works nicely (the same for working and non-working source IMAP servers): Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: prefetch(migtest@office): passdb didn't return userdb entries, trying the next userdb Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest@office): user search: base=cn=mail,dc=office,dc=intra scope=subtree filter=(&(objectClass=mailUser)(uid=migtest@office)) fields=uid Sep 10 11:49:02 dovecot-test dovecot: auth: Debug: ldap(migtest@office): result: uid=migtest@office Any help is welcome, - Gábor

2 2

[Dovecot] [PATCH] Generalize HMAC implementation
by Florian Zeitz 11 Sep '12

11 Sep '12

Hello everyone and Timo in particular, about a year ago I implemented a SHA-1 variant of the HMAC(-MD5) present in Dovecot. I had always disliked this a bit, because it replicates a lot of code. This patch generalizes the HMAC function to take a hash_method struct as parameter, and changes existing code which uses the "old" HMAC function to use this new one. I'm not really sure this is actually a good idea, but I still felt I should provide the code in case you would want to merge it upstream. Attached is the patch as a hg export based on the revision of dovecot-2.2 current at the time of writing. Regards, Florian Zeitz

2 2