- dovecot - dovecot.org

[Dovecot] Lack of external documentation?
by Steve Campbell 16 Mar '12

16 Mar '12

Firstly, this isn't meant to be critical, and I realize the subject line probably suggest criticism, so... I was sort of forced into using dovecot as my imap/pop server due to upgrading 3 versions of OS on my mail servers. So far, that's not bad. What surprises me is that one of the first things I usually do whenever I start using different software is to purchase a book that seems to suit me. Searching all of the common places like amazon, ebay, etc for manuals turned up little to nothing on dovecot. I'm wondering why and is this so new that people just haven't written books about it yet? The one thing I'm a little critical of, though, is that trying to make heads or tails of dovecot by following the online documentation is a little problematic. I'm constantly jumping to another page and then back to the original page, and for the most part, I just don't know enough about it all yet to know what I'm looking for. Does anyone know of any manuals/books that have been written that might introduce me to most of the stuff in dovecot? So far, the list has been great, but once the "dsync" threads started popping up, I find there's even more I don't know about. Thanks for all the help I've received so far and I think I'm really going to like dovecot. Once I get the hang of it, I'll probably reduce the amount of noise on the list by half. steve campbell

8 8

[Dovecot] v2.1.2 released
by Timo Sirainen 16 Mar '12

16 Mar '12

http://dovecot.org/releases/2.1/dovecot-2.1.2.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.2.tar.gz.sig There are a ton of proxying related improvements in this release. You should now be able to do pretty much anything you want with Dovecot proxy/director. This release also includes the initial version of dsync-based replication. I'm already successfully using it for @dovecot.fi mails, but it still has some problems. See http://dovecot.org/list/dovecot/2012-March/064243.html for some details how to configure it. + Initial implementation of dsync-based replication. For now this should be used only on non-critical systems. + Proxying: POP3 now supports sending remote IP+port from proxy to backend server via Dovecot-specific XCLIENT extension. + Proxying: proxy_maybe=yes with host=<hostname> (instead of IP) works now properly. + Proxying: Added auth_proxy_self setting + Proxying: Added proxy_always extra field (see wiki docs) + Added director_username_hash setting to specify what part of the username is hashed. This can be used to implement per-domain backends (which allows safely accessing shared mailboxes within domain). + Added a "session ID" string for imap/pop3 connections, available in %{session} variable. The session ID passes through Dovecot IMAP/POP3 proxying to backend server. The same session ID is can be reused after a long time (currently a bit under 9 years). + passdb checkpassword: Support "credentials lookups" (for non-plaintext auth and for lmtp_proxy lookups) + fts: Added fts_index_timeout setting to abort search if indexing hasn't finished by then (default is to wait forever). - doveadm sync: If mailbox was expunged empty, messages may have become back instead of also being expunged in the other side. - director: If user logged into two directors while near user expiration, the directors might have redirected the user to two different backends. - imap_id_* settings were ignored before login. - Several fixes to mailbox_list_index=yes - Previous v2.1.x didn't log all messages at shutdown. - mbox: Fixed accessing Dovecot v1.x mbox index files without errors.

4 4

[Dovecot] problems with SSH-based clustering dovecot 2.1.1
by Michescu Andrei 16 Mar '12

16 Mar '12

Hello, I'm using dovecot 2.1.1 with vpopmail 5.4.30 with multiples domains and I have problems setting up synchronization in between multiple computers. All act like master (my clients can connect to any of the them and read their emails either via POP3 either via IMAP, inbound email gets on any of the machines). Each machine is on a different continent, there is no shared drive in between and the synchronization is supposed to be asynchronous via cron scripts. To simplify the case, let's consider 2 machines (mx1.a and mx2.a) with 3 virtual domains (a and b and c). On both machine domain a is the default domain (needs only username to connect to imap). Initially I synchronize mx1.a with mx2.a using rsync. I check that I can login using dovecot. I tried any of the following commands to synchronize the 2 machines: mx1.a$ doveadm -Dv sync -u user1@a -f ssh mx2.a doveadm dsync-server -u user1@a mx1.a$ doveadm -Dv sync -u user1@a ssh mx2.a doveadm -u user1@a mx1.a$ doveadm -Dv sync -u user1@a user1@a The only thing that happens is that the on each machine the folders get doubled with some random extension (eg. Inbox becomes Inbox_3e3ff3g3gb3bb3b22). Also, another bug, if there is a domain setup as default (auth_default_realm) dsync simply ignores the specified -u <user> and attempts to sync the first email in the default domain. Please advise. # 2.1.1: /etc/dovecot/dovecot/dovecot.conf # OS: Linux 2.6.38-b i686 Slackware 13.0.0.0.0 auth_debug = yes auth_debug_passwords = yes auth_default_realm = a first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 listen = * log_path = /dev/stderr login_greeting = A login_trusted_networks = 192.168.20.64/26 mail_debug = yes mail_gid = vchkpw mail_location = maildir:~/Maildir mail_privileged_group = vchkpw mail_uid = vpopmail passdb { driver = vpopmail } protocols = imap pop3 service auth-worker { unix_listener auth-worker { user = vpopmail } user = vpopmail } service auth { user = vpopmail } service imap-login { user = vpopmail } service pop3-login { user = vpopmail } ssl = no userdb { driver = vpopmail }

5 24

[Dovecot] Compiler warnings in dovecot-2.1.2 and pigeonhole 0.3.0
by Tom Talpey 15 Mar '12

15 Mar '12

I'm seeing a few warnings emitted when building for x86. They're pretty obvious, but if you want the configure options etc, I can provide those. In Dovecot 2.1.2 (I also see some of these in 2.1.1): 1) src/lib-index/mail-cache-fields.c (comparison between two last_used fields) mail-cache-fields.c: In function 'mail_cache_header_fields_read': mail-cache-fields.c:406: warning: comparison between signed and unsigned 2) src/director/user-directory.c (comparison with ioloop_time) user-directory.c: In function 'user_directory_user_is_recently_updated': user-directory.c:147: warning: comparison between signed and unsigned 3) src/replication/replicator/replicator-brain.c (comparison with ioloop_time) replicator-brain.c: In function 'doveadm_replicate': replicator-brain.c:113: warning: comparison between signed and unsigned 4) src/replication/replicator/replicator-queue.c (comparison with ioloop_time) replicator-queue.c: In function 'replicator_queue_pop': replicator-queue.c:201: warning: comparison between signed and unsigned In Pigeonhole 0.3.0: 5) src/managesieve-login/client-authenticate.c (passing size_t * not uoff_t *) client-authenticate.c: In function 'managesieve_client_auth_read_response': client-authenticate.c:214: warning: passing argument 3 of 'i_stream_get_size' from incompatible pointer type

3 3

[Dovecot] 2.1.1: doveadm backup errors
by e-frog 15 Mar '12

15 Mar '12

Hello Timo, I'm experimenting with 'doveadm backup' on 2.1.1 (latest hg, full dovecot -n output attached) and haven't managed to get it working. This is what I have done: 1. Create the directory /tmp/backup which is empty 2. Run doveadm -v backup -u testuser(a)ubuntu-test.localdomain mdbox:/tmp/backup/ Then I see the following errors: doveadm -v backup -u testuser(a)ubuntu-test.localdomain mdbox:/tmp/backup/ dsync(testuser(a)ubuntu-test.localdomain): Error: Can't delete mailbox INBOX: INBOX can't be deleted. dsync(testuser(a)ubuntu-test.localdomain): Info: INBOX: only in dest (guid=9e4b88178b905f4f456e0000381555a6) dsync(testuser(a)ubuntu-test.localdomain): Info: INBOX: only in source (guid=bd05451f2fbb574d40600000ec8d17cd) dsync(testuser(a)ubuntu-test.localdomain): Error: Trying to open a non-listed mailbox with guid=9e4b88178b905f4f456e0000381555a6 dsync(testuser(a)ubuntu-test.localdomain): Error: msg iteration failed: Couldn't open mailbox 9e4b88178b905f4f456e0000381555a6 dsync(testuser(a)ubuntu-test.localdomain): Error: Trying to open a non-listed mailbox with guid=9e4b88178b905f4f456e0000381555a6 dsync(testuser(a)ubuntu-test.localdomain): Error: Mailbox INBOX changed its GUID (bd05451f2fbb574d40600000ec8d17cd -> 9e4b88178b905f4f456e0000381555a6) dsync(testuser(a)ubuntu-test.localdomain): Error: msg iteration failed: Couldn't open mailbox bd05451f2fbb574d40600000ec8d17cd dsync(testuser(a)ubuntu-test.localdomain): Error: Mailbox INBOX changed its GUID (bd05451f2fbb574d40600000ec8d17cd -> 9e4b88178b905f4f456e0000381555a6) It somehow finds INBOX in destination however the backup directory is newly created and empty. Thanks, e-frog

2 5

[Dovecot] Multiple locations, 2 servers - planning questions...
by Charles Marcus 15 Mar '12

15 Mar '12

Hello all/Timo, Up until now, my main Clients office has consisted of a single location, and I have never had to deal with the situation of multiple locations for a single company. They have just told me that they are acquiring an additional floor at a building that is about 4 minutes away - but obviously far enough away that I now have to deal with supporting users in the same domain but at two disparate physical locations. These two locations will be connected via a private Gb ethernet connection, and each location will have its own internet connection (I think - still waiting on some numbers to present to the owner to see what he wants to do in that regard, but that will be my recommendation), so bandwidth for replication won't be an issue. I have a couple of months to get this done, and I am already planning on hiring Timo's new commercial support company to help with the final and actual design and implementation, but obviously first I need to know what my actual options are. Just a rough idea of what I'd like to do is: 1. Set up one dovecot server at each location (these will be VMs), so users at each location are accessing the local server for that office 2. Full replication between the two for the mail/indexes, and configure them such that each can act as a failover for the other in case one goes down for whatever reason This is my first/main question... I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? Or, if it is going to take more work than my client is willing to pay for (I'm hoping not, since you said it was on the map for 2.1, not 2.2+), maybe the notify plugin could be leveraged in some way to provide something 'close enough' until it is fully implemented in dsync? On that note (something 'close enough' until dsync fully supports this natively), would setting up a dsync cron job, say, every 5 or 10 minutes, be asking for trouble? Our mail server is not all that busy, really, so in 5 or 10 minutes, there wouldn't be many changes at all. If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? and 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Thanks to any/all for reading this far and for any thoughts, suggestions and/or ideas... -- Best regards, Charles

9 19

[Dovecot] director lmtp -> smtp problem
by Jim Lawson 15 Mar '12

15 Mar '12

Hi Timo & Dovecot users, We have a 2-node director setup which front-ends for 4 nodes which share a clustered filesystem (GFS). All nodes run Dovecot 2.0.18. Approximately 40k users, but typically only a few thousand active at any time. The director nodes run sendmail, which deliver mail "locally" using LMTP to the director, which then feeds to SMTP on the real servers (also sendmail.) Why sendmail? Because procmail is used for mail filtering and as the delivery agent. Here's the problem, on the director: Mar 14 20:40:08 imapdir2 dovecot: lmtp(10692): Connect from local Mar 14 20:40:38 imapdir2 dovecot: lmtp(10692): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input ->eof) Mar 14 20:40:38 imapdir2 dovecot: lmtp(10692): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3d99a) [0x7f79156c499a] -> /usr/lib/doveco t/libdovecot.so.0(+0x3d9e6) [0x7f79156c49e6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f791569df8f] -> dovecot/lmtp() [0x406e77] -> /usr/l ib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xd4) [0x7f79156d0044] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x5b) [0x7f79156d 0c3b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f79156cfca8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f7915 6bdfc3] -> dovecot/lmtp(main+0x154) [0x403f84] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f7914ef8cdd] -> dovecot/lmtp() [0x403d69] Mar 14 20:40:38 imapdir2 sendmail[6905]: q2D8KodI018432: SYSERR(root): timeout writing message to localhost: Broken pipe Most mail goes through OK, but some messages do not and end up queued until they run into the queue time limit. So far as I have been able to tell, all of the messages have this failure when the following conversation takes place between sendmail (on director), the Dovecot LMTP proxy, and sendmail on the backend node (SMTP): (names mangled to protect the guilty) (first, sendmail -> director LMTP) > [root@imapdir2 ~]# sendmail -qIq2EFZt1p004708 -v > > Running /var/spool/mqueue/qd2/q2EFZt1p004708 (sequence 1 of 1) > <ntssdfwe(a)pobox2.uvm.edu>... Connecting to > /var/lib/dovecot/lmtp-socket via cyrusv2... > 220 imapdir2.uvm.edu Dovecot LMTP ready > >>> LHLO imapdir2.uvm.edu > 250-imapdir2.uvm.edu > 250-8BITMIME > 250-ENHANCEDSTATUSCODES > 250 PIPELINING > >>> MAIL From:<owner-dasfqeiuasd*Abrasf*-Gorpwe**UVM*-EDU(a)LIST.UVM.EDU> > 250 2.1.0 OK > >>> RCPT To:<ntssdfwe> > >>> DATA > 250 2.1.5 OK > 354 OK > timeout writing message to localhost: Broken pipe > <ntssdfwe(a)pobox2.uvm.edu>... Deferred > Closing connection to localhost The conversation between the director (LMTP) and the backend (sendmail SMTP) goes like this: > 250-penguinc.uvm.edu Hello imapdir2.uvm.edu [132.198.100.150], pleased > to meet you > 250-ENHANCEDSTATUSCODES > 250-PIPELINING > 250-8BITMIME > 250-SIZE 10485760 > 250-ETRN > 250-AUTH DIGEST-MD5 CRAM-MD5 > 250-DELIVERBY > 250 HELP > MAIL FROM:<owner-dasfqeiuasd*Abrasf*-Gorpwe**UVM*-EDU(a)LIST.UVM.EDU> > 250 2.1.0 <owner-dasfqeiuasd*Abrasf*-Gorpwe**UVM*-EDU(a)LIST.UVM.EDU>... > Sender ok > RCPT TO:<ntssdfwe> > 552 5.2.2 User ntssdfwe mailbox is full At this point Dovecot should return the failed RCPT TO: status back to sendmail over LMTP, but instead it sits there (waiting for a timeout to expire?) and eventually dies. doveconf -n output: # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.4.2.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.2 (Santiago) base_dir = /var/run/dovecot/ default_client_limit = 6000 default_process_limit = 10240 director_mail_servers = penguina.uvm.edu penguinb.uvm.edu penguinc.uvm.edu penguind.uvm.edu director_servers = imapdir1.uvm.edu imapdir2.uvm.edu lmtp_proxy = yes login_trusted_networks = [REDACTED] passdb { args = proxy=y nopassword=y protocol=smtp driver = static } service anvil { client_limit = 40000 } service auth { client_limit = 45960 unix_listener auth-userdb { group = mail mode = 0660 user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director service_count = 0 } service imap { process_limit = 10240 vsz_limit = 1 G } service lmtp { client_limit = 1 inet_listener lmtp { port = 24 } unix_listener /var/lib/dovecot/lmtp-socket { group = root mode = 0600 user = root } } service pop3-login { executable = pop3-login director service_count = 0 } service pop3 { process_limit = 5000 } shutdown_clients = no ssl_cert = <[REDACTED].pem ssl_key = <[REDACTED].key userdb { driver = passwd } verbose_proctitle = yes version_ignore = yes protocol lmtp { auth_socket_path = director-userdb } protocol imap { mail_max_userip_connections = 100 } Hope you can help, Jim Lawson

2 6

[Dovecot] v2.0.19 released
by Timo Sirainen 15 Mar '12

15 Mar '12

http://dovecot.org/releases/2.0/dovecot-2.0.19.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.19.tar.gz.sig Hopefully one of the last v2.0.x releases. - IMAP: ENABLE CONDSTORE/QRESYNC + STATUS for a mailbox might not have seen latest external changes to it, like new mails. - imap_id_* settings were ignored before login. - doveadm altmove did too much work sometimes, retrying moves it had already done. - mbox: Fixed accessing Dovecot v1.x mbox index files without errors.

1 0

[Dovecot] sieve.before script is taking preceedence over user defined rules
by Alexis Lelion 15 Mar '12

15 Mar '12

Hello, In my current setup, I have a spam filter upstream that adds a specific header - X-Spam-Level on every incoming mail. Based on this level, the mail will be moved to the user spam folder using sieve by doing "fileinto :create 'spam';" Unfortunately, some legitimate email may end up in this spam folder, so I have kind of a whitelist that performs an explicit keep over specific trusted domains. So, my complete spam filtering rule is : if address :domain :contains "From" ["mycompany.tld", "trusted.tld" ]{ keep; elseif header :contains "X-Spam-Level" ["0","1","2"] { fileinto :create "__spam__"; } This rule is stored in "/var/lib/dovecot/sieve/before.sieve", which is my "sieve_before" file as defined in /etc/dovecot/conf.d/90-sieve.conf This works as expected except that it doesn't take into account users filtering for domains that were matched for the explicit keep. For example, I have the following rule : if address :domain "From" "trusted.tld" { fileinto "trusted" } But mail coming from that domain are still delivered in my mailbox. Is there something I'm missing here? I guess yes, otherwise it would work as I want ^_^ Any help/comment is appreciated Thanks! Alexis

2 4

Re: [Dovecot] doveadm -A stops processing at first uid<first_valid_uid
by Joseph Tam 15 Mar '12

15 Mar '12

On Sun, 4 Mar 2012, Timo Sirainen writes: > > I would like to run various doveadm commands that involves all (mail) users like > > > > doveadm expunge -A mailbox Trash savedbefore 30d > > > > but any doveadm command that uses "-A" to iterate through all users will > > stop processing at the first account with UID<first_valid_uid. > > What userdb are you using? userdb passwd should already skip users that > aren't in the valid range. And what Dovecot version are you using? passwd-file under dovecot 2.0.16. > And one more thing: Does it really even stop there? Looking at the code > it's supposed to log an error and continue to next user. Note that it says > "Failed to iterate through SOME users". The wording did not escape my notice, which is why I suspect it's not doing what it was designed to do. This is my test: # Command doveadm mailbox list -A # Start of password file sysdaemon:*:500:500:System daemon:/:/dev/null ... and the rest ... # dovecot.conf ... first_valid_uid = 10000 first_valid_gid = 10000 ... In this situation, doveadm will exit immediately with an UID error message. If I change the UID>10000, it will produce the analogous GID error message. If I satify both UID and GID constraints, it will fail on the next daemon entry. If I move the sysdaemon entry all the way to the bottom of the pasword file, I get user1 saved-messages user1 sent-mail user1 postponed-msgs user1 temp user1 temp/temp user1 INBOX user2 sent-mail user2 101 user2 345 user2 ckf ... all user's mailbox with UID>10000, then ... doveadm(sysdaemon): Error: user sysdaemon: Couldn't drop privileges: Mail access for users with UID 500 not permitted (see first_valid_uid in config file, uid from userdb lookup). doveadm(sysdaemon): Error: User init failed doveadm: Error: Failed to iterate through some users > Oh, it says about first_valid_gid. Is sysdaemon's UID within valid range? Sorry for this error mismatch -- I cut&pasted the wrong test output; however, the problem I witnessed applies to both UID and GID (if either constraint is not met, user iteration terminates). > I also added this today: http://hg.dovecot.org/dovecot-2.1/rev/85a8d582d37f It looks like I'll be upgrading. Oh, I just spotted this in the ChangeLog -- maybe you are undoing this? (2010-10-21) * src/auth/auth-settings.c, src/auth/auth-settings.h, src/auth/userdb- passwd.c: auth: userdb passwd iteration now lists only users within first_valid_uid..last_valid_uid range. [745ef289b0ea] Joseph Tam <jtam.home(a)gmail.com>

2 1