- dovecot - dovecot.org

Re: How to detect out-of-sync condition
by Joseph Tam 14 Mar '15

14 Mar '15

On Fri, 13 Mar 2015, Cliff Hayes writes: >> By closing off other avenues other than dovecot imap/pop/lda/etc., >> the indices will stay sync'd. > > I use dovecot's lda and dovecot's sieve filter. Then I'm not sure how mailboxes ever get out of sync. > So it looks like I need to compare the index/mailbox mtimes as you suggest. > > What am I looking for? > I see that the indexes are updated when I run the resync. > > I checked my mailbox (that was not resynced) and noticed that > dovecot.index last update was 16 days ago. Just to be clear, you're talking "doveadm index ...", yes? You might have to check dovecot.index.log* as well as they could contain the latest changes that have yet to propagate to the main index file. http://wiki2.dovecot.org/Design/Indexes/TransactionLog > So am I resyncing if the gap is over x days? > If so, is there a way to resync just those mailboxes with a doveadm > command or do I have to write a program look for that condition and run > doveadm when matched? It's possible that "doveadm index" checks modification times as an optimization measure. My recommended game plan: 1) Are your indices *really* out of date (checks logs as Steffen recommends)? 2) If so, how do they get out of date and can you avoid it? 3) If you can't avoid it, does it cause real problems? Most of the time, dovecot will seemlessly rebuild it and it's transparent to the user. 4) If you got this far, run a trial "doveadm index -A INBOX" (assuming you're just concerned about INBOXs) to see if it's really such a bad operation. As Knuth (hallowed be thy name!) said, "premature optimization is the root of all evil." Only after all these steps are found to be unsatisfactory would I consider writing your own scan and fix tool. Joseph Tam <jtam.home(a)gmail.com>

2 1

Re: How to detect out-of-sync condition
by Joseph Tam 14 Mar '15

14 Mar '15

Cliff Hayes writes: > I didn't see anything but the log is huge and I could have easily missed > it. > Is there a certain error or phrase I should look for? > If so please advise. In my setup using mbox, typical log entries would be like (Common) Warning: UIDVALIDITY changed ({int} -> {int}) in mbox file ... Error: Next message unexpectedly corrupted in mbox file ... (Rare) Error: Corrupted index cache file {cache-file}: Broken physical size for mail UID ... Error: read({file}) failed: Next message unexpectedly lost from mbox file ... Error: read({file}) failed: Cached message size larger than expected ... Joseph Tam <jtam.home(a)gmail.com>

1 0

JMAP support
by List 13 Mar '15

13 Mar '15

Just found http://jmap.io/ which is a JSON based RPC protocol for synchronizing messages/contacts/calendars. Any plans to support this protocol in Dovecot?

1 0

How to detect out-of-sync condition
by Cliff Hayes 13 Mar '15

13 Mar '15

Hello list, I recently had a user whose mailbox had gone out of sync. Webmail and client inbox totals had become vastly different and response times had become unacceptable. A doveadm force-resync fixed the problem nicely. This will no doubt happen again as we grow the service and I would like to have some type of indication at the server level so I can run a resync on a per-user basis as required. However, I don't know what to look for. I did not see any signs of the problem on the server - but perhaps I did not know what to look for. The only other alternative is for me to do a global resync in the cron.daily but my fear is that this won't scale well for thousands of users. Please advise. Thanks in advance.

3 6

Different realm for different listeners?
by Chris Adams 13 Mar '15

13 Mar '15

I see this has been asked a few times over the years (but not in several years), and the response was along the lines of "maybe someday", so I figured I'd see if someday was here yet... I have a need to have a different default realm for different listeners. Basically, I've a bunch of different domains, and a few of them insist their users not be required to authenticate with "user@domain" and just use "user", and I'm trying to make this work without separate daemons and/or VMs. I'll be using Dovecot 2.2 (most likely 2.2.10 as provided by RHEL/CentOS 7). Thanks. If there's not a way to do this directly in Dovecot, does anybody have any suggestion that doesn't require several different VMs? -- Chris Adams <cma(a)cmadams.net>

3 4

Active directory bind and quota stuff
by Fran 13 Mar '15

13 Mar '15

Hi list, first of all, please excuse me for any grammar mistake on my text, I'm still learning english. I'm making a Dovecot installation using my Active Directory like pass and user DB. I'm setting the per-user quota from an AD field (Initials) and everything seems to work fine, but I would like to know if you could help me to improve a couple of things. --------------------------------------------------------------------------- --------------------------------------------------------------------------- I'm using CentOS 7, this is my Dovecot build: --------------------------------------------------------------------------- # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL driver plugins: mysql postgresql sqlite Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap(plugin) nss passwd prefetch passwd-file sql # dovecot --version 2.2.10 And this is my /etc/dovecot/dovecot-ldap.conf.ext file: --------------------------------------------------------------------------- hosts = adserver1.dom, adserver2.dom base = DC=dom ldap_version = 3 auth_bind = yes auth_bind_userdn = dom\%u dn = cn=imapUser,cn=Users,dc=dom dnpass = ******** user_filter = (&(samaccountname=%u)(objectClass=person)(|(mail=%u@dom.com)(othermailbox=%u@dom.com))) pass_filter = (&(samaccountname=%u)(objectClass=person)(|(mail=%u@dom.com)(othermailbox=%u@dom.com))) pass_attrs = userPassword=password user_attrs = Initials=quota_rule=*:storage=%$MB --------------------------------------------------------------------------- --------------------------------------------------------------------------- My questions: 1) My AD doesn't allow anonymous searches and I was thinking in a way to avoid the need of have a specific user account with his password stored in a configuration file to do the bind. And I don't know if this is posible, but, does it exist a way to use the same user you are trying to validate to do bind with the AD server? 2) As you can see (user_attrs = Initials=quota_rule=*:storage=%$MB), to set the per-user quota value, I require a numeric value in "Initials" field of my AD. I've realized that if some wrong value is introduced in that field, Dovecot throws an error when you try to validate the user, and hence, imap logon fails. Does it exist any way to bypass that and to use the default quota value if something is wrong with the per-user quota set? Error log with wrong quota values (I wrote the value e34 in the Initials field of user1 in AD, instead of a valid numeric value): * quota refresh command: [root@dmail02 dovecot]# doveadm quota recalc -u user1 doveadm(user1): Error: user user1: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:storage=e34MB: Invalid rule limit value 'storage=e34MB': Unknown unit: e34MB * imap login: Mar 13 14:12:10 dmail02 dovecot: imap(user1): Error: user user1: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:storage=e34MB: Invalid rule limit value 'storage=e34MB': Unknown unit: e34MB Mar 13 14:12:10 dmail02 dovecot: imap(user1): Error: Invalid user settings. Refer to server log for more information. Thank you very much in advance, Fran

1 0

Dovecot fails to compile --with-cdb (and attempted fix)
by Jeffrey 'jf' Lim 13 Mar '15

13 Mar '15

Dovecot version: 2.2.15 Tinycdb: 0.78 I'm attempting to compile dovecot --with-cdb, but keep getting into a problem with cdb: ======================== libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o anvil-settings.o connect-limit.o penalty.o -Wl,--export-dynamic ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_read' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_find' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_free' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_init' collect2: ld returned 1 exit status make[3]: *** [anvil] Error 1 make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' ========================== After some troubleshooting, I've finally managed to get dovecot to compile, albeit with a hack: --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 @@ -230,7 +230,7 @@ LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ libimap-storage.la LIBICONV = LIBOBJS = -LIBS = -lrt +LIBS = -lrt -lcdb LIBTOOL = $(SHELL) $(top_builddir)/libtool LIBWRAP_LIBS = LINKED_STORAGE_LDADD = This isn't the best fix, admittedly (I wish I were more familiar with autotools), but I'll say that the configure process isn't right yet for cdb. Can anybody produce a better fix? Would love to learn more. thanks, -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. Mensan / Full-Stack Technical Polymath / System Administrator 12 years over the entire web stack: Performance, Sysadmin, Ruby and Frontend

2 6

doveadm sync subscribed folders only
by Jernej Porenta 13 Mar '15

13 Mar '15

Heya, is there a simple way to use "doveadm sync" for subscribed folders only? Since the migration from UW-IMAP, our users can actually subscribe to non-mbox files and now, when we are pushing new servers out, we are migrating our mboxes to dbox format and we would like to migrate only subscribed folders. I know, there is a -m switch which can be used to transfer subscribed folders for each user, but it makes it quite unusable with escaping and all other problems with such volume, so just wondering if there is a simple way to migrate subscribed folders only. Thank you in advance, cheers, Jernej

1 0

Re: How to detect out-of-sync condition
by Cliff Hayes 13 Mar '15

13 Mar '15

Thanks. I use dovecot's lda and dovecot's sieve filter. So it looks like I need to compare the index/mailbox mtimes as you suggest. What am I looking for? I see that the indexes are updated when I run the resync. I checked my mailbox (that was not resynced) and noticed that dovecot.index last update was 16 days ago. So am I resyncing if the gap is over x days? If so, is there a way to resync just those mailboxes with a doveadm command or do I have to write a program look for that condition and run doveadm when matched? Cliff Hayes <chayes(a)afo.net> writes: > I recently had a user whose mailbox had gone out of sync. > Webmail and client inbox totals had become vastly different and response > times had become unacceptable. > A doveadm force-resync fixed the problem nicely. > > This will no doubt happen again as we grow the service and I would like to have some type of indication at the server level so I can run a resync on a per-user basis as required. Perhaps what you ought to look for is how are they getting out of sync in the first place. Assuming your webmail is a IMAP/POP client, I would look at ways that your user's INBOX can be modified out of Dovecot's sight. For example, does your LDA use dovecot's lda/lmtp, or do your users run mail clients that directly manipulate mailbox files, or do you use procmail/mail filters? By closing off other avenues other than dovecot imap/pop/lda/etc., the indices will stay sync'd. If you really have to do manual resyncing, comparing index/mailbox mtimes might be one way to do it. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: How to detect out-of-sync condition
by Joseph Tam 13 Mar '15

13 Mar '15

Cliff Hayes <chayes(a)afo.net> writes: > I recently had a user whose mailbox had gone out of sync. > Webmail and client inbox totals had become vastly different and response > times had become unacceptable. > A doveadm force-resync fixed the problem nicely. > > This will no doubt happen again as we grow the service and I would like > to have some type of indication at the server level so I can run a > resync on a per-user basis as required. Perhaps what you ought to look for is how are they getting out of sync in the first place. Assuming your webmail is a IMAP/POP client, I would look at ways that your user's INBOX can be modified out of Dovecot's sight. For example, does your LDA use dovecot's lda/lmtp, or do your users run mail clients that directly manipulate mailbox files, or do you use procmail/mail filters? By closing off other avenues other than dovecot imap/pop/lda/etc., the indices will stay sync'd. If you really have to do manual resyncing, comparing index/mailbox mtimes might be one way to do it. Joseph Tam <jtam.home(a)gmail.com>

1 0