- dovecot - dovecot.org

[Dovecot] mdbox + gzip and rsync
by Jean-Daniel Beaubien 12 Mar '12

12 Mar '12

Hi, After reading the following paragraph from the dovecot doc, I've been wondering how it would affect rsync (when combined with gzip): "Expunging a message only decreases the message's refcount. The space is later freed in "purge" step. This is typically done in a nightly cronjob when there's less disk I/O activity. The purging first finds all files that have refcount=0 mails. Then it goes through each file and copies the refcount>0 mails to other mdbox files (to the same files as where newly saved messages would also go), updates the map index and finally deletes the original file. So there is never any overwriting or file truncation." How will the mailbox files (m.X) files be modified when I move or delete emails using mdbox+gzip. Will the resulting gzipped mdbox files be rsync-able or will they need a full re-upload? If I plan on using rsync for backups, am I better off not using the gzip feature (if i can spare the extra storage)??? Thanks, -JD

3 3

[Dovecot] dot named folders
by Micah Anderson 12 Mar '12

12 Mar '12

When a user makes a folder called 'x.y' it actually creates a folder called 'x' with a folder called 'y' inside, rather than a folder called 'x.y'. I'm guessing this has to do with an internal folder separator namespace configuration, but I'm a bit confused by how this works. I'm using 2.0.15 with mdbox and this is what I have configured for my namespaces: namespace { separator = . prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no } I migrated from courier maildirs, so perhaps I no longer need some of these now that the conversion is finished? thanks for any suggestions, I've got my head mixed up on this issue, micah --

4 5

[Dovecot] Extracting mbox format from Dovecot IMAP (mdbox)
by Giles Coochey 12 Mar '12

12 Mar '12

Hi, I'm looking for a quick tool that can connect to my IMAP account grab all the messages in a particular folder and dump them to a mbox format file? Anyone know a quick easy tool to do that? This is a spam folder that I'd like to do some Bayes spam learning on, but since I've migrated to mdbox I don't think I can do this directly on the mailbox. Any thoughts appreciated. -- Best Regards, Giles Coochey NetSecSpec Ltd UK Mobile: +44 7983 877 438 Business Email: giles.coochey(a)netsecspec.co.uk Email/MSN/Live Messenger: giles(a)coochey.net Skype: gilescoochey

2 4

Re: [Dovecot] Question about folder creation/delete and logging
by Maria Arrea 12 Mar '12

12 Mar '12

mail_log plugin is just what we need, thank you for your support. Yes, we are using bacula+doveadm import for this project. If you are interested, we have in production an home-brew message-tracking system for our end-users. We parse qmail / postfix / clamav / spamassassin/ dovecot / sieve logs and insert them in a mysql database, and a logged user in our webapp can see what happened with her mails (sent mails or messages waiting for arrival). Affero GPL software, of course. Some screenshots (Spanish only, sorry) here: https://gestionproyectos.us.es/attachments/download/321/Seguimiento_de_mens… https://gestionproyectos.us.es/attachments/download/145/Captura_de_pantalla… https://gestionproyectos.us.es/attachments/download/156/Seguimiento_de_mens… URL of the project (Seguimiento, spanish word for "tracking"): https://gestionproyectos.us.es/projects/seguimiento Regards Maria ----- Original Message ----- From: Timo Sirainen Sent: 03/12/12 02:04 PM To: Maria Arrea Subject: Re: [Dovecot] Question about folder creation/delete and logging On Mon, 2012-03-12 at 12:56 +0100, Maria Arrea wrote: > > - Is there any way of Dovecot logging to write when a folder is > deleted or created? We do not want to increase too much our "normal" > logging level. We use Dovecot 2.0.18+mdbox+zlib You can configure mail_log plugin to only log mailbox creations and deletions. http://wiki2.dovecot.org/Plugins/MailLog > - Does anybody know of any other project to create an easy-restore > for Dovecot? I guess you're using "doveadm import"? So other than that, haven't heard of any.

1 0

[Dovecot] Question about folder creation/delete and logging
by Maria Arrea 12 Mar '12

12 Mar '12

Hello We are working in a web based restore system for our Dovecot users. In this web form a user must log-in and after successful login can estore a deleted folder from date X. We will release it under the GPL. I have a couple of questions: - Is there any way of Dovecot logging to write when a folder is deleted or created? We do not want to increase too much our "normal" logging level. We use Dovecot 2.0.18+mdbox+zlib - Does anybody know of any other project to create an easy-restore for Dovecot? Regards Maria

3 2

[Dovecot] Dovecot LDA breaking .qmail forwarding?
by Wouter van der Schagt 12 Mar '12

12 Mar '12

Good morning all, I've a problem, I'm using the Dovecot LDA in my .qmail file: |/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER However when I add a forward, such as: |/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $EXT@$USER <mailto:&wouter@vdschagt.com> &wouter(a)vdschagt.com The mail isn't forwarded and stays in the queue on the mail server, the same is the case when I reverse the lines. The error in the log file is: @400000004f5de5f7033bc434 delivery 824026: deferral: qmail-inject:_fatal:_qq_trouble_creating_files_in_queue_(#4.3.0)/system_erro r_calling_qmail-inject/ When using another LDA, for example when specifying the Maildir, it works, but then i cannot use Sieve scripts. Any ideas? Am I doing anything wrong? Sincerely, - Wouter van der Schagt

1 0

[Dovecot] nfs error fcntl(read-lock) locking failed for file
by Rajesh M 10 Mar '12

10 Mar '12

hi i am using qmailtoaster with dovecot version 2 mailbox format is maildir i have a domain with around 5000 users which are distributed over 2 servers webmail (squirrelmail) runs using dovecot v2 is being used from server number one server number 2 had all the data stored in it and also has pop and smtp running from it. i am not using dovecot for pop as yet on the server with dovecot i get such errors in the log file access to data on server number 2 is via nfs on server number 1 i get errors as such Error: fcntl(read-lock) locking failed for file Input/output error squirrelmail gives error imap connection closed and i am not able to login so i set the parameters as such in the dovecot conf file and the error stopped mmap_disable=yes dotlock_use_excl = yes lock_method = dotlock can somebody please advise me if the above is correct ? or is it preferred to use fcntl with lockd (note that my mailbox is maildir format) thanks very much for your help rajesh

2 1

[Dovecot] Lock down Shared Mail Accounts?
by Charles Marcus 10 Mar '12

10 Mar '12

On 3/5/2012 1:30 PM, Steve Campbell wrote: > I've been looking at some documentation on shared mail accounts. > But I'm getting mixed thoughts on how this can or should be done. This brings up a question I have been meaning to ask. One thing I want to do on my new converted system is to implement shared mail. There will be two different scenarios - users sharing 'folders', which looks to be fairly simple using virtual ACL files - but for the other scenario, I'm not sure about a specific requirement we will have... I want to give multiple people shared access to some actual accounts with all of the special use folders, with the following requirements: 1. They can all read/reply to new messages as they come in, 2. They use shared \seen, \replied and \forwarded flags, so once someone else has read/dealt with a message, the others see that, 3. When they reply to/forward a message, the Sent message gets saved to that accounts 'Sent' folder, 4. They can *move* messages to other folders in that account (ie, 'file' them), and last (this is the tricky part) 5. No one other than a designated user or users (Master User(s)? Users in a specified Group?) can delete any messages in this account, in any of the folders. These emails deal with financial transactions (AP and AR issues) and Faxes, thus the requirement to not be able to delete them. Can this be accomplished with the current state of things? Or would this require some coding? If the latter, could it be done as a plug-in, or would it require changes to the core code? Thanks, -- Best regards, Charles

4 5

Re: [Dovecot] dovecot Digest, Vol 107, Issue 20 Fscking warnings
by Stephen Davies 10 Mar '12

10 Mar '12

Yes that is the google thread that I saw. I don't see the relevance of your reference to dsync. As I read the man pages for dsync it is used to sync separate servers, to make backups or to convert mailbox formats. When I upgraded from 1.2.15 to 2.1.1 I saw nothing in the doco to suggest that dsync was relevant to my scenario. In a previous thread here (Log sync errors), Timo suggested that the migration fix was to delete all .imap directories. My understanding was that this should fix any differences between 1.2.15 files and 2.1.1. If that were the case, mentioning the migration again would seem irrelevant. However, it seems that deleting the .imap files did not fix the log sync errors or the fscking warnings. Both are still happening continuously. Cheers, Stephen On Thu, 8 Mar 2012 08:26:55 PM dovecot-request(a)dovecot.org wrote: > Date: Wed, 07 Mar 2012 15:03:35 -0600 > From: Stan Hoeppner <stan(a)hardwarefreak.com> > To: dovecot(a)dovecot.org > Subject: Re: [Dovecot] Fscking warnings > Message-ID: <4F57CD27.3000207(a)hardwarefreak.com> > Content-Type: text/plain; charset=ISO-8859-1 > > On 3/6/2012 5:07 PM, Stephen Davies wrote: > > Google tells me that these "should go away" but they don't. > > > > > > > > Seems to happen continuously while a user is viewing email. > > Is this thread what "Google tells you"? > > http://dovecot.org/list/dovecot/2010-October/053909.html > > Timo is the creator of Dovecot, if you didn't know. So you can take his > words for gospel. Also note his last statement in that thread: > > "The next time you could do it with dsync to avoid these kind of > problems." > > It would seem you omitted a very important detail from your problem > report, which is that you recently performed a migration. Please don't > omit such critical details in future requests for help. Provide as much > relevant detail as possible. This speeds the process up for everyone, > and avoids guesswork on our part. > > -- > Stan > > > ------------------------------ > > Message: 10 > Date: Thu, 8 Mar 2012 00:26:55 +0100 > From: "Marc" <M.Roos(a)roosit.eu> > To: <dovecot(a)dovecot.org> > Subject: [Dovecot] FW: Centos 6 + dovecot 2 + mail.app + imap -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583

2 1

[Dovecot] POP3C storage backend
by Tom Talpey 10 Mar '12

10 Mar '12

I see a new "POP3C" lib-storage client backend in dovecot 2.1, but I don't see anything in the 2.1 doc directory or in the wiki. Can this be used to synchronize dovecot with external pop servers? Doing away with my current fetchmail and lmtp solution for this would be quite interesting. Thanks for any pointers to configuring and using this, if so...

2 1

[Dovecot] Fscking warnings
by Stephen Davies 10 Mar '12

10 Mar '12

Google tells me that these "should go away" but they don't. Seems to happen continuously while a user is viewing email. Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Archive/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Davies/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/FieldNET/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Invoices Out/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Lawrence and Hanson/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Logger Call/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Logger Reset/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/River Murray/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/SMS Emails/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Soil Moisture Alert/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Water Management Alarm/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Water Usage/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Weather Summaries/dovecot.index Mar 7 09:29:52 server dovecot: imap(john): Warning: fscking index file /home/john/Mail/INBOX/.imap/Zerna/dovecot.index -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583

3 2

[Dovecot] Single instance storage
by Jean-Daniel Beaubien 10 Mar '12

10 Mar '12

I have read most of the doc on the dovecot website, and couldn't find any info on the single instance storage feature, so I'm posting my questions here. - Are these 3 parameters the only one necessary for single instance storage? I cannot find any doc on this feature on the website; is there anything specific I need to know about them? (the last one isn't exactly self-explanatory). - mail_attachment_dir = /srv/vmail/attachments - mail_attachment_hash = %{sha256} - mail_cache_min_mail_count = 2 - Is this feature ready for production? Thanks, -JD

2 1

[Dovecot] migrating/converting from system users -> virtual users
by Steve Platt 10 Mar '12

10 Mar '12

Hello all, We currently have a traditional mail server where all users have system accounts (ie entries in the NIS passwd map) and mbox-format mail folders in their (system) home directories. I'm trying to setup a dovecot server in which we want all users to have "virtual" accounts (in dovecot) and no entry in the passwd file at all (ie no access to the mail server). This is for security reasons, we want to imagine that users connecting to the mail server in future will not be able to access any resources on the server other than access their email messages. To this end we've set up a "vmail" user that owns all the messages in the Maildir directories. Most of this is working but I'm stuck on how to convert users' mail folders from the existing setup to the new one. I'm using the convert plugin but of course the problem is that the plugin executes as the "vmail" user and cannot access the existing mail folders that belong to the users: and I'd be worried if it could, of course! So I've converted a few users already by making copies of their old-world mailboxes, changing the ownership of the copies and pointing the convert plugin to these copies, instead of the originals. This works (modulo the dreaded "timeouted" error messages) but is very ugly. So I tried looking at the convert-tool but can't find a man page for it so am not sure what I'm doing with it! I have the idea that I should be able to run some command (as a privileged user) on the mail server and have it do the conversion for me, changing the ownership/permissions on the way. Can convert-tool do this? I'd prefer to go with the automatic (plugin) conversion if I can bodge the ownership issues somehow. Failing that, some tool or script may be the next best answer. This is Dovecot 1.2 by the way on Solaris 10. Any suggestions would be welcome! Steve Platt

2 3

[Dovecot] Whether the THREAD command support cross-mailbox thread?
by Andy YB Hu 10 Mar '12

10 Mar '12

In the real world, the mails which belongs to one thread could be dispersed in different mailbox, at least Inbox and SENT, so whether the command can search different mailboxes and grap them in one talk with dovecot? If Not, any other approach to do that? BTW, what the THREAD=REFS stands for? In the RFC, http://tools.ietf.org/html/rfc5256, i didn't find this algorithm.

2 1

[Dovecot] Advise on upgrading from a jurassic version - please help.
by Martin Mielke 10 Mar '12

10 Mar '12

Hi all, I have inherited an old Dovecot installation which is causing headaches almost every day. I know that one of the rules says "Don't bother asking questions about v0.99.x versions. They're no longer supported."...but please bear with me, this will be quick as I only need some advise from experienced Dovecot gurus out there. I have read the Dovecot documentation and there are instructions to upgrade from 0.99.x to 1.x and so on... my question is: can I upgrade from 0.99.11 to 2.x directly or is it a massive leap? If so, what do I have to keep in mind? This is a production system so I should not break anything... or at least have a rollback plan... Thanks a lot in advance! Regards, Martin

2 1

[Dovecot] sieve and utf-7 foldernames
by Andreas Schulze 10 Mar '12

10 Mar '12

Hi all, since many dovecot/pigeonhole versions I have an error: Mails are delivered into wrong folders if the foldername contain a german umlaut. ( ä, ö, ü ) setup: dovecot-2.1.1 / pigeonhole-0.3.0 postfix deliver to dovecot-lda .dovecot.sieve contains this: require ["fileinto","reject","vacation","relational","comparator-i;ascii-numeric","regex"]; if header :contains "To" "green(a)example.org" { fileinto "INBOX.gr&APw-n"; stop; } a mail to the mentioned address produces this logging: Mar 9 22:23:10 test dovecot: lda(foo): save: box=INBOX.gr&APw-n, uid=2, msgid=<4711(a)example.org>, size=4642 also a mail_debug=yes log no more lines related to the foldername but at the end I found the mail not in the expected folder. It places in "INBOX.gr&-APw-n" notice the extra minus after & I think it's pigeonholes fault but I may be wrong ... # doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686 i686 Debian 5.0.9 ext3 ... lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 127.0.0.1, ::1 protocols = " imap sieve" protocol lda { mail_plugins = quota notify mail_log sieve } ... Andreas

2 1

[Dovecot] Has dovecot 2.1.1 been built and tested on AIX 6.1???
by Bennett, Tony 09 Mar '12

09 Mar '12

I have downloaded and built dovecot 2.1.1 using gcc on AIX 6.1. (The output of "dovecot -n" is at the bottom of this email.) I'm trying "baby steps" to get it up, before I give it the final configuration. (My apologies: I was pointed to RFC3501 and told to get an IMAP server, build it, configure it, and bring it up) What is currently occurring when I start dovecot is: Error: service(pop3-login): listen(::, 110) failed: Address already in use Error: service(pop3-login): listen(::, 995) failed: Address already in use Error: service(imap-login): listen(::, 143) failed: Address already in use Error: service(imap-login): listen(::, 993) failed: Address already in use Fatal: Failed to start listeners Using TRUSS and recompiling with log messages I've determined that dovecot is successfully creating and binding to AF_INET sockets... but is failing when trying to do the "bind" the same port to an AF_INET6 socket. The failure is "EADDRINUSE". The logic in the dovecot sources seems driven off of the define of HAVE_IPV6 (defined in config.h by configure) So, the questions I have are: - Is this the correct behavior - If this is the correct behavior, has this been tested against AIX 6.1, and if so, does anyone have an idea of what I did wrong...??? If it has not been tested against AIX 6.1 and is NOT the correct behavior, should I just change "config.h", and undefined HAVE_IPV6 ... or is there a better way to move beyond this issue... (like a change to "configure")??? Thanks, -tony Here is the output of "dovecot -n": # 2.1.1: /attic/usr/local/etc/dovecot/dovecot.conf # OS: AIX 1 00C30F654C00 default_login_user = dovecot disable_plaintext_auth = no namespace { inbox = yes location = mailbox { special_use = \Drafts name = Drafts } mailbox { special_use = \Junk name = Junk } mailbox { special_use = \Sent name = Sent } mailbox { special_use = \Sent name = Sent Messages } mailbox { special_use = \Trash name = Trash } prefix = name = inbox } passdb { args = scheme=CRYPT username_format=%u /attic/usr/local/etc/dovecot/users driver = passwd-file } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = username_format=%u /attic/usr/local/etc/dovecot/users driver = passwd-file } protocol lmtp { service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } protocol lda { service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } protocol pop3 { service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } protocol imap { service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } }

2 2

Re: [Dovecot] disabling SSLv2 in dovecot 1.2.17
by Steve Platt 09 Mar '12

09 Mar '12

Hi Mark, I think I may not have been clear enough in my query, sorry! What I'm trying to do is to prevent SSLv2 connections being made to our IMAP server while allowing SSLv3 and TLSv1 connections. I think I've prevented the use of SSLv2 ciphers but this does not prevent SSLv2 protocol connections (as far as I can tell). (Once connected, the SSLv2 client finds it has no ciphers so the session fails at that point but this is not enough to satisfy our security audit. I want to disable the use of the SSLv2 protocol itself, not just the SSLv2 ciphers) steve.platt(a)mrc-bsu.cam.ac.uk said: > I see Dovecot2 had the following change a year or so ago, in file src/ > login-common/ssl-proxy-openssl.c: > > - SSL_CTX_set_options(ssl_ctx, SSL_OP_ALL); > + SSL_CTX_set_options(ssl_ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2); > > I tried making the same change to dovecot1's src tree on our test system and > it seems to have the desired effect ... I'm testing this by using: openssl s_client -ssl2 -connect mailhost:993 This should fail immediately with "ssl handshake failure" (for a happy audit!). Thanks again, Steve

1 0

[Dovecot] seeking advice: dovecot versions; mailbox formats.
by Vincent Schut 09 Mar '12

09 Mar '12

Hi, I'm currently migrating our old (colocated) mail server (running a [terribly outdated, I know] dovecot 1.1.11) to a new VPS (virtual private server). The old server was running gentoo linux (which is mainly the culprit of the old dovecot version: gentoo was too much trouble to keep updating); the new server will run debian (stable: 6). Debian currently has dovecot 1.2.15 in its repositories; not that much newer... I read in the docs about the auto-generated-from-hg debian dovecot packages for 2.0, 2.1 and 2.2. Which leaves me to the choice what version to use... OK, 2.2 is development, which leaves the choice to: 1.2.15; 2.0.x, or 2.1.x. I would appreciate any consideration or thoughts on what version to choose. On a related note, there is the possibility to switch from maildir to dbox. I did not really find much pros or cons, except from performance and standards-compliance (ability to use e.g. mutt on the server itself). Any thoughts? About the server: we're just a small company. Think about 15 accounts, normal mail traffic, sometimes relatively large attachments (20mb+). Some accounts have many folders; some accounts are very large (5Gb+). Storage is on ext3, raid10. Performance has never been an issue; reliability and ease of maintenance is more important. Thanks, Vincent Schut.

5 7

[Dovecot] disabling SSLv2 in dovecot 1.2.17
by Steve Platt 09 Mar '12

09 Mar '12

I've set up a list of ciphers that excludes SSLv2 ciphers (and other weak ones) in the hope of preventing SSLv2 connections: ssl_cipher_list = TLSv1+HIGH : !SSLv2 : RC4+MEDIUM : !aNULL : !eNULL : !3DES : @STRENGTH However, this doesn't prevent the SSLv2 connection being allowed as our Nessus scans show and I'm tasked with trying to plug that "hole". I see Dovecot2 had the following change a year or so ago, in file src/login-common/ssl-proxy-openssl.c: - SSL_CTX_set_options(ssl_ctx, SSL_OP_ALL); + SSL_CTX_set_options(ssl_ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2); I tried making the same change to dovecot1's src tree on our test system and it seems to have the desired effect; however I am very hesitant about putting this into our production system without seeking advice here first :-) Have I missed anything that's obviously bad about doing this please? Thanks again, Steve Platt

2 1

[Dovecot] [PATCH] Pop3 order in courier migration script
by Christoph Bußenius 08 Mar '12

08 Mar '12

Hi, I found a problem in the courier conversion script (courier-dovecot-migrate.pl). In some cases, it does not correctly preserve the order of POP3 UIDLs. Consider the following case, when courierpop3dsizelist contains: /2 9000 1327481629 AAA1:2,S 10000 21:1327481629 AAA2:2,S 10000 22:1327481629 AAA3:2,S 10000 23:1327481629 AAA4:2,S 10000 24:1327481629 AAA5:2,S 10000 25:1327481629 BBB11:2,S 10000 11:1327481629 BBB12:2,S 10000 12:1327481629 BBB13:2,S 10000 13:1327481629 BBB14:2,S 10000 14:1327481629 BBB15:2,S 10000 15:1327481629 and courierimapuiddb contains: 1 1327480919 9000 11 BBB11 12 BBB12 13 BBB13 14 BBB14 15 BBB15 16 CCC16 17 CCC17 18 CCC18 19 CCC19 20 CCC20 21 AAA1 22 AAA2 23 AAA3 24 AAA4 25 AAA5 then call courier-dovecot-migrate.pl --to-dovecot --convert and it will create dovecot-uidlist: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 :CCC17 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 :CCC16 9003 PUID15-1327481629 W10000 :BBB15:2,S 9004 :CCC20 9005 PUID14-1327481629 W10000 :BBB14:2,S 9006 PUID13-1327481629 W10000 :BBB13:2,S 9007 :CCC18 9008 PUID11-1327481629 W10000 :BBB11:2,S 9009 :CCC19 The problem here is that the BBB files are not in the same order as in courierpop3dsizelist, and will therefore have a different order in the UIDL response in POP3. Also, the CCC files are interleaved while they should appear at the end. The reason is how the script assigns new UIDs: It does not do this in the order of courierpop3dsizelist. In fact the order is random because it depends on how perl traverses hashes. The attached patch fixes this by sorting the messages: First the POP3 messages get their new UIDs in order of their appearance in courierpop3dsizelist (so that the UIDL order will really be the same as in Courier), after that, the IMAP-only messages will be checked for UID clashes. The patch also changes how the $prev_uid variable is handled because that was also needed to always get the order right. The algorithm could probably be optimized to reduce the number of UID re-assignments to a minimum, however this would complicate the code and would only be noticed in a few cases. The patched courier-dovecot-migrate.pl will create dovecot-uidlist like this: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 PUID11-1327481629 W10000 :BBB11:2,S 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 PUID13-1327481629 W10000 :BBB13:2,S 9003 PUID14-1327481629 W10000 :BBB14:2,S 9004 PUID15-1327481629 W10000 :BBB15:2,S 9005 :CCC16 9006 :CCC17 9007 :CCC18 9008 :CCC19 9009 :CCC20 Cheers, Christoph -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

2 3

[Dovecot] v2.1 latest hg: untagged reply to namespace command
by e-frog 08 Mar '12

08 Mar '12

Hello Timo, There seems to be something broken in v2.1 latest hg version: # 2.1.1 (94de7605f50f) 1 namespace * NAMESPACE (("" "/")("virtual/" "/")) NIL NIL * OK Namespace completed. Please note that the "OK Namespace completed." is send untagged. It worked on below version: # 2.1.1 (315f0d8cc2b2) 1 namespace * NAMESPACE (("" "/")("virtual/" "/")) NIL NIL 1 OK Namespace completed. Thanks, e-frog

1 1

[Dovecot] Shared mboxes
by Steve Campbell 08 Mar '12

08 Mar '12

I've been looking at some documentation on shared mail accounts. But I'm getting mixed thoughts on how this can or should be done. I use mbox for all my pop and imap folders since I've converted from a uw-imap server. The first thing that makes me wonder about setup is that I've been told to not use maildir and mbox on the same machine, although I'm not really sure why since it seems this would work OK, but anyway, I'm guessing I should stick with mbox for the shared accounts. Secondly, I'm sure I'd need a namespace to use which ever format, so there's private, public, and shared types. Most of the stuff I'm reading seems to suggest "public" as a type instead of "shared". So what's shared for anyway? I want to use this shared account so that email can be sent to this account, and be shared by only a few people, but I'm reading where locks and such don't work with mbox, so in my mind, how do you avoid corruption and why not just make a normal account and let people hack away at the data? I've not even got to the questions in my mind about how to set up the account, but figured if I could get the above straight, I might be able to fuddle my way through it. Help would be truly appreciated. steve campbell

2 6

Re: [Dovecot] dsync replication available for testing
by Michael Grimm 08 Mar '12

08 Mar '12

Hi -- On 08.03.2012 12:35, Timo Sirainen wrote: > On Thu, 2012-03-08 at 11:26 +0100, Michael Grimm wrote: > You can do for example: > > service config { > unix_listener config { > user = vmail > } > } I will try that later. >> It seems to me, that whenever a larger number of mails arrive on >> both >> servers simultaneously, the replicator gets into trouble [1]. I am >> unsure if one can expect that a replicator should deal with such >> stress, >> though. Or? > > Were these mails delivered via LMTP or dovecot-lda? LMTP > The locks should prevent duplicates I think, so there's something > still > going wrong. Just to be sure that I didn't misunderstand your proposed configuration: @mx1: plugin { mail_replica = remote:vmail@mx2.example.tld } @mx2: plugin { mail_replica = remote:vmail@mx1.example.tld } I do need to define one mail_replica plugin at each server pointing to the other one, correct? Regards, Michael

1 0

[Dovecot] duplicates with multiple To/CC and sieve redirect copy
by Adam Szpakowski 08 Mar '12

08 Mar '12

Hi, In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Each of this emails has the same Message-ID. Is there a way to detect those duplicates and discard them when saving copies into boss account? There was a discussion on the list a while ago (Feb 10/11, 2011) mentioning this problem, but with no clear solution. We are using Dovecot 2.0.15. Mail is delivered via LMTP (MDA/postfix is on separate machine then MAA/dovecot). -- Adam Szpakowski

5 10

[Dovecot] Failing: doveadm sync <--remote host--> dsync mirror
by Michael Grimm 08 Mar '12

08 Mar '12

Hi -- I'm used to dsync my two mailservers running 2.0.16. Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: mail> doveadm sync -f -u test ssh vmail(a)remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Same error with old running command: vmail> dsync -v -f -u test mirror ssh vmail(a)remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here's an excerpt of my doveconf -n: mail> doveconf -n mail_home = /var/mail/homedirs/%n mail_location = mdbox:/var/mail/maildirs/%n passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } Reverted back for the time being (production servers) until I do understand the issue. Searching Google with "Fatal: All your namespaces have a location setting." only shows closed bugs back in 2010. Do I have to upgrade both servers to get syncing running? Any hint, help is highly appreciated. Thanks, Michael

2 18

[Dovecot] FW: Centos 6 + dovecot 2 + mail.app + imap
by Marc 08 Mar '12

08 Mar '12

Anybody also experiencing that imap processes are kept running/open by mac osx mail.app, so eventually users are getting to the mail_max_userip_connections limit? Outlook / other clients seem to run fine. Thanks, Marc

1 0

[Dovecot] http://xi.rename-it.nl down?
by Jim Knuth 07 Mar '12

07 Mar '12

Hello, you knows, that http://xi.rename-it.nl is down? -- Mit freundlichen Grüßen, with kind regards, Jim Knuth --------- Die Oper ist eine hübsche Unterhaltung, die noch besser wäre, wenn nicht dabei gesungen würde. (Claude Debussy)

2 2

[Dovecot] Log sync errors (again)
by Stephen Davies 07 Mar '12

07 Mar '12

As suggested earlier, I deleted all .imap directories and the log sync errors stopped - for a while. They have now returned. It seems to happen for every mailbox that gets accessed. Dovecot version 2.1.1 with pidgeonhole 3.0.0 on Mandriva Linux. Could this interfere with sieve filters? Several users have filters but none of them seem to do anything. Mar 7 09:25:51 server dovecot: imap(john): Error: Log synchronization error at seq=2,offset=38708 for /home/john/Mail/INBOX/.imap/Weather Summaries/dovecot.index: Extension header update points outside header size Mar 7 09:25:51 server dovecot: imap(john): Error: Log synchronization error at seq=2,offset=41576 for /home/john/Mail/INBOX/.imap/Zerna/dovecot.index: Extension header update points outside header size Cheers and thanks, Stephen -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583

1 0

[Dovecot] Master Users
by Kelsey Cummings 06 Mar '12

06 Mar '12

I have a setup where I need to use a Master User account to login on behalf of users normally authed via PAM. Is there any existing mechanism that will allow master users to be wired down to specific ip address rather than having these very magic user/pass combos be valid from any random host? It would be totally acceptable to be able to say that master logins were only valid from a specific list of hosts rather than wiring specific master users to specific hosts. -- Kelsey Cummings - kgc(a)corp.sonic.net sonic.net, inc. System Architect 2260 Apollo Way 707.522.1000 Santa Rosa, CA 95407

1 1

[Dovecot] Dovecot saves mails in "wrong" folder.
by Khoroshyy Petro 06 Mar '12

06 Mar '12

Hi all I have installed dovecot 1.2.15 and try to use it together with offlineimap and gnus. my problem is that it saves emails into /var/mail/petro instead of ~/Maildir Thanks. Petro. This is my .dovecot.conf default_mail_env = maildir:%h/Maildir And this is my .offlineimaprc [general] accounts = Gmail maxsyncaccounts = 1 [Account Gmail] localrepository = Local remoterepository = Remote [Repository Local] type = IMAP remotehost = localhost port = 143 remoteuser = petro [Repository Remote] type = IMAP remotehost = imap.gmail.com remoteuser = myname(a)gmail.com ssl = yes maxconnections = 1 realdelete = no folderfilter = lambda foldername: foldername in ['INBOX'] --

1 0

[Dovecot] LDAP auth_bind fails
by Pol Bettinger 06 Mar '12

06 Mar '12

Hello, I wanted to configure dovecot for using auth_bind but didn't succeed to me it seems like it does always an anonymous bind. Dovecot version 2.1.1 (I started with 2.1.0 and hoped 2.1.1 would fix it) I tried to play around with the base, pass_attrs,pass_filter to no avail but didn't succeed. Looking at a wireshark trace i only saw 7 packets and it seemed to me dovecot did only an anonymous bind. any help would appreciated Sincerely Pol Bettinger output of mail.log: Mar 6 12:16:34 Dell dovecot: auth: Debug: client in: AUTH#0112#011CRAM-MD5#011service=imap#011secured#011lip=192.168.16.27#011rip=192.168.16.20#011lport=993#011rport=51838 Mar 6 12:16:34 Dell dovecot: auth: Debug: client out: CONT#0112#011PDQ1NjgyMjE3NjYyMDk3NjkuMTMzMTAzMjU5NEBEZWxsPg== Mar 6 12:16:34 Dell dovecot: auth: Debug: client in: CONT<hidden> Mar 6 12:16:34 Dell dovecot: auth: Debug: password(arvi(a)arvoreen.net,192.168.16.20): passdb doesn't support credential lookups Mar 6 12:16:36 Dell dovecot: auth: Debug: client out: FAIL#0112#011user=arvi(a)arvoreen.net output of dovecot -n: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-15-generic i686 Ubuntu 11.10 ext4 auth_debug = yes auth_default_realm = arvoreen.net auth_mechanisms = plain digest-md5 cram-md5 auth_verbose = yes base_dir = /var/run/dovecot/ mail_location = maildir:/var/mail/%d/%1n/%n:INDEX=/var/indexes/%d/%1n/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Archive { auto = create special_use = \Archive } mailbox Drafts { auto = create special_use = \Drafts } mailbox Junk { auto = create special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = create special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap_pass.conf.ext driver = ldap } plugin { sieve = /var/sieve/%d/%1n/%n sieve_dir = /var/sieve/%d/%1n/%n } protocols = imap lmtp sieve service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/certs/webmail.arvoreen.pem ssl_key = </etc/ssl/private/webmail.arvoreen.key userdb { args = /etc/dovecot/dovecot-ldap_user.conf.ext driver = ldap } protocol lmtp { mail_plugins = " sieve" } output of cat /etc/dovecot/dovecot-ldap_pass.conf.ext: hosts = 127.0.0.1:389 auth_bind = yes auth_bind_userdn = uid=%n,ou=Users,dc=arvoreen,dc=net base = ou=Users,dc=arvoreen, dc=net ldap_version = 3 olcAccess infomation: olcSuffix: dc=arvoreen,dc=net olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymou s auth by dn="cn=admin,dc=arvoreen,dc=net" write by * none olcAccess: {1}to dn.base="" by * read olcAccess: {2}to * by self write by dn="cn=admin,dc=arvoreen,dc=net" write by * read

2 1

[Dovecot] dovecot 2.1.1 + pigeonhole + avelsieve
by Frank Elsner 06 Mar '12

06 Mar '12

Hello all, I've squirrelmail-webmail-1.4.22, dovecot 2.1.1, dovecot-2.1-pigeonhole-0.3.0 installed and working. But I've problems to get the avelsieve plugin for squirrelmail working with dovecot. The "Message Filters" show up in "Options" of squirrelmail, but "Could not log on to timsieved daemon on your IMAP server ........." dovecot log shows: Mar 6 00:00:47 seymour dovecot: managesieve-login: Disconnected: Too many invalid commands. (no auth attempts in 0 secs): 192.168.28.53, secured Where to look for configuration error(s)? | root@seymour: /usr/local/dovecot/src<156> dovecot -n | # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf | # OS: Linux 2.6.35.14-106.fc14.i686.PAE i686 Fedora release 14 (Laughlin) ext3 | default_vsz_limit = 512 M | disable_plaintext_auth = no | first_valid_uid = 200 | last_valid_uid = 65534 | listen = * | lmtp_save_to_detail_mailbox = yes | login_greeting = c64.shuttle.de - IMAPs Service (dovecot) ready. | login_log_format_elements = %u %r %c | mail_location = maildir:/var/spool/mail/%u | mail_log_prefix = "%Us(%u,%r): " | mail_plugin_dir = /usr/dovecot/lib/dovecot/ | mail_plugins = " notify quota" | managesieve_notify_capability = mailto | managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave | passdb { | args = dovecot | driver = pam | } | plugin { | mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append | mail_log_fields = uid box from subject msgid size flags | mail_log_group_events = yes | quota = maildir:User quota | quota_rule = *:storage=2G | quota_rule2 = Trash:storage=+100M | sieve = ~/.sieve | sieve_dir = ~/sieve | } | postmaster_address = postmaster(a)moltke28.b.shuttle.de | protocols = imap sieve | service anvil { | client_limit = 1027 | } | service auth { | unix_listener auth-client { | group = exim | mode = 0660 | user = exim | } | } | service imap-login { | inet_listener imap { | port = 143 | } | inet_listener imaps { | port = 993 | ssl = yes | } | process_limit = 512 | process_min_avail = 10 | } | service imap-postlogin { | executable = script-login /usr/dovecot/bin/imap-post-login | } | service imap { | executable = imap imap-postlogin | } | service lmtp { | inet_listener lmtp { | address = 0.0.0.0 | port = 24 | } | } | service managesieve-login { | inet_listener sieve { | port = 4190 | } | } | service pop3-login { | inet_listener pop3 { | port = 110 | } | inet_listener pop3s { | port = 995 | ssl = yes | } | } | service pop3 { | process_limit = 1024 | } | ssl_cert = </usr/local/etc/c64.shuttle.de.CRT | ssl_cipher_list = ALL:!LOW:!SSLv2 | ssl_key = </usr/local/etc/c64.shuttle.de-dovecot.KEY | ssl_parameters_regenerate = 6 hours | userdb { | driver = passwd | } | verbose_proctitle = yes | protocol lmtp { | mail_plugins = " notify quota quota" | } | protocol lda { | mail_plugins = " notify quota quota" | } | protocol imap { | imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags | imap_logout_format = [%i/%o] | mail_max_userip_connections = 0 | mail_plugins = " notify quota mail_log quota imap_quota listescape" | } And the avelsieve config.php has: | /* **** NEW OPTION (as of 0.9.4) **** | * Space separated list of preferred SASL mechanisms for the authentication to | * timsieved */ | | $preferred_mech = "PLAIN"; Ok, looking forward for any tip, Frank Elsner

2 1

[Dovecot] Concurrent dovecot instances on same spool?
by Jacek Osiecki 06 Mar '12

06 Mar '12

Hi all, While reading the "Multiple locations, 2 servers - planning questions..." thread, it just hit me that I was planning to do something that seems to be a bit hazardous... I am setting up an high-availability server set, which consists of two (or more) servers with common disk space, all set behind redundant hardware load balancers. At first, there will be two servers and disk space will be kept on NFS server or on both servers using DRBD+OCFS2 filesystem (what creates kind of networked RAID1 storage space). This will be done mostly to keep WWW service available in case when one of servers fails. However, if we have everything redundant, why not have the same with SMTP and POP3/IMAP? But - won't anything fail if two (or more) dovecots are accessing the same disk space, both for IMAP/POP3 and LDA/LMTP? If it is possible, that would be a good solution for the mentioned thread. If not - where are possible dangers, and are there any plans to make it possible in the future? Greetings, -- Jacek Osiecki joshua(a)ceti.pl GG:3828944 I don't want something I need. I want something I want.

3 4

[Dovecot] testing fts-solr?
by Jean-Daniel Beaubien 05 Mar '12

05 Mar '12

hi everyone, However, how can I verify that solr is working properly? Background: - I'm trying to setup 2.1 with fts_solr and eventually test maildir/mdbox. - I used the following webpage as reference to setup solr: http://www.roessner-network-solutions.com/2012/02/19/full-text-search-with-… I'm looking at /var/log/tomcat6/catalina.out file, but see nothing telling me that it's being used (same for /var/log/mail.log, etc.) Any tips? Thanks

10 34

[Dovecot] \NoSelect on missing folders in LIST
by Peter Mogensen 05 Mar '12

05 Mar '12

Hi, I noticed a difference between courier and dovecot, and I'm not sure which of them is wrong wrt. RFC3501 - if any. I have a Maildir which has been accessed by an Apple Mail client, so it got folders like: INBOX INBOX.Trash INBOX.INBOX.folder INBOX.INBOX.folder.a INBOX.INBOX.folder.b The INBOX.INBOX folder does not exist on disk and is not subscribed. Courier responds to: . list "" "*" with * LIST (\Noselect \HasChildren) "." "INBOX.INBOX" But dovecot does not list that folder using "*". However, if you issue: . list "" "INBOX.%" Dovecot answers: * LIST (\Noselect \HasChildren) "." "INBOX.INBOX" This makes some clients using "*" to get the folder list ignore the folderes below "INBOX.INBOX". I know the recommended client way is to use "%", but I'm still curious about which is the correct behaviour. /Peter

2 3

[Dovecot] GSSAPI auth failing for kmail
by Mark Davies 05 Mar '12

05 Mar '12

Hi, have dovecot 2.0.13 running fine on a NetBSD box, with lots of clients happily authenticating to it with GSSAPI. I've just updated an ArchLinux box to latest new version which has meant its switched from using heimdal to using mit krb5 and its kmail is now also slightly newer. Thunderbird successfully authenticates to dovecot from this machine but kmail now fails. Kmail reports: Unable to authenticate via GSSAPI. The server ... replied: SASL(-1): generic failure: GSSAPI Error: Invalid token was supplied (Token header is malformed or corrupt) The dovecot log reports: Feb 27 13:42:18 bats dovecot: auth: GSSAPI(mark(a)ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in continued response Any suggestions on how to track down what specifically has changed and how to fix it? cheers mark

2 11

Re: [Dovecot] testing fts-solr?
by Charles Marcus 05 Mar '12

05 Mar '12

On 2012-03-05 3:43 AM, kadafax(a)gmail.com <kadafax(a)gmail.com> wrote: > Le 02/03/12 13:40, Charles Marcus a écrit : >> On 2012-02-28 11:28 AM, Charles Marcus <CMarcus(a)Media-Brokers.com> wrote: >>> On 2012-02-28 11:05 AM, kfx <kadafax(a)gmail.com> wrote: >>>> Ok I feel ashame... it was a third party init scrip who was the >>>> problem :( >>> So... you're saying that Thunderbird now correctly uses server side >>> search? >> Please respond... I need to know whether or not I need to pursue this, >> since we use Thunderbird in house and will be switching soon to >> dovecot... > Yes, for me thunderbird correctly use server side search. Just got confirmation from the OP that his problem with Thunderbird indeed is now gone, so it was a 3rd party init script causing his problems... One less thing to worry about... -- Best regards, Charles

1 0

[Dovecot] 1.0beta to latest and greatest?
by Charles C 05 Mar '12

05 Mar '12

dear dovecot users, This is my first post and with a rather embarassing question. To soften the question up I just want to say I've been very pleased with Dovecot thus far and its low maintenance requirements - perhaps to pleased! I am running an ancient version of Dovecot, version 1.0.beta9. Do I risk corrupting indices etc by upgrading in one go to 2.1.2? Thanks for your input. Charles

2 1

[Dovecot] BlackBerry will not setup my POP3 email, all other mail clients fine
by Ian Solecki 05 Mar '12

05 Mar '12

Hello, hoping someone here might be able to help me or at least point me in the right direction. My company recently (last week) moved to a new dedicated server for website and email hosting. It is a fairly run-of-the-mill Linux machine running cPanel and Dovecot (with the BlackBerry Fastmail service enabled) as a mail server. When we first set up the server, I was able to create a mailbox, access it via any desktop mail client, webmail, AND by setting it up as a basic POP3 account on my BlackBerry. Mail was running fine to and from the BlackBerry, no problems. I deleted that account from my BlackBerry as it was a test account, and went to add my actual account but was unable to do so. Received the "Cannot log in. Verify your email address, user name and password. If the error persists, contact (my domain name)" message. I tried the test account that had been running successfully and sending/receiving emails not minutes earlier, and it would not set up either. I have since tried multiple accounts on multiple BlackBerry devices (different models) on multiple carriers on several of my different domains (all of which point to the same server, of course), to no avail. Yet, any of these accounts still work flawlessly in any desktop mail client (Outlook, OE, Thunderbird, iPad, iPhone, etc). Also, any of these BlackBerrys work flawlessly with any other mail server. The username to log in to these mailboxes is not standard, it's mailboxname+ example.com if the email address is mailboxname(a)example.com and the mail server is mail.example.com. So, I know I have to access the "Advanced Settings" in BlackBerry email setup in order to put this username in. Still, no effect. So, there is something wrong with how mail SETUP works (not sending/receiving, though that may also not work, I have no way of knowing now) between RIM and my server, and it's something that has changed since the server was set up a week ago. My carrier(s) are clueless, my dedicated server provider (Lunarpages) is clueless. Can anyone help?

3 2

[Dovecot] doveconf: Warning: Obsolete settings.... in /etc/dovecot/dovecot.conf: ...
by D Chen 05 Mar '12

05 Mar '12

Got duplicated Warnings from the "doveconf -n" output ! Can anyone explain and fix them ? thx! admin@server:/etc/dovecot$ doveconf -n # 2.0.13: /etc/dovecot/dovecot.conf doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:716: protocol managesieve {} has been replaced by protocol sieve { } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:888: add auth_ prefix to all settings inside auth {} and remove the auth {} section completely doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:926: passdb pam {} has been replaced by passdb { driver=pam } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1039: userdb passwd {} has been replaced by userdb { driver=passwd } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1101: auth_user has been replaced by service auth { user } doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:716: protocol managesieve {} has been replaced by protocol sieve { } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:888: add auth_ prefix to all settings inside auth {} and remove the auth {} section completely doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:926: passdb pam {} has been replaced by passdb { driver=pam } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1039: userdb passwd {} has been replaced by userdb { driver=passwd } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1101: auth_user has been replaced by service auth { user } # OS: Linux 3.0.0-16-server x86_64 Ubuntu 11.10 log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { driver = pam - Ignored: } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth-client { group = postfix mode = 0660 user = postfix } unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } user = root } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = passwd } userdb { driver = passwd } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 10 } protocol pop3 { mail_max_userip_connections = 10 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = postmaster quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r }

4 4

[Dovecot] Log sybnch error
by Stephen Davies 05 Mar '12

05 Mar '12

My mail log has many entries like: Mar 2 12:34:13 server dovecot: imap(john): Error: Log synchronization error at seq=2,offset=4264 for /home/john/Mail/INBOX/.imap/SMS Emails/dovecot.index: Extension header update points outside header size Version 2.1.1 dovecot -n gives: # 2.1.1: /usr/etc/dovecot/dovecot.conf # OS: Linux 2.6.33.5-desktop-2mnb i686 Mandriva Linux 2010.2 mail_access_groups = mail mail_privileged_group = mail passdb { driver = pam } protocols = pop3 imap service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 0 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0666 user = } unix_listener login/login { group = mode = 0666 user = } user = root vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = . client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service indexer-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = indexer-worker extra_groups = group = idle_kill = 0 privileged_group = process_limit = 10 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service indexer { chroot = client_limit = 0 drop_priv_before_exec = no executable = indexer extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener indexer { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service ipc { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = ipc extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener ipc { group = mode = 0600 user = } unix_listener login/ipc-proxy { group = mode = 0600 user = $default_login_user } user = $default_internal_user vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 1 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log unix_listener log-errors { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 ssl = no } inet_listener pop3s { address = port = 995 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 18446744073709551615 B } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0600 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } ssl_cert = </etc/pki/tls/certs/dovecot.pem ssl_key = </etc/pki/tls/private/dovecot.pem userdb { driver = passwd } -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583

2 4

[Dovecot] doveadm -A stops processing at first uid<first_valid_uid
by Joseph Tam 04 Mar '12

04 Mar '12

I would like to run various doveadm commands that involves all (mail) users like doveadm expunge -A mailbox Trash savedbefore 30d but any doveadm command that uses "-A" to iterate through all users will stop processing at the first account with UID<first_valid_uid. doveadm(sysdaemon): Error: user sysdaemon: Couldn't drop privileges: Mail access for users with GID 5551 not permitted (see first_valid_gid in config file, gid from userdb lookup). doveadm(sysdaemon): Error: User init failed doveadm: Error: Failed to iterate through some users However, these accounts are system accounts (locked password, no shell) and are in userdb to provide UID<->name mapping for utilities like ls, chown, etc. There are various workaround like iterating manually, or setting first_valid_uid to zero, or even reordering users to put all system accounts at the end, but is there a better way to do this? Joseph Tam <jtam.home(a)gmail.com>

2 3

[Dovecot] Multiple namespaces seems to be used at the same time
by Steve Campbell 04 Mar '12

04 Mar '12

I've just converted from an old Centos 3 box to a Centos 6.2 box. I've switched from UW-imap to dovecot in the process. In my configurations, I've placed the multiple namespace sections as suggested by the "Backward compatability" part of the wiki. I use mbox since I mostly copied the home directories from the old to the new server. On some of the clients, it appears that the client is using multiple namespaces at the same time. When they view their subscribed folders, they see multiple "mail" folders instead of just the single "mail" folder under their home directory. The .subscription files are more than likely not correct (haven't looked yet, but will fix them as a user calls), but should this ever happen? I'm also sure the client's prefix isn't set since the old system never required it and there are just so many other things that are required right now on this conversion. Speaking of prefixes, I'd like to get the default of "" (nothing entered) to work for the majority of the users to avoid having to add this to the multiple users we have. Hopefully, by going through each user's home directory and copying the old .mailboxlist to a new .subscriptions file and ensuring the imap folders are in ~/mail will do this. Does this sound resonable? Thanks for any help. The learning curve is getting less steep, but it's still a ways off to the top of the hill. steve campbell

2 1

[Dovecot] doveadm fetch prints duplicate results in 2.1
by Christoph Bußenius 04 Mar '12

04 Mar '12

Hi, when the private namespace has "prefix = INBOX." and you use doveadm fetch to search for "mailbox INBOX", then it prints every message twice: # doveadm fetch -u testuser 'mailbox uid guid' mailbox INBOX mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 Without "prefix = INBOX." in the configuration, the messages are only printed once. In 2.0, they were never printed twice. doveconf -n # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mdbox mail_plugins = acl mail_uid = vmail namespace { inbox = yes location = prefix = INBOX. separator = . } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocol imap { mail_plugins = acl imap_acl } Cheers, Christoph -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

2 2

[Dovecot] 2.1.1: Incorrect quoting of RFC 2822 personal parts in ENVELOPE data
by Michael M Slusarz 04 Mar '12

04 Mar '12

I'm seeing this: 1 UID FETCH 31734 (ENVELOPE) * 23 FETCH (UID 31734 ENVELOPE ("Fri, 2 Mar 2012 19:05:24 -0500 (EST)" "XXXXXX" (({22} XXXXX \"X-XX\" XXXXXX NIL "XXXXXXX" "XXXXXXXXX.XXX")) (({22} XXXXX \"X-XX\" XXXXXX NIL "XXXXXXX" "XXXXXXXXX.XXXXXX.XXX")) ((NIL NIL "XXXXXXX" "XXXXXXXXX.XXX")) ((NIL NIL "slusarz" "curecanti.org")) NIL NIL NIL "<1109380587237.1109118788902.20323.7.35190001@scheduler>")) It should be: 1 UID FETCH 31734 (ENVELOPE) * 23 FETCH (UID 31734 ENVELOPE ("Fri, 2 Mar 2012 19:05:24 -0500 (EST)" "XXXXXX" (({20} XXXXX "X-XX" XXXXXX NIL "XXXXXXX" "XXXXXXXXX.XXX")) (({20} XXXXX "X-XX" XXXXXX NIL "XXXXXXX" "XXXXXXXXX.XXXXXX.XXX")) ((NIL NIL "XXXXXXX" "XXXXXXXXX.XXX")) ((NIL NIL "slusarz" "curecanti.org")) NIL NIL NIL "<1109380587237.1109118788902.20323.7.35190001@scheduler>")) since the RFC 2822 quoting characters must be removed. michael

2 1

[Dovecot] LIST-STATUS issue
by Michael M Slusarz 04 Mar '12

04 Mar '12

Looked at the hg commits since 2.1.1, but didn't see anything that was exactly on-point to this... UNSEEN counts seem to be broken when done in a LIST-STATUS call. I see this, for example: >> Timestamp: Thu, 01 Mar 2012 10:55:47 -0700 [...] C: 3 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) S: * LIST () "." "IN.dovecot" S: * STATUS "IN.dovecot" (UNSEEN 0) S: * LIST () "." "INBOX" S: * STATUS "INBOX" (UNSEEN 3) S: 3 OK List completed. 3 seconds later... >> Timestamp: Thu, 01 Mar 2012 10:55:50 -0700 S: * OK [CAPABILITY IMAP4rev1 LITERAL+ LOGIN-REFERRALS ID ENABLE AUTH=LOGIN XIMAPPROXY] Dovecot ready. C: [LOGIN Command - username: slusarz] S: * OK [XPROXYREUSE] IMAP connection reused by squirrelmail-imap_proxy S: 1 OK User logged in C: 2 EXAMINE IN.dovecot (QRESYNC (1254166849 25800 18484,20260,20393,20705,20860:20868)) S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft impflag1 impflag2 impflag0 Junk seen NonJunk) S: * OK [PERMANENTFLAGS ()] Read-only mailbox. S: * 13 EXISTS S: * 0 RECENT S: * OK [UNSEEN 5] First unseen. S: * OK [UIDVALIDITY 1254166849] UIDs valid S: * OK [UIDNEXT 20869] Predicted next UID S: * OK [HIGHESTMODSEQ 25800] Highest S: 2 OK [READ-ONLY] Select completed. [...] C: 7 SEARCH RETURN (COUNT) UNSEEN S: * ESEARCH (TAG "7") COUNT 8 S: 7 OK Search completed (0.000 secs). I can confirm that I did not receive 8 dovecot list messages in 3 seconds. :) Once the mailbox is accessed, I am seeing correct counts in LIST-STATUS return again: >> Timestamp: Thu, 01 Mar 2012 10:57:28 -0700 [...] C: 3 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) S: * LIST () "." "IN.dovecot" S: * STATUS "IN.dovecot" (UNSEEN 8) S: * LIST () "." "INBOX" S: * STATUS "INBOX" (UNSEEN 3) S: 3 OK List completed. I am using imapproxy for these connections, so all access are happening in the same Dovecot access. michael

2 8

[Dovecot] directly addressable public folders issues
by Tom Fernandes 04 Mar '12

04 Mar '12

Hi, I would like to have an address info(a)example.com whose mails are stored in a public folder. I also want certain users to be able to create sieve-filter-rules and subfolders. From what I understand it's a good idea to have a separate location for home and for mail_location. For my normal accounts I have: home = /var/vmail/<user> mail_location = ~/Maildir Is there a way to have the same for public folders? This are my current settings: namespace public { separator = / prefix = public/ location = maildir:/var/vmail/public subscriptions = no } user_attrs = homeDirectory=home=/var/vmail/%$, =mail=maildir:~/Maildir The LDAP-homeDirectory-attribute for info(a)example.com is "public/.info" Like this an incoming mail is stored below /var/vmail/public/.info/Maildir which is good. The MUA seems to read to read from /var/vmail/public/.info though. How can I make the MUA read from /var/vmail/public/.info/Maildir instead? And in case this works - is it possible to have only certain users modify the sieve-rule for this public folder or do I definitely need a passdb-entry for info(a)example.com? Modifying scripts via commandline is not an option... I'm using dovecot 1.2. regards, Tom Fernandes

2 1

[Dovecot] Dovecot clustering with dsync-based replication
by Timo Sirainen 03 Mar '12

03 Mar '12

This document describes a design for a dsync-replicated Dovecot cluster. This design can be used to build at least two different types of dsync clusters, which are both described here. Ville has also drawn overview pictures of these two setups, see http://www.dovecot.org/img/dsync-director-replication.png and http://www.dovecot.org/img/dsync-director-replication-ssh.png First of all, why dsync replication instead of block level filesystem replication? - dsync won't replicate filesystem corruption. - A cold restart of replication won't go through all of the data in the disks, but instead quickly finds out what has changed. - Split brain won't result in downtime or losing any data. If both sides did changes, the changes are merged without data loss. - If using more than 2 storages, the users' replicas can be divided among the other storages. So if one storage goes down, the extra load is shared by all the other storages, not just one. Replication mail plugin ----------------------- This is a simple plugin based on notify plugin. It listens for all changes that happen to mailboxes (new mails, flag changes, etc.) Once it sees a change, it sends an asynchronous (username, priority) notification to replication-notify-fifo. The priority can be either high (new mails) or low (everything else). Optionally the replication plugin can also support synchronous replication of new mail deliveries. In this way it connects to replication-notify UNIX socket, tells it to replicate the user with sync (=highest) priority and waits until it is done or replication_sync_timeout occurs. The IMAP/LMTP client won't see an "OK" reply until the mail is replicated (or the replication has failed). The synchronous replication probably adds a noticeable delay, so it might not be acceptable for IMAP, but might be for LMTP. So, what is listening in those replication-notify* sockets? It depends on if Dovecot is running on director-based setup or not. Aggregator ---------- When running in Dovecot director-based setup, all of the Dovecot backends (where replication plugin runs) also run "aggregator" process. Its job is very simple: It proxies the notifications from mail plugin and sends them via a single TCP connection to the replicator process running in Dovecot proxies. This is simply an optimization to avoid tons of short lived TCP connections directly from replication plugin to director server. When not running in Dovecot director setup (i.e. there is only a single Dovecot instance that handles all of the users), there is no point in having an aggregator proxy, because the replicator process is running on the same server. In this kind of setup the replicator process directly listens on the replication-notify* sockets. Replicator ---------- The initial design for replicator isn't very complex either: It keeps a priority queue of all users, and replicates those users at the top of the queue. Notifications about changes to user's mailboxes (may) move the user up in the priority queue. If the user at the top of the queue already has been replicated "recently enough", the replicator stops its work until new changes arrive or the "recently enough" is no longer that. dsync can do two types of syncs: quick syncs and full syncs. A quick sync trusts indexes and does the replication with the least amount of work and network traffic. A quick sync is normally enough to replicate all changes, but just in case something has gone wrong there's also the full sync option, which guarantees that the mailboxes end up being fully synced. A full sync is slower though, and uses more network traffic. The priority queue is sorted by: 1. Priority (updated by a notification from replication plugin) 2. If priority!=none: Last fast sync (those users are replicated first whose last replication time is oldest) 2. If priority=none: Last full sync (these users should already be fully synced, but do a full sync for them once in a while anyway) All users get added to the replication queue at replicator startup with "none" priority. The list of users is looked up via userdb iteration. If the previous replication state is found from a disk dump, it's used to update the priorities, last_*_sync timestamps and other replication state. Replicator process creates such dumps periodically [todo: every few mins? maybe a setting?]. Replicator starts replicating users at the top of the queue, setting their priorities to "none" before starting. This means that if another change notification arrives during replication, the priority is bumped up and no changes get lost. replication_max_conns setting specifies how many users are replicated simultaneously. If the user's last_full_sync is older than replication_full_sync_interval setting, a full sync is done instead of a fast sync. If the user at the top of the queue has "none" priority and the last_full_sync is newer than replication_full_sync_interval, the replication stops. [todo: it would be nice to prefer doing all the full syncs at night when there's hopefully less disk I/O] (A global replication_max_conns setting isn't optimal in proxy-based setup, where different backend servers are doing the replication. There it should maybe be a per-backend setting. Then again, it doesn't account for the replica servers that also need to do replication work. Also to properly handle this each backend should have its own replication queue, but this requires doing a userdb lookup for each user to find out their replication server, and this would need to be done periodically in case the backend changes, which can easily happen often with director-based setup. So all in all, none of this is being done in the initial implementation. Ideally the users are distributed in a way that a global replication queue would work well enough.) In director-based setup each director runs a replicator server, but only one of them (master) actually asks the backends to do the replication. The rest of them just keep track of what's happening, and if the master dies or hangs, one of the others becomes the new master. The server with lowest IP address is always the master. The replicators are connected to a ring like the directors, using the same director_servers setting. The communication between them is simply about notifications of what's happening to users' priorities. Preferably the aggregators would always connect to the master server, but this isn't required. In general there's not much that can go wrong, since it's not a problem if two replicators request a backend to start replication for the same user or if the replication queue states aren't identical. If the replication is running too slowly [todo: means what exactly?], log a warning and send an email to admin. So, how does the actual replication happen? Replicator connects to doveadm server and sends a "sync -u user@domain" command. In director-based setup the doveadm server redirects this command to the proper backend. doveadm sync ------------ This is an independent feature from all of the above. Even with none of it implemented, you could run this to replicate a user. Most of this is already implemented. The only problem is that currently you need to explicitly tell it where to sync. So, when the destination isn't specified, it could do a userdb lookup and use the returned "mail_replica" field as the destination. Multiple (sequentially replicated) destinations could be supported by returning "mail_replica2", "mail_replica3" etc. field. In NFS-based (or shared filesystem-based in general) setup the mail_replica setting is identical to mail_location setting. So your primary mail_location would be in /storage1/user/Maildir, while the secondary mail_replica would be in /storage2/user/Maildir. Simple. In non-NFS-based setup two Dovecot servers talk dsync protocol to each others. Currently dsync already supports SSH-based connections. It would also be easy to implement direct TCP-based connections between two doveadm servers. In future these connections could be SSL-encrypted. Initially I'm only supporting SSH-based connections, as they're already implemented. So what does the mail_replica setting look like in this kind of a setup? I'm not entirely sure. I'm thinking that it could be either "ssh:host" or "ssh:user@host", where user is the SSH login user (this is opposite of the current doveadm sync command line usage). In future then it could support also tcp:host[:port]. Both of these ssh: and tcp: prefixes would also be supported by doveadm sync command line usage (and perhaps the prefixless user@domain be deprecated). dsync can run without any long lived locking and it typically works fine. In case mailbox was modified during dsync, the replicas may not end up being identical, but nothing breaks. dsync currently usually notices this and logs a warning. When these conflicting changes was caused by imap/pop3/lda/etc. this isn't a problem, they've already notified replicator already to perform another sync that will fix it. Running two dsyncs at the same time is more problematic though, mainly related to new emails. Both dsyncs notice that mail X needs to be replicated, so both save it and it results in having a duplicate. To avoid this, there should be a dsync-lock. If this lock exists, dsync should wait until the previous dsync is done and then do it again, just in case there were more changes since the previous sync started. This should conclude everything needed for replication itself. High-availability NFS setup --------------------------- Once you have replication, it's of course nice if the system automatically recovers from a broken storage. In NFS-based setups the idea is to do soft mounts, so if the NFS server goes away things start failing with EIO errors, which Dovecot notices and switches to using the secondary storage(s). In v2.1.0 Dovecot already keeps track of mounted filesystems. Initially they're all marked as "online". When multiple I/O errors occur in a filesystem [todo: how many exactly? where are these errors checked, all around in the code or checking the log?] the mountpoint is marked as "offline" and the connections accessing that storage are killed [todo: again how exactly?]. Another job for replication plugin is to hook into namespace creation. If mail_location points to a mountpoint marked as "offline", it's replaced with mail_replica. This way the user can access mails from the secondary storage without downtime. If the replica isn't fully up to date, this means that some of the mails (or other changes) may temporarily be lost. These will come back again after the original storage has come back up and replication has finished its job. So as long as mails aren't lost in the original storage, there won't be any permanent mail loss. When an offline storage comes back online, its mountpoint's status is initially changed to "failover" (as opposed to "online"). During this state the replication plugin works a bit differently when the user's primary mail_location is in this storage: It first checks if the user is fully replicated, and if so uses the primary storage, otherwise it uses the replica storage. Long running IMAP protocesses check the replication state periodically and kill themselves once the user is replicated, to move back to primary storage. Once replicator notices that all users have been replicated, it tells the backends' to change the "failover" state to "online" (via doveadm server). High-availability non-NFS setup ------------------------------- One possibility is to use Dovecot proxies, which know which servers are down. Instead of directing users to those servers, it would direct them to replica servers. The server states could be handled similar to NFS setup's online vs. failover vs. offline states. Another possibility would be to do the same as above, except without separate proxy servers. Just make "mail.example.com" DNS point to two IP addresses, and if one Dovecot notices that it's not the user's primary server, it proxies to the secondary server, unless it's down. If one IP is down, clients hopefully connect to the other.

3 5

[Dovecot] keywords/flags questions
by Arnaud Abélard 03 Mar '12

03 Mar '12

Hello, I am currently using dovecot 2.0.13 and I have been working on keywords handling our webmail and I have a few questions about how dovecot handles them. First, if I am not mistaken keywords neeed to be UTF-7 encoded. That means I need to encode special caracters using values between & and - chars. UTF-7 encoding is case sensitive, &AOA- isn't the same chars as &aoa-. But docevot save keywords in lowercase or am I mistaken? For example: . STORE 1:1 flags &AOA-_refaire * 1 FETCH (FLAGS (&aoa-_refaire)) This makes retrieving the keyword properly impossible. What did I miss? My other question is about the permanent flags being displayed upon selecting a mailbox. I'm trying to understand why unused keywords are still showing up in there. Is there a way to force the definitive removal of a keyword from a mailbox? are old keywords kept undefinitely? Thanks in advance, Arnaud -- Arnaud Abélard jabber: arnaud.abelard(a)univ-nantes.fr / twitter: ArnY Administrateur Système DSI Université de Nantes -

3 2