- dovecot - dovecot.org

Re: [Dovecot] Replication plans
by Bill Boebel 18 May '07

18 May '07

On Fri, May 18, 2007 1:10 pm, Timo Sirainen <tss(a)iki.fi> said: > On Fri, 2007-05-18 at 12:20 -0400, Bill Boebel wrote: >> So what about tackling this replication problem from a different >> angle... Make it Dovecot's job to replicate the index and control >> files between servers, and make it the file system's job to replicate >> just the mail data. This would require further disconnecting the >> index and control files from the mail data, so that there is less >> syncing required. i.e. remove the need to check directory mtimes and >> to compare directory listings against the index; and instead assume >> that the indexes are always correct. Periodically you could still >> check to see if a sync is needed, but you'd do this must less >> frequently. > > This would practically mean that you want either cydir or dbox storage. > > This kind of a hybrid replication / clustered filesystem implementation > would simplify the replication a bit, but all the difficult things > related to UID conflicts etc. will still be there. So I wouldn't mind > implementing this, but I think implementing the message content sending > via TCP socket wouldn't add much more code anymore. > > The clustered filesystem could probably be used to simplify some things > though, such as UID allocation could be done by renaming a "uid-<next > uid>" file. If the rename() succeeded, you allocated the UID, otherwise > someone else did and you'll have to find the new filename and try again. > But I'm not sure if this kind of a special-case handling would be good. > Unless of course I decide to use the same thing for non-replicated > cydir/dbox. Dbox definitely sounds promising. I'd avoid putting this uid-<nextuid> file in the same location as the mail storage though, because if you can truly keep mail storage isolated and infrequently accessed, then you can do cool things like store your mail data remotely on Amazon S3 or equivalent.

1 0

Re: [Dovecot] LDAP for Virtual Domains
by Bryan Vyhmeister 18 May '07

18 May '07

On May 16, 2007, at 10:29 PM, razor wrote: > 17.05.07 в 05:06 Bryan Vyhmeister в своём письме > писал(а): > >> Is anyone using LDAP along with Dovecot where mail is being >> accessed in the form of /var/vmail/${domain}/${user}? I have not >> figured out how to extract the domain from LDAP in order to make >> this work. I know this is sparse information but maybe there is an >> easy fix. If not, I can post more information. >> >> Bryan > > i am using exim+dovecot+lda+openldap Thank you for the info. Bryan

1 0

[Dovecot] Wiki relating to subscriptions
by Bruce Bodger 17 May '07

17 May '07

On this wiki page http://wiki.dovecot.org/Migration/UW appears this information: "It's possible to keep using the .mailboxlist}} filename (as long as it's in the same directory) by modifying {{{SUBSCRIPTION_FILE_NAME define in src/lib-storage/index/mbox/mbox-storage.h" However, if a user wanted to continue using .mailboxlist wouldn't they also have to change the same define in src/lib-storage/index/ maildir/maildir-storage.h? Also, (regarding v1.0.0 code) the default define in mbox-storage.h is ".subscriptions" whereas the default define in maildir-storage.h is "subscriptions" (no dot) B. Bodger New York, NY

2 1

[Dovecot] Maildir format migration
by Tony Tsang 17 May '07

17 May '07

Hi all, I am planning to use dovecot IMAP on my server. Currently it is running UW-IMAP server with Maildir format storing users' mail. I've found that dovecot uses different Maildir format from UW-IMAP. Is there any configuration option for adapting dovecot to use format used by UW-IMAP? Thanks a lot. Tony.

4 7

[Dovecot] Processing too fast?
by Daniel L. Miller 17 May '07

17 May '07

I'm trying to use Dovecot to make administering ASSP a bit easier. I've created a mailbox with a number of subfolders (Spam, Ham, etc.) and have ASSP setup to save mail in the appropriate folder e.g. /var/mail/ASSP/.INBOX.Ham/new. But I'm having problems with ASSP complaining about the newly created files - I believe the file gets opened, then it disappears before ASSP finishes writing to it. Is Dovecot grabbing the file from "new" and moving it to "cur" faster than ASSP can write? How can I get around this? ASSP only allows for a folder to save each category to - I can't specify an LDA. -- Daniel

4 3

[Dovecot] CONVERT Plugin
by Bruce Bodger 17 May '07

17 May '07

Timo, Running v1.0.0 The convert plugin works pretty damn good! No need to mess with external utilities. One comment, though: Running here on OS X, I configured it with this convert_mail line: convert_mail = mbox:~/Library/Mail/IMAP/:INBOX=/var/mail/%u and mail_location = maildir:~/Maildir The IMAP directory was renamed to IMAP-converted after the process but the inbox (in /var/mail) does not get renamed. The WIKI sounds like it should be renamed to something, too. B. Bodger New York, NY

2 2

[Dovecot] Digest settings
by Kenny Dail 17 May '07

17 May '07

Timo, Wonder if you can tweak the digests to hold more messages? In 5 hours this morning, I've received 6 digests, seems a little excessive to me. -- Kenny Dail <kend(a)amigo.net>

2 1

[Dovecot] dbmail benchmarking
by Timo Sirainen 16 May '07

16 May '07

I thought I'd try benchmarking with dbmail (v2.2.4) to see how much slower a SQL backend could actually be. Skip to bottom for the conclusions. Originally I ran the tests with the databases being in XFS filesystem. MySQL's performance was horrible. It went 3-7x faster with ext3. MySQL 5.0.30 backend (innodb): ./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0 Logi Sele Appe 100% 100% 100% 5% 1 291 303 So that's 10 messages/sec saved. Now how about with 5 concurrent clients? Logi Sele Appe 100% 100% 100% 5% 5 1259 1332 Pretty well. Then something more generic: ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 30% 5% 1 37 36 75 73 110 34 24 73 78 Without Fetc (header/metadata fetching): Logi List Stat Sele Fet2 Stor Dele Expu Appe 100% 50% 50% 100% 100% 50% 100% 100% 100% 30% 5% 1 94 94 199 283 102 85 198 210 PostgreSQL 8.1.5 backend: ./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0 Logi Sele Appe 100% 100% 100% 5% 1 267 277 ./imaptest clients=5 - append=100 seed=1 secs=30 msgs=1000000 logout=0 Logi Sele Appe 100% 100% 100% 5% 5 1094 1144 ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 30% 5% 9 29 40 74 72 99 22 12 64 71 ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 fetch=0 Logi List Stat Sele Fet2 Stor Dele Expu Appe 100% 50% 50% 100% 100% 50% 100% 100% 100% 30% 5% 35 105 95 200 277 54 70 165 175 The last two tests gave "Unexpected tagged reply:" errors that I didn't get with MySQL. So apparently there's some PostgreSQL-specific bug in dbmail. The same values with Dovecot 1.0 + maildir: ./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0 Logi Sele Appe 100% 100% 100% 5% 1 346 364 ./imaptest clients=5 - append=100 seed=1 secs=30 msgs=1000000 logout=0 Logi Sele Appe 100% 100% 100% 5% 5 1408 1470 ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 30% 5% 1 130 133 259 258 382 123 127 257 271 ./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 fetch=0 Logi List Stat Sele Fet2 Stor Dele Expu Appe 100% 50% 50% 100% 100% 50% 100% 100% 100% 30% 5% 1 155 163 339 478 169 175 338 354 So, what are the conclusions? - In raw append speed dbmail is almost as fast as maildir. - In raw read/write speed maildir is about 1,6 times faster - When adding metadata fetches Dovecot is 4 times faster than dbmail. This is most likely because dbmail doesn't have a cache equivalent to dovecot.index.cache so it has to do the fetches the slow way. I would have liked to also run the generic tests with more than 1 client, but then I start hitting dbmail bugs and the test stops (reported already to their bugtracker).

3 5

[Dovecot] OS X Mail Administration
by Bruce Bodger 16 May '07

16 May '07

I'd like to request some feed back from any OS X Mail Admins on the list, please. Currently running 10.4.9 here using postfix / dovecot and I'm about to take the plunge from mbox to maildir. I just migrated from uw-imap to dovecot. As you're aware, users inbox's are customarily stored in /var/mail/% username and personal folders under /Users/%username/Library/Mail/IMAP. I'd appreciate if you'd share with me the directory structure that you use, or would use in my situation. All mail storage will be on the server so disk space is not an issue. I'm a bit confused whether or not to keep the inbox separate from the other imap structure as is the OS X default: should all mailboxes be under inbox, etc. Your input would be appreciated. Thank you, B. Bodger New York, NY

3 3

[Dovecot] Maildir POP3 "UID larger than next_uid" bug
by Alexander Moisseev 16 May '07

16 May '07

dovecot 1.0.0; Free BSD 6.2; x86; UFS; no NFS IMAP not uses at all, POP3 only Problem description: POP3 client can't fetch second e-mail message from Maildir if first POP3 connect occurs before any message placed in Maildir. Details: 1. POP3-client first time tries to receive mail: May 14 16:17:36 host dovecot: pop3-login: Login: user=<user(a)mydomain.ru>, method=PLAIN, rip=192.168.0.100, lip=192.168.0.3 May 14 16:17:36 host dovecot: POP3(user(a)mydomain.ru): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 dovecot creates Maildir for that user: /var/mail/virtual/mydomain.ru/user/Maildir/ /cur /new /tmp dovecot.index dovecot.index.cache dovecot.index.log 2. Postfix MTA receives message for that user and places it at /var/mail/virtual/mydomain.ru/user/Maildir/new/1179145269.V43I3f5a4M116738.host.mydomain.local 3. POP3-client tries to receive mail: May 14 16:23:09 host dovecot: pop3-login: Login: user=<user(a)mydomain.ru>, method=PLAIN, rip=192.168.0.100, lip=192.168.0.3 May 14 16:23:09 host dovecot: POP3(user(a)mydomain.ru): Disconnected: Logged out top=0/0, retr=1/786, del=1/1, size=770 >>>>> dovecot creates file dovecot-uidlist with next_uid=1 (!!!): 1 1179145056 1 1 1179145269.V43I3f5a4M116738.host.mydomain.local:2, Message fetched successfully. 4. POP3-client tries to receive mail: May 14 16:23:29 host dovecot: pop3-login: Login: user=<user(a)mydomain.ru>, method=PLAIN, rip=192.168.0.100, lip=192.168.0.3 May 14 16:23:29 host dovecot: POP3(user(a)mydomain.ru): UID larger than next_uid in file /var/mail/virtual/mydomain.ru/user/Maildir/dovecot-uidlist (1 >= 1) May 14 16:23:29 host dovecot: POP3(user(a)mydomain.ru): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 File dovecot-uidlist disappears. 5. Postfix MTA receives message for that user and places it at /var/mail/virtual/mydomain.ru/user/Maildir/new/1179145534.V43I3f5a4M116827.host.mydomain.local 6. POP3-client tries to receive mail: May 14 16:47:20 host dovecot: pop3-login: Login: user=<user(a)mydomain.ru>, method=PLAIN, rip=192.168.0.100, lip=192.168.0.3 May 14 16:47:20 host dovecot: POP3(user(a)mydomain.ru): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 Dovecot creates file dovecot-uidlist: 1 1179145056 2 1 1179145534.V43I3f5a4M116827.host.mydomain.local Dovecot moves message from new to cur directory and adds flag ":2,": /var/mail/virtual/mydomain.ru/user/Maildir/cur/1179145534.V43I3f5a4M116827.host.mydomain.local:2, Message not fetched. And POP3 client can't fetch it at all. All next messages fetches normally as they arrives. # /usr/local/etc/dovecot.conf protocols: pop3 pop3s ssl_cert_file: /etc/ssl/certs/host.mydomain.ru.crt ssl_key_file: /etc/ssl/private/host.mydomain.ru.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/pop3-login login_greeting: POP3 server ready verbose_proctitle: yes first_valid_gid: 0 mail_extra_groups: mail mail_location: maildir:~/Maildir mail_executable: /usr/local/libexec/dovecot/pop3 mail_plugin_dir: /usr/local/lib/dovecot/pop3 pop3_uidl_format: %08Xu%08Xv pop3_client_workarounds: outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login digest-md5 cram-md5 apop ntlm default_realm: mydomain.ru passdb: driver: pam passdb: driver: passwd-file args: /etc/auth/%d/passwd userdb: driver: passwd userdb: driver: static args: uid=5000 gid=5000 home=/var/mail/virtual/%d/%n socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:

2 1