May 2007 - dovecot - dovecot.org

[Dovecot] Index file rewrite status
by Timo Sirainen 30 May '07

30 May '07

The biggest problem with getting v1.0 out was getting its index file code stable. v1.1 plans included doing large changes to the index file code, so it's important to get the new code stable as soon as possible. Since I've managed to stay pretty productive for the last few weeks, I've been mostly just coding the index changes. Once I'm sure that the code is again fully working I'll commit it to hg, relax a bit and then I'll again start answering mails in this list. It looks like the code is almost working, but it's still giving failures once in a while. Below are anyway a few benchmarks for you to see how it's progressing. I haven't yet done all the optimizations I've planned, but looks like it already gives 25-30% better performance in cydir tests. The cydir benchmarks are the most important ones because cydir relies completely on index files, so the benchmarks are practically index benchmarks. And the 25-30% is mostly about CPU usage I think, reduced dovecot.index file rewrites will help with disk I/O as well. The new index code should be pretty NFS friendly. I think there is going to be only one place where I'll need to force attribute cache flush and then probably a couple of places where data cache needs to be flushed (I'm not exactly sure about that yet, I'll have to know more about NFS data caching). One other thing I'm hoping to implement is that the index functions would almost never fail. If anything breaks, the indexes should be moved to memory. Any corrupted data should be ignored and anything that looks good would still be used (important with cydir/dbox). I've even managed to remove about 500 lines of code: 26 files changed, 3258 insertions(+), 3753 deletions(-) (about 2000 insert+deletions are just cut&pasting code to different files) Append-only tests: mmap_disable=no, fsync_disable=yes Current hg (before index rewrite), cydir: ./imaptest - append=100,0 logout=0 select=0 clients=1 secs=5 Logi Appe 100% 100% 1 2602 1/ 1 0 2671 1/ 1 0 2750 1/ 1 0 2363 1/ 1 Totals: 1 13206 Index rewrite, cydir: Totals: Logi Appe 100% 100% 1 4251 1/ 1 0 3914 1/ 1 0 3514 1/ 1 0 2985 1/ 1 0 2538 1/ 1 Totals: 1 17202 Maildir (not much difference between rewrite/no rewrite): Logi Appe 100% 100% 1 917 1/ 1 0 498 1/ 1 0 284 1/ 1 0 273 1/ 1 0 296 1/ 1 Totals: 1 2268 mbox (+300 with index rewrite): Logi Appe 100% 100% 1 1827 1/ 1 0 1688 1/ 1 0 1811 1/ 1 0 2040 1/ 1 Totals: 1 9114 Then a bit more generic test: Old index code, cydir: ./imaptest secs=10 clients=10 Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 3417 1713 1713 3416 3403 4840 1719 1260 3403 1999 6816 New index code, cydir: Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 4686 2349 2317 4678 4675 6636 2323 1628 4675 2585 9353

1 0

[Dovecot] Converting mbox to Maildir, all prior read messages are downloaded as new
by Eric and Barbara Sammons 30 May '07

30 May '07

I am running dovecot 0.99. Everything works great; however, now I am looking at several different tools to convert mbox to Maildir. It seems the tools, mb2md.pl, perfect_maildir.pl, and others all do what they claim; however, when the messages are converted and they are stored in Maildir/cur, the email clients download these messages as new. This typically wouldn't be a problem; however, I am dealing with 100s of users with 1000s of emails and this really isn't acceptable behavior to them. Any thoughts on how to correct this issue and prevent the client from downloading these messages again? As a note, my test client is thunderbird and my preferred mbox to Maildir tool is mb2md-3.20.pl. Thanks!

5 6

[Dovecot] deliver rejection message
by Timo Sirainen 30 May '07

30 May '07

Currently the typical rejection message is: --------- Your message was automatically rejected by Dovecot Mail Delivery Agent. The following reason was given: Quota exceeded. --------- Then there is MDN + message headers in other MIME parts. But of course there sucky clients that can't display MDNs and users get confused. Any suggestions how to improve this message? I'm not sure if the "Dovecot Mail Delivery Agent" even needs to be in there. How about something like: --------- Your message to <foo(a)bar.org> was automatically rejected: Quota exceeded. See the attachment for the original message's headers. --------- And should I change this for v1.0.1 already?

5 6

Re: [Dovecot] [dovecot] IMAP basics
by Paul Scott 30 May '07

30 May '07

Picking up a broken thread after gettin exim4 to send my mail out. version 1.0.0 on Debian sid: I now have fetchmail delivering three POP mailboxes to my Maildir. How do I tell Dovecot about this mail? TIA, Paul Scott

1 0

Re: [Dovecot] Dovecot] dovecot performance question
by Kenny Dail 30 May '07

30 May '07

> Hi, using the latest dovecot with pop3/imap. Using mostly outlook 2003 for > pop3 and squirrel mail imap. > > I have a lot of users reporting back that imap is very slow. We average > about 300 imap and another 700 pop session at any given time. reporting that imap is slow or that squirrelmail is slow? ;) How is the load average on the server? We found at our site to keep Apache/squirrelmail happy, we dedicated a server for our webmail users. Mostly being a RAM issue in that case. Just how beefy is the server you are using? > I am having a time issue on this server and im not sure if its affecting > dovecot. > > May 29 11:42:19 pop dovecot: POP3(xxx): Time just moved backwards by 1 > seconds. I'll sleep now until w > e're back in present. > May 29 11:42:19 pop dovecot: POP3(xxx): Time just moved backwards by 1 > seconds. I'll sleep now until w > e're back in present. > May 29 11:42:31 pop dovecot: POP3(xxx): Time just moved backwards by 1 > seconds. I'll sleep now until w > e're back in present. > May 29 11:42:42 pop dovecot: POP3(xxx): Time just moved backwards by 1 > seconds. I'll sleep now until w > e're back in present. It'll affect it in that you will be having 1 sec delays frequently it seems. You should definitely look into fixing that. That is bad behavior for a busy mail server. -- Kenny Dail <kend(a)amigo.net>

2 1

[Dovecot] namespace problem
by Rainer Sigl 29 May '07

29 May '07

Hi all, after an update from dovecot beta version to 1.0.0.-1 today morning I had different effects: thunderbird-, kmail-, Apple-clients, pine and squirrelmail could receive email correct with imaps. But not MSOutlook!!! Varying the namespace configurations in dovecot.conf I got different results with MSOutlook working but with thunderbird, squirrelmail etc. all folders are displayed but not the root Inbox etc. Its important to know that I migrated from courier-imap to dovecot. My well working namespace config BEFORE upgrade to 1.0.0-1: namespace private { #separator = / prefix = INBOX. # prefix = inbox = yes #hidden = yes } #the second namespace I need to work with Apple because they use / namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } Here my actual namespace configs: namespace private { separator = prefix = INBOX. inbox = no hidden = no } namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } namespace private { separator = . prefix = INBOX. inbox = yes hidden = yes } In varying inbox and hidden keywords I found this configuration. Most things are working with this but in squirrelmal I can't access inbox, the main folder. How do I have to configure that everythings works satisfying?? Many thanks Rainer Sigl [0]root@mpemail:/etc/dovecot# dovecot -n # /etc/dovecot/dovecot.conf log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot-info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps pop3s ssl_cert_file: /etc/ssl/certs/mpemail.pem ssl_key_file: /etc/ssl/private/mpemail-key.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_chroot: no verbose_proctitle: yes mail_extra_groups: mail default_mail_env: maildir:/home/virtual_mail/%u mail_location: maildir:/home/virtual_mail/%u mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): UID%u-%v namespace: type: private prefix: INBOX. namespace: type: private separator: / prefix: INBOX/ hidden: yes namespace: type: private separator: . prefix: INBOX. inbox: yes hidden: yes auth default: passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=1002 gid=1002 home=/home/virtual_mail/%u userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf -- ----------------- Rainer Sigl, | email: sigl(a)mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 30000-3557 Postfach 1312, | fax: +49 (89) 30000-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/

1 0

[Dovecot] Namespace Problems repost without attachment
by Rainer Sigl 29 May '07

29 May '07

Hi all, after an update from dovecot beta version to 1.0.0.-1 today morning I had different effects: thunderbird-, kmail-, Apple-clients, pine and squirrelmail could receive email correct with imaps. But not MSOutlook!!! Varying the namespace configurations in dovecot.conf I got different results with MSOutlook working but with thunderbird, squirrelmail etc. all folders are displayed but not the root Inbox etc. Its important to know that I migrated from courier-imap to dovecot. My well working namespace config BEFORE upgrade to 1.0.0-1: namespace private { #separator = / prefix = INBOX. # prefix = inbox = yes #hidden = yes } #the second namespace I need to work with Apple because they use / namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } Here my actual namespace configs: namespace private { separator = prefix = INBOX. inbox = no hidden = no } namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } namespace private { separator = . prefix = INBOX. inbox = yes hidden = yes } In varying inbox and hidden keywords I found this configuration. Most things are working with this but in squirrelmal I can't access inbox, the main folder. How do I have to configure that everythings works satisfying?? Many thanks Rainer Sigl -- ----------------- Rainer Sigl, | email: sigl(a)mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 30000-3557 Postfach 1312, | fax: +49 (89) 30000-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/

1 0

[Dovecot] Namespace Problems
by Rainer Sigl 29 May '07

29 May '07

Hi all, after an update from dovecot beta version to 1.0.0.-1 today morning I had different effects: thunderbird-, kmail-, Apple-clients, pine and squirrelmail could receive email correct with imaps. But not MSOutlook!!! Varying the namespace configurations in dovecot.conf I got different results with MSOutlook working but with thunderbird, squirrelmail etc. all folders are displayed but not the root Inbox etc. Its important to know that I migrated from courier-imap to dovecot. My well working namespace config BEFORE upgrade to 1.0.0-1: namespace private { #separator = / prefix = INBOX. # prefix = inbox = yes #hidden = yes } #the second namespace I need to work with Apple because they use / namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } Here my actual namespace configs: namespace private { separator = prefix = INBOX. inbox = no hidden = no } namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } namespace private { separator = . prefix = INBOX. inbox = yes hidden = yes } In varying inbox and hidden keywords I found this configuration. Most things are working with this but in squirrelmal I can't access inbox, the main folder. How do I have to configure that everythings works satisfying?? dovecot.conf and dovecot-sql.conf I send in the attachment. Many thanks Rainer Sigl -- ----------------- Rainer Sigl, | email: sigl(a)mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 30000-3557 Postfach 1312, | fax: +49 (89) 30000-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ -- ----------------- Rainer Sigl, | email: sigl(a)mpe.mpg.de MPI f. extraterrestrische Physik, | phone: +49 (89) 30000-3557 Postfach 1312, | fax: +49 (89) 30000-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/ ## Dovecot configuration file # '#' character and everything after it is treated as comments. Extra spaces # and tabs are ignored. If you want to use either of these explicitly, put the # value inside quotes, eg.: key = "# char and trailing whitespace " # Default values are shown after each value, it's not required to uncomment # any of the lines. Exception to this are paths, they're just examples # with real defaults being based on configure options. The paths listed here # are for configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var # --with-ssldir=/etc/ssl # Base directory where to store runtime data. #base_dir = /var/run/dovecot/ # Protocols we want to be serving: # imap imaps pop3 pop3s #protocols = imap imaps protocols = imaps pop3s # IP or host address where to listen in for connections. It's not currently # possible to specify multiple addresses. "*" listens in all IPv4 interfaces. # "[::]" listens in all IPv6 interfaces, but may also listen in all IPv4 # interfaces depending on the operating system. If you want to specify ports # for each service, you will need to configure these settings inside the # protocol imap/pop3 { ... } section, so you can specify different ports # for IMAP/POP3. #listen = * # IP or host address where to listen in for SSL connections. Defaults # to above if not specified. #ssl_listen = # Disable SSL/TLS support. #ssl_disable = no # PEM encoded X.509 SSL/TLS certificate and private key. They're opened before # dropping root privileges, so keep the key file unreadable by anyone but # root. #ssl_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem #ssl_key_file = /etc/ssl/private/ssl-cert-snakeoil.key ssl_cert_file = /etc/ssl/certs/mpemail.pem ssl_key_file = /etc/ssl/private/mpemail-key.pem # If key file is password protected, give the password here. Alternatively # give it when starting dovecot with -p parameter. #ssl_key_password = # File containing trusted SSL certificate authorities. Usually not needed. #ssl_ca_file = # Request client to send a certificate. #ssl_verify_client_cert = no # How often to regenerate the SSL parameters file. Generation is quite CPU # intensive operation. The value is in hours, 0 disables regeneration # entirely. #ssl_parameters_regenerate = 168 # SSL ciphers to use #ssl_cipher_list = ALL:!LOW # Disable LOGIN command and all other plaintext authentications unless # SSL/TLS is used (LOGINDISABLED capability). Note that 127.*.*.* and # IPv6 ::1 addresses are considered secure, this setting has no effect if # you connect from those addresses. #disable_plaintext_auth = yes # Use this logfile instead of syslog(). /dev/stderr can be used if you want to # use stderr for logging (ONLY /dev/stderr - otherwise it is closed). log_path = /var/log/dovecot.log # For informational messages, use this logfile instead of the default info_log_path = /var/log/dovecot-info.log # Prefix for each line written to log file. % codes are in strftime(3) # format. log_timestamp = "%Y-%m-%d %H:%M:%S " # Syslog facility to use if you're logging to syslog. Usually if you don't # want to use "mail", you'll use local0..local7. Also other standard # facilities are supported. #syslog_facility = mail ## ## Login processes ## # Directory where authentication process places authentication UNIX sockets # which login needs to be able to connect to. The sockets are created when # running as root, so you don't have to worry about permissions. Note that # everything in this directory is deleted when Dovecot is started. #login_dir = /var/run/dovecot/login # chroot login process to the login_dir. Only reason not to do this is if you # wish to run the whole Dovecot without roots. # http://wiki.dovecot.org/Rootless login_chroot = no # User to use for the login process. Create a completely new user for this, # and don't use it anywhere else. The user must also belong to a group where # only it has access, it's used to control access for authentication process. # Note that this user is NOT used to access mails. # http://wiki.dovecot.org/UserIds login_user = dovecot # Set max. process size in megabytes. If you don't use # login_process_per_connection you might need to grow this. #login_process_size = 32 # Should each login be processed in it's own process (yes), or should one # login process be allowed to process multiple connections (no)? Yes is more # secure, espcially with SSL/TLS enabled. No is faster since there's no need # to create processes all the time. #login_process_per_connection = yes # Number of login processes to create. If login_process_per_connection is # yes, this is the number of extra processes waiting for users to log in. #login_processes_count = 3 # Maximum number of extra login processes to create. The extra process count # usually stays at login_processes_count, but when multiple users start logging # in at the same time more extra processes are created. To prevent fork-bombing # we check only once in a second if new processes should be created - if all # of them are used at the time, we double their amount until limit set by this # setting is reached. This setting is used only if login_process_per_use is yes. #login_max_processes_count = 128 # Maximum number of connections allowed in login state. When this limit is # reached, the oldest connections are dropped. If login_process_per_connection # is no, this is a per-process value, so the absolute maximum number of users # logging in actually login_processes_count * max_logging_users. #login_max_logging_users = 256 # Greeting message for clients. #login_greeting = Dovecot ready. # Space-separated list of elements we want to log. The elements which have # a non-empty variable value are joined together to form a comma-separated # string. #login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c # Login log format. %$ contains login_log_format_elements string, %s contains # the data we want to log. #login_log_format = %$: %s ## ## Mail processes ## # Maximum number of running mail processes. When this limit is reached, # new users aren't allowed to log in. #max_mail_processes = 1024 # Show more verbose process titles (in ps). Currently shows user name and # IP address. Useful for seeing who are actually using the IMAP processes # (eg. shared mailboxes or if same uid is used for multiple accounts). verbose_proctitle = yes # Show protocol level SSL errors. #verbose_ssl = no # Valid UID range for users, defaults to 500 and above. This is mostly # to make sure that users can't log in as daemons or other system users. # Note that denying root logins is hardcoded to dovecot binary and can't # be done even if first_valid_uid is set to 0. #first_valid_uid = 500 #last_valid_uid = 0 # Valid GID range for users, defaults to non-root/wheel. Users having # non-valid GID as primary group ID aren't allowed to log in. If user # belongs to supplementary groups with non-valid GIDs, those groups are # not set. #first_valid_gid = 1 #last_valid_gid = 0 # Grant access to these extra groups for mail processes. Typical use would be # to give "mail" group write access to /var/mail to be able to create dotlocks. mail_extra_groups = mail # ':' separated list of directories under which chrooting is allowed for mail # processes (ie. /var/mail will allow chrooting to /var/mail/foo/bar too). # This setting doesn't affect login_chroot or auth_chroot variables. # WARNING: Never add directories here which local users can modify, that # may lead to root exploit. Usually this should be done only if you don't # allow shell access for users. See # /usr/share/doc/dovecot-common/configuration.txt for more information. #valid_chroot_dirs = # Default chroot directory for mail processes. This can be overridden for # specific users in user database by giving /./ in user's home directory # (eg. /home/./user chroots into /home). Note that usually there is no real # need to do chrooting, Dovecot doesn't allow users to access files outside # their mail directory anyway. #mail_chroot = # Enable mail process debugging. This can help you figure out why Dovecot # isn't finding your mails. mail_debug = yes # Default MAIL environment to use when it's not set. By leaving this empty # dovecot tries to do some automatic detection as described in # /usr/share/doc/dovecot-common/mail-storages.txt. There's a few special # variables you can use, eg.: # # %u - username # %n - user part in user@domain, same as %u if there's no domain # %d - domain part in user@domain, empty if there's no domain # %h - home directory # # See /usr/share/doc/dovecot-common/variables.txt for full list. Some examples: # # default_mail_env = maildir:/var/mail/%1u/%u/Maildir # default_mail_env = mbox:~/mail/:INBOX=/var/mail/%u # default_mail_env = mbox:/var/mail/%d/%n/:INDEX=/var/indexes/%d/%n # default_mail_env = maildir:/home/virtual_mail/%u # If you need to set multiple mailbox locations or want to change default # namespace settings, you can do it by defining namespace sections: # # You can have private, shared and public namespaces. The only difference # between them is how Dovecot announces them to client via NAMESPACE # extension. Shared namespaces are meant for user-owned mailboxes which are # shared to other users, while public namespaces are for more globally # accessible mailboxes. # # REMEMBER: If you add any namespaces, the default namespace must be added # explicitly, ie. default_mail_env does nothing unless you have a namespace # without a location setting. Default namespace is simply done by having a # namespace with empty prefix. namespace private { separator = prefix = INBOX. inbox = no hidden = no } namespace private { separator = / prefix = INBOX/ inbox = no hidden = yes } namespace private { separator = . prefix = INBOX. inbox = yes hidden = yes } #namespace private { # Hierarchy separator to use. You should use the same separator for all # namespaces or some clients get confused. '/' is usually a good one. # separator = / # Prefix required to access this namespace. This needs to be different for # all namespaces. For example "Public/". # prefix = INBOX/ # prefix = # Physical location of the mailbox. This is in same format as # default_mail_env, which is also the default for it. #location = # There can be only one INBOX, and this setting defines which namespace # has it. # inbox = no # If namespace is hidden, it's not advertised to clients via NAMESPACE # extension or shown in LIST replies. This is mostly useful when converting # from another server with different namespaces which you want to depricate # but still keep working. For example you can create hidden namespaces with # prefixes "~/mail/", "~%u/mail/" and "mail/". # hidden = yes #} # Space-separated list of fields to initially save into cache file. Currently # these fields are allowed: # Space-separated list of fields to initially save into cache file. Currently # these fields are allowed: # # flags, date.sent, date.received, size.virtual, size.physical # mime.parts, imap.body, imap.bodystructure # # Different IMAP clients work in different ways, so they benefit from # different cached fields. Some do not benefit from them at all. Caching more # than necessary generates useless disk I/O, so you don't want to do that # either. # # Dovecot attempts to automatically figure out what client wants and it keeps # only that. However the first few times a mailbox is opened, Dovecot hasn't # yet figured out what client needs, so it may not perform optimally. If you # know what fields the majority of your clients need, it may be useful to set # these fields by hand. If client doesn't actually use them, Dovecot will # eventually drop them. # # Usually you should just leave this field alone. The potential benefits are # typically unnoticeable. #mail_cache_fields = # Space-separated list of fields that Dovecot should never save to cache file. # Useful if you want to save disk space at the cost of more I/O when the fields # needed. #mail_never_cache_fields = # The minimum number of mails in a mailbox before updates are done to cache # file. This allows optimizing Dovecot's behavior to do less disk writes at # the cost of more disk reads. #mail_cache_min_mail_count = 0 # When IDLE command is running, mailbox is checked once in a while to see if # there are any new mails or other changes. This setting defines the minimum # time to wait between those checks. Dovecot is however able to use dnotify # and inotify with Linux to reply immediately after the change occurs. #mailbox_idle_check_interval = 30 # Allow full filesystem access to clients. There's no access checks other than # what the operating system does for the active UID/GID. It works with both # maildir and mboxes, allowing you to prefix mailboxes names with eg. /path/ # or ~user/. #mail_full_filesystem_access = no # Maximum allowed length for mail keyword name. It's only forced when trying # to create new keywords. #mail_max_keyword_length = 50 # Save mails with CR+LF instead of plain LF. This makes sending those mails # take less CPU, especially with sendfile() syscall with Linux and FreeBSD. # But it also creates a bit more disk I/O which may just make it slower. # Also note that if other software reads the mboxes/maildirs, they may handle # the extra CRs wrong and cause problems. #mail_save_crlf = no # Use mmap() instead of read() to read mail files. read() seems to be a bit # faster with my Linux/x86 and it's better with NFS, so that's the default. # Note that OpenBSD 3.3 and older don't work right with mail_read_mmaped = yes. #mail_read_mmaped = no # Don't use mmap() at all. This is required if you store indexes in remote # filesystems (NFS or clustered filesystem). #mmap_disable = no # Don't write() to mmaped files. This is required for some operating systems # which use separate caches for them, such as OpenBSD. #mmap_no_write = no # Locking method for index files. Alternatives are fcntl, flock and dotlock. # Dotlocking uses some tricks which may create more disk I/O than other locking # methods. NOTE: If you use NFS, remember to change also mmap_disable setting! # Solaris doesn't support flock, so Solaris users need to change this to fcntl. #lock_method = flock # By default LIST command returns all entries in maildir beginning with dot. # Enabling this option makes Dovecot return only entries which are directories. # This is done by stat()ing each entry, so it causes more disk I/O. # (For systems setting struct dirent->d_type, this check is free and it's # done always regardless of this setting) #maildir_stat_dirs = no # Copy mail to another folders using hard links. This is much faster than # actually copying the file. This is problematic only if something modifies # the mail in one folder but doesn't want it modified in the others. I don't # know any MUA which would modify mail files directly. IMAP protocol also # requires that the mails don't change, so it would be problematic in any case. # If you care about performance, enable it. #maildir_copy_with_hardlinks = no # Which locking methods to use for locking mbox. There's four available: # dotlock: Create <mailbox>.lock file. This is the oldest and most NFS-safe # solution. If you want to use /var/mail/ like directory, the users # will need write access to that directory. # fcntl : Use this if possible. Works with NFS too if lockd is used. # flock : May not exist in all systems. Doesn't work with NFS. # lockf : May not exist in all systems. Doesn't work with NFS. # # You can use multiple locking methods; if you do the order they're declared # in is important to avoid deadlocks if other MTAs/MUAs are using multiple # locking methods as well. Some operating systems don't allow using some of # them simultaneously. #mbox_read_locks = fcntl #mbox_write_locks = dotlock fcntl # Maximum time in seconds to wait for lock (all of them) before aborting. #mbox_lock_timeout = 300 # If dotlock exists but the mailbox isn't modified in any way, override the # lock file after this many seconds. #mbox_dotlock_change_timeout = 30 # When mbox changes unexpectedly we have to fully read it to find out what # changed. If the mbox is large this can take a long time. Since the change # is usually just a newly appended mail, it'd be faster to simply read the # new mails. If this setting is enabled, Dovecot does this but still safely # fallbacks to re-reading the whole mbox file whenever something in mbox isn't # how it's expected to be. The only real downside to this setting is that if # some other MUA changes message flags, Dovecot doesn't notice it immediately. # Note that a full sync is done with SELECT, EXAMINE, EXPUNGE and CHECK # commands. #mbox_dirty_syncs = yes # Like mbox_dirty_syncs, but don't do full syncs even with SELECT, EXAMINE, # EXPUNGE or CHECK commands. If this is set, mbox_dirty_syncs is ignored. #mbox_very_dirty_syncs = no # Delay writing mbox headers until doing a full write sync (EXPUNGE and CHECK # commands and when closing the mailbox). This is especially useful for POP3 # where clients often delete all mails. The downside is that our changes # aren't immediately visible to other MUAs. #mbox_lazy_writes = yes # If mbox size is smaller than this (in kilobytes), don't write index files. # If an index file already exists it's still read, just not updated. #mbox_min_index_size = 0 # Maximum dbox file size in kilobytes until it's rotated. #dbox_rotate_size = 2048 # Minimum dbox file size in kilobytes before it's rotated # (overrides dbox_rotate_days) #dbox_rotate_min_size = 16 # Maximum dbox file age in days until it's rotated. Day always begins from # midnight, so 1 = today, 2 = yesterday, etc. 0 = check disabled. #dbox_rotate_days = 0 # umask to use for mail files and directories #umask = 0077 # Drop all privileges before exec()ing the mail process. This is mostly # meant for debugging, otherwise you don't get core dumps. It could be a small # security risk if you use single UID for multiple users, as the users could # ptrace() each others processes then. #mail_drop_priv_before_exec = no # Set max. process size in megabytes. Most of the memory goes to mmap()ing # files, so it shouldn't harm much even if this limit is set pretty high. #mail_process_size = 256 # Log prefix for mail processes. See # /usr/share/doc/dovecot-common/variables.txt for list of possible variables #you can use. #mail_log_prefix = "%Us(%u): " ## ## IMAP specific settings ## protocol imap { # Login executable location. #login_executable = /usr/lib/dovecot/imap-login # IMAP executable location. Changing this allows you to execute other # binaries before the imap process is executed. # # This would write rawlogs into ~/dovecot.rawlog/ directory: # mail_executable = /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap # # This would attach gdb into the imap process and write backtraces into # /tmp/gdbhelper.* files: # mail_executable = /usr/lib/dovecot/gdbhelper /usr/lib/dovecot/imap # #mail_executable = /usr/lib/dovecot/imap # Maximum IMAP command line length in bytes. Some clients generate very long # command lines with huge mailboxes, so you may need to raise this if you get # "Too long argument" or "IMAP command line too large" errors often. #imap_max_line_length = 65536 # Support for dynamically loadable modules. #mail_use_modules = no #mail_modules = /usr/lib/dovecot/modules/imap # Send IMAP capabilities in greeting message. This makes it unnecessary for # clients to request it with CAPABILITY command, so it saves one round-trip. # Many clients however don't understand it and ask the CAPABILITY anyway. #login_greeting_capability = no # Workarounds for various client bugs: # delay-newmail: # Send EXISTS/RECENT new mail notifications only when replying to NOOP # and CHECK commands. Some clients ignore them otherwise, for example # OSX Mail. Outlook Express breaks more badly though, without this it # may show user "Message no longer in server" errors. Note that OE6 still # breaks even with this workaround if synchronization is set to # "Headers Only". # outlook-idle: # Outlook and Outlook Express never abort IDLE command, so if no mail # arrives in half a hour, Dovecot closes the connection. This is still # fine, except Outlook doesn't connect back so you don't see if new mail # arrives. # netscape-eoh: # Netscape 4.x breaks if message headers don't end with the empty "end of # headers" line. Normally all messages have this, but setting this # workaround makes sure that Netscape never breaks by adding the line if # it doesn't exist. This is done only for FETCH BODY[HEADER.FIELDS..] # commands. Note that RFC says this shouldn't be done. # tb-extra-mailbox-sep: # With mbox storage a mailbox can contain either mails or submailboxes, # but not both. Thunderbird separates these two by forcing server to # accept '/' suffix in mailbox names in subscriptions list. # The list is space-separated. #imap_client_workarounds = outlook-idle } ## ## POP3 specific settings ## protocol pop3 { # Login executable location. #login_executable = /usr/lib/dovecot/pop3-login # POP3 executable location #mail_executable = /usr/lib/dovecot/pop3 # Don't try to set mails non-recent or seen with POP3 sessions. This is # mostly intended to reduce disk I/O. With maildir it doesn't move files # from new/ to cur/, with mbox it doesn't write Status-header. #pop3_no_flag_updates = no # Support LAST command which exists in old POP3 specs, but has been removed # from new ones. Some clients still wish to use this though. Enabling this # makes RSET command clear all \Seen flags from messages. #pop3_enable_last = no # POP3 UIDL format to use. You can use following variables: # # %v - Mailbox UIDVALIDITY # %u - Mail UID # %m - MD5 sum of the mailbox headers in hex (mbox only) # %f - filename (maildir only) # # If you want UIDL compatibility with other POP3 servers, use: # UW's ipop3d : %08Xv%08Xu # Courier version 0 : %f # Courier version 1 : %u # Courier version 2 : %v-%u # Cyrus (<= 2.1.3) : %u # Cyrus (>= 2.1.4) : %v.%u # Older Dovecots : %v.%u # # Note that Outlook 2003 seems to have problems with %v.%u format which was # Dovecot's default, so if you're building a new server it would be a good # idea to change this. %08Xu%08Xv should be pretty fail-safe. # # NOTE: Nowadays this is required to be set explicitly, since the old # default was bad but it couldn't be changed without breaking existing # installations. %08Xu%08Xv will be the new default, so use it for new # installations. # pop3_uidl_format = UID%u-%v # POP3 logout format string: # %t - number of TOP commands # %T - number of bytes sent to client as a result of TOP command # %r - number of RETR commands # %R - number of bytes sent to client as a result of RETR command # %d - number of deleted messages # %m - number of messages (before deletion) # %s - mailbox size in bytes (before deletion) #pop3_logout_format = top=%t/%T, retr=%r/%R, del=%d/%m, size=%s # Support for dynamically loadable modules. #mail_use_modules = no #mail_modules = /usr/lib/dovecot/modules/pop3 # Workarounds for various client bugs: # outlook-no-nuls: # Outlook and Outlook Express hang if mails contain NUL characters. # This setting replaces them with 0x80 character. # oe-ns-eoh: # Outlook Express and Netscape Mail breaks if end of headers-line is # missing. This option simply sends it if it's missing. # The list is space-separated. #pop3_client_workarounds = } ## ## dovecot-lda specific settings ## # protocol lda { # If you wish to use plugins you need to specify plugin directory # For example quota enforcing is implemented by plugin #module_dir = /usr/local/lib/dovecot/lda # Address from LDA should send MDNs like out of quota # postmaster_address = postmaster(a)your.dom # If there is no user-specific Sieve-script, global Sieve script is # executed if set. #global_script_path = # UNIX socket path to master authentication server to find users. #auth_socket_path = /var/run/dovecot-auth-master # } ## ## Authentication processes ## # Executable location #auth_executable = /usr/lib/dovecot/dovecot-auth # Set max. process size in megabytes. #auth_process_size = 256 # Authentication cache size in kilobytes. 0 means it's disabled. # Note that bsdauth, PAM and vpopmail require cache_key to be set for caching # to be used. Also note that currently auth cache doesn't work very well if # you're using multiple passdbs with same usernames in them. #auth_cache_size = 0 # Time to live in seconds for cached data. After this many seconds the cached # record is no longer used, *except* if the main database lookup returns # internal failure. #auth_cache_ttl = 3600 # Space separated list of realms for SASL authentication mechanisms that need # them. You can leave it empty if you don't want to support multiple realms. # Many clients simply use the first one listed here, so keep the default realm # first. #auth_realms = # Default realm/domain to use if none was specified. This is used for both # SASL realms and appending @domain to username in plaintext logins. #auth_default_realm = # List of allowed characters in username. If the user-given username contains # a character not listed in here, the login automatically fails. This is just # an extra check to make sure user can't exploit any potential quote escaping # vulnerabilities with SQL/LDAP databases. If you want to allow all characters, # set this value to empty. #auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ # Username character translations before it's looked up from databases. The # value contains series of from -> to characters. For example "#@/@" means # that '#' and '/' characters are translated to '@'. #auth_username_translation = # Username to use for users logging in with ANONYMOUS SASL mechanism #auth_anonymous_username = anonymous # More verbose logging. Useful for figuring out why authentication isn't # working. auth_verbose = no # Even more verbose logging for debugging purposes. Shows for example SQL # queries. auth_debug = no # In case of password mismatches, log the passwords and used scheme so the # problem can be debugged. Requires auth_debug=yes to be set. #auth_debug_passwords = no # Maximum number of dovecot-auth worker processes. They're used to execute # blocking passdb and userdb queries (eg. MySQL and PAM). They're # automatically created and destroyed as needed. #auth_worker_max_count = 30 # Kerberos keytab to use for the GSSAPI mechanism. Will use the system # default (usually /etc/krb5.keytab) if not specified. #auth_krb5_keytab = auth default { # Space separated list of wanted authentication mechanisms: # plain digest-md5 cram-md5 apop anonymous gssapi # mechanisms = plain userdb static { args = uid=1002 gid=1002 home=/home/virtual_mail/%u } # passdb passwd-file { # args = /etc/passwd # } ## ## dovecot-lda specific settings ## # socket listen { # master { # path = /var/run/dovecot-auth-master # mode = 0600 # user = vmail # User running Dovecot LDA # #group = mail # Or alternatively mode 0660 + LDA user in this group # } # } # # Password database is used to verify user's password (and nothing more). # You can have multiple passdbs and userdbs. This is useful if you want to # allow both system users (/etc/passwd) and virtual users to login without # duplicating the system users into virtual database. # # http://wiki.dovecot.org/Authentication # # Users can be temporarily disabled by adding a passdb with deny=yes. # If the user is found from that database, authentication will fail. # The deny passdb should always be specified before others, so it gets # checked first. Here's an example: #passdb passwd-file { # File contains a list of usernames, one per line #args = /etc/dovecot.deny #deny = yes #} # PAM authentication. Preferred nowadays by most systems. # Note that PAM can only be used to verify if user's password is correct, # so it can't be used as userdb. If you don't want to use a separate user # database (passwd usually), you can use static userdb. # passdb pam { # [session=yes] [cache_key=<key>] [<service name>] # # session=yes makes Dovecot open and immediately close PAM session. Some # PAM plugins need this to work, such as pam_mkhomedir. # # cache_key can be used to enable authentication caching for PAM # (auth_cache_size also needs to be set). It isn't enabled by default # because PAM modules can do all kinds of checks besides checking password, # such as checking IP address. Dovecot can't know about these checks # without some help. cache_key is simply a list of variables (see # /usr/share/doc/dovecot-common/variables.txt) which must match for the # cached data to be used. # Here are some examples: # %u - Username must match. Probably sufficient for most uses. # %u%r - Username and remote IP address must match. # %u%s - Username and service (ie. IMAP, POP3) must match. # # If service name is "*", it means the authenticating service name # is used, eg. pop3 or imap. # # Some examples: # args = session=yes * # args = cache_key=%u dovecot #args = dovecot # } # /etc/passwd or similar, using getpwnam() # In many systems nowadays this uses Name Service Switch, which is # configured in /etc/nsswitch.conf. # passdb passwd { # } # /etc/shadow or similiar, using getspnam(). Deprecated by PAM nowadays. #passdb shadow { #} # BSD authentication. Used by at least OpenBSD. #passdb bsdauth { # [cache_key=<key>] - See cache_key in PAM for explanation. #args = #} # passwd-like file with specified location #passdb passwd-file { # Path for passwd-file #args = #} # checkpassword executable authentication # NOTE: You will probably want to use "userdb prefetch" with this. #passdb checkpassword { # Path for checkpassword binary #args = #} # SQL database passdb sql { # Path for SQL configuration file, see /etc/dovecot/dovecot-sql.conf for # example args = /etc/dovecot/dovecot-sql.conf } # LDAP database #passdb ldap { # Path for LDAP configuration file, see /etc/dovecot/dovecot-ldap.conf for # example #args = #} # vpopmail authentication #passdb vpopmail { # [cache_key=<key>] - See cache_key in PAM for explanation. #args = #} # # User database specifies where mails are located and what user/group IDs # own them. For single-UID configuration use "static". # # http://wiki.dovecot.org/Authentication # http://wiki.dovecot.org/VirtualUsers # # /etc/passwd or similar, using getpwnam() # In many systems nowadays this uses Name Service Switch, which is # configured in /etc/nsswitch.conf. userdb passwd { } # passwd-like file with specified location #userdb passwd-file { # Path for passwd-file #args = #} # static settings generated from template #userdb static { # Template for settings. Can return anything a userdb could normally # return, eg.: uid, gid, home, mail, nice # # A few examples: # # args = uid=500 gid=500 home=/var/mail/%u # args = uid=500 gid=500 home=/home/%u mail=mbox:/home/%u/mail nice=10 # #args = #} # SQL database userdb sql { # Path for SQL configuration file, see /etc/dovecot/dovecot-sql.conf for # example args = /etc/dovecot/dovecot-sql.conf } # LDAP database #userdb ldap { # Path for LDAP configuration file, see /etc/dovecot/dovecot-ldap.conf for # example #args = #} # vpopmail #userdb vpopmail { #} # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # This can be made to work with SQL and LDAP databases, see their example # configuration files for more information how to do it. # http://wiki.dovecot.org/AuthSpecials #userdb prefetch { #} # User to use for the process. This user needs access to only user and # password databases, nothing else. Only shadow and pam authentication # requires roots, so use something else if possible. Note that passwd # authentication with BSDs internally accesses shadow files, which also # requires roots. Note that this user is NOT used to access mails. # That user is specified by userdb above. user = root # Directory where to chroot the process. Most authentication backends don't # work if this is set, and there's no point chrooting if auth_user is root. # Note that valid_chroot_dirs isn't needed to use this setting. #chroot = # Number of authentication processes to create #count = 1 # Require a valid SSL client certificate or the authentication fails. #ssl_require_client_cert = no # Take the username from client's SSL certificate, using X509_NAME_oneline() # which typically uses subject's Distinguished Name. #ssl_username_from_cert = no } # It's possible to export the authentication interface to other programs, # for example SMTP server which supports talking to Dovecot. Client socket # handles the actual authentication - you give it a username and password # and it returns OK or failure. So it's pretty safe to allow anyone access to # it. Master socket is used to a) query if given client was successfully # authenticated, b) userdb lookups. # listener sockets will be created by Dovecot's master process using the # settings given inside the auth section #auth default_with_listener { # mechanisms = plain # passdb pam { # } # userdb passwd { # } # socket listen { # master { # path = /var/run/dovecot-auth-master # # WARNING: Giving untrusted users access to master socket may be a # # security risk, don't give too wide permissions to it! # #mode = 0600 # # Default user/group is the one who started dovecot-auth (root) # #user = # #group = # } # client { # path = /var/run/dovecot-auth-client # mode = 0660 # } # } #} # connect sockets are assumed to be already running, Dovecot's master # process only tries to connect to them. They don't need any other settings # than path for the master socket, as the configuration is done elsewhere. # Note that the client sockets must exist in login_dir. #auth external { # socket connect { # master { # path = /var/run/dovecot-auth-master # } # } #} plugin { # Here you can give some extra environment variables to mail processes. # This is mostly meant for passing parameters to plugins. %variable # expansion is done for all values. # Quota plugin #quota = dirsize:%h/mail # Convert plugin. If set, specifies the source storage path which is # converted to destination storage (default_mail_env). #convert_mail = mbox:%h/mail }

2 1

[Dovecot] Migration from tpop3d to dovecot - troubles galore!
by Odhiambo WASHINGTON 29 May '07

29 May '07

Hi people, I am trying to migrate some users from tpop3d (Maildir mailboxes) to dovecot. The most important aspect of this migration, and which I cannot seem to get right, is to first be able to make dovecot to authenticate against the current DB being used by tpop3d so that I can use imapsync to transfer the user mails. And I do believe that is what I am unable to get right!! So here goes what I have tried so far: Take a case of one username, eddie, who belongs to the domain, demo.wananchi.com. 1. This query is what I am using for password_query. I do suspect it is wrong, but well, this is what it yields from the current DB: mysql> SELECT CONCAT(domain.path, '/', popbox.mbox_name), popbox.password_hash FROM popbox, domain WHERE popbox.local_part = 'eddie' AND popbox. domain_name = 'demo.wananchi.com' AND popbox.domain_name = domain.domain_name; +--------------------------------------------+---------------------------------------+ | CONCAT(domain.path, '/', popbox.mbox_name) | password_hash | +--------------------------------------------+---------------------------------------+ | /var/spool/virtual/demo.wananchi.com/eddie | {md5}897fad9cbef974ff95254417234ff1df | +--------------------------------------------+---------------------------------------+ 2. This is what I use for user_query and the result: mysql> SELECT CONCAT(domain.path, '/', popbox.mbox_name) as home, 6 as uid, 6 as gid FROM popbox, domain WHERE popbox.local_part = 'eddie' AND popbox.domain_name = 'demo.wananchi.com' AND popbox.domain_name = domain.domain_name; +-------------------------------------------------+-----+-----+ | home | uid | gid | +-------------------------------------------------+-----+-----+ | /var/spool/virtual/nyerimunicipal.co.ke/eddie | 6 | 6 | +-------------------------------------------------+-----+-----+ 3. Getting a typical user would need this query, for example: mysql> select CONCAT(local_part,'\@',domain_name) as user from popbox limit 10; +-------------------------+ | user | +-------------------------+ | eddie(a)demo.wananchi.com | +-------------------------+ 4. My tables, and the data they contain: (a) mysql> desc popbox; +----------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+--------------+------+-----+---------+-------+ | domain_name | varchar(255) | | PRI | | | | local_part | varchar(255) | | PRI | | | | password_hash | varchar(255) | YES | | NULL | | | mbox_name | varchar(255) | YES | | NULL | | | password_hash2 | varchar(255) | YES | | NULL | | +----------------+--------------+------+-----+---------+-------+ (b) mysql> select * from popbox limit 1; +-------------------+------------+---------------------------------------+-----------+----------------+ | domain_name | local_part | password_hash | mbox_name | password_hash2 | +-------------------+------------+---------------------------------------+-----------+----------------+ | demo.wananchi.com | eddie | {md5}7bb1be8a78498a9de40098a593199819 | eddie | 9IYjWHc4Cs9/w | +-------------------+------------+---------------------------------------+-----------+----------------+ PS: Not all users have password_hash2, which I think is Unix crypt!!! (c) mysql> desc domain; +---------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +---------------+--------------+------+-----+---------+-------+ | domain_name | varchar(255) | | PRI | | | | unix_user | varchar(255) | YES | | NULL | | | password_hash | varchar(255) | YES | | NULL | | | path | varchar(255) | YES | | NULL | | | max_popbox | int(11) | YES | | NULL | | +---------------+--------------+------+-----+---------+-------+ (d) mysql> select * from domain limit 1; +--------------+-----------+---------------------------------------+--------------------------------------+-----------------+ | domain_name | unix_user | password_hash | path | max_popbox | +--------------+-----------+---------------------------------------+--------------------------------------+-----------------+ | dlsnovit.org | exim | {md5}d108219901e577db69c249645ae3a49f | /var/spool/virtual/demo.wananchi.com | 5 | +--------------+-----------+---------------------------------------+--------------------------------------+-----------------+ [root@ns2]#less /var/log/dovecot.log dovecot: May 29 17:42:57 Warning: Killed with signal 15 dovecot: May 29 17:42:59 Info: Dovecot v1.0.0 starting up dovecot: May 29 17:42:59 Info: auth-worker(default): mysql: Connected to localhost (virtualemail) dovecot: May 29 17:44:02 Info: auth(default): client in: AUTH 1 PLAIN service=POP3 secured lip=62.8.64.4 rip=62.8.64.4 resp=AGVk ZGllQGRlbW8ud2FuYW5jaGkuY29tAGJvZWluZzg= dovecot: May 29 17:44:02 Info: auth-worker(default): sql(eddie(a)demo.wananchi.com,62.8.64.4): query: SELECT CONCAT(domain.path, '/', popbox.mbox_n ame), popbox.password_hash FROM popbox, domain WHERE popbox.local_part = 'eddie' AND popbox.domain_name = 'demo.wananchi.com' AND popbox.domain_n ame = domain.domain_name dovecot: May 29 17:44:02 Error: child 40402 (auth-worker) killed with signal 11 dovecot: May 29 17:44:03 Info: auth(default): client out: FAIL 1 user=eddie(a)demo.wananchi.com temp dovecot: May 29 17:44:05 Info: pop3-login: Aborted login: user=<eddie(a)demo.wananchi.com>, method=PLAIN, rip=62.8.64.4, lip=62.8.64.4, secured o That password_hash - is it usable at all with dovecot? o Could password_hash2 be used? I could popolate the table with a known value and use it, just for imapsync to complete!! o What default_pass_scheme should I use in my userdb configuration? I'll very much appreciate help on this. I am currently testing with pop3, but once I get successful authentication, I will switch to imap for imapsync. Thanks for your time, and help, and advise:-) -Wash http://www.netmeister.org/news/learn2quote.html DISCLAIMER: See http://www.wananchi.com/bms/terms.php -- +======================================================================+ |\ _,,,---,,_ | Odhiambo Washington <wash(a)wananchi.com> Zzz /,`.-'`' -. ;-;;,_ | Wananchi Online Ltd. www.wananchi.com |,4- ) )-,_. ,\ ( `'-'| Tel: +254 20 313985-9 +254 20 313922 '---''(_/--' `-'\_) | GSM: +254 722 743223 +254 733 744121 +======================================================================+ One, with God, is always a majority, but many a martyr has been burned at the stake while the votes were being counted. -- Thomas B. Reed

1 0

[Dovecot] About create mailbox by dovecot
by gaoqiang.qian＠nokia.com 29 May '07

29 May '07

Hi, I am a beginner of Dovecot. Actually I have just used yum install dovecot in my red hat linux host server. Could anyone tell me how can I start to use the dovecot to create a mailbox in my domain name. It seems there are lots of documents in the homepage of dovecot, but I do not know which one is the one specifically start to create mailbox. Thanks Qiang

1 0