- dovecot - dovecot.org

Re: [Dovecot] Dovecot+Postfix+Maildir with ActiveDirectory userbase
by Wayne Thursby 16 Feb '10

16 Feb '10

> Wayne Thursby put forth on 2/15/2010 10:24 AM: > > > My Setup: > > Postfix and Dovecot running on the same virtual machine on a Dell 2950 > > with 1x Xeon E5440 on ESXi 4. Maildirs served up by 10x146GB SCSI drives > > on RAID-10 via direct attached Dell MD-1000. > > Interesting setup Wayne. > The same ESXi/RAID combo are also running several other VMs using 7 of its 8GB of RAM. > > Serving 600+ users, mailbox sizes up to 14GiB with constant heavy > > searching (no fts yet) and filtering (sieve and IMAP clients). 80,000 > > messages delivered per week, 95,000 messages rejected, 17,000 spam. 15 > > minute Load average stays under .50 all day with 4 CPU cores. > > How many CPUs are listed in /proc/cpuinfo under that Ubuntu VM? ESX by > default only exposes one CPU(core) to a VM guest OS. Any chance you didn't > tweak the VM config and that your Ubuntu guest is only running on time slices > of a single core? Your results are even more impressive if that's the case. Sorry if it's less impressive, but I've certainly configured the VM for 4 cores. The cores are shared equally between all VMs, I've done no prioritization because I've not had any performance issues related to resource contention. Having 10x15k RPM spindles to seek across, and 512MB of cache on the controller, means I can give Postfix/Amavis/Dovecot 3GB of RAM to use, and 1GB of it is always used for cache, without ever dipping into swap. I get occasional spikes to 30ms of disk latency, but it stays below 5ms for most of the workday. > Also, are you seeing the classic system clock drift of ESX guests, > and if so, is it causing problems with smtp and imap? Not at all, that's what the VMware tools are for. The ESXi host is configured to use our NTP server inside the VPN, and the VMware tools package synchronizes the guest clock with the host. I ran into this on our webmail client as well, then I discovered how easy it really is to install the VMware tools on Ubuntu, so it's no longer a problem. Don't want to stray off topic, so I'll start a new thread, but I'm now attempting to bring High Availability to the mix, and could use some suggestions. -- Wayne Thursby System Administrator Physicians Group, LLC

1 0

[Dovecot] sieve addflag has stopped working
by Andrew Schulman 16 Feb '10

16 Feb '10

Dovecot 1.2.10 in Debian, using Dovecot sieve. I have a sieve script, generated by Ingo, that's supposed to mark spam messages as "Seen" and file them into the Junk folder. In part it reads: require ["regex", "body", "imapflags", "fileinto"]; if exists "X-Spam-Flag" { addflag "\\Seen"; fileinto "Junk"; removeflag "\\Seen"; stop; } This has been working correctly for a long time, but since about 2 weeks ago now, the "Seen" flag isn't being set, although the messages are still being filed into the Junk folder. In two different mail clients, new messages in the Junk folder show up as being unread. Looking at the mail files that hold those messages, the file names don't have an "S" flag appended, while older messages all have the "S" and show up as already seen in the mail clients. I don't believe that I've updated dovecot recently, so I'm not sure why the behavior has changed. Note that I do have plugin { sieve_extensions = +imapflags } in dovecot.conf. I know that imapflags is deprecated, but it is still supposed to be supported. Unfortunately ingo1 doesn't support imap4flags yet (http://bugs.horde.org/ticket/8784) I'm trying to find log information that might indicate an error, but I'm not finding much. /var/log/exim4/mainlog just says: 2010-02-15 14:22:14 1Nh6WK-000200-8z <xxx(a)xxx.xxx>: deliver_pipe transport output: deliver(andrex): Info: sieve: msgid=<ad58db2a42700ec0e5c924c8882d48c2(a)yourmailnews.net>: stored mail into mailbox 'Junk' The wiki says that there should be a log file ~/.dovecot.sieve.log, but there isn't one, which I guess means there are no error messages. Output of dovecot -a is below. Any ideas why the Seen flag isn't being set any more? Thanks, Andrew. # dovecot -a # 1.2.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26 x86_64 Debian squeeze/sid base_dir: /var/run/dovecot log_path: info_log_path: log_timestamp: %Y-%m-%d %H:%M:%S syslog_facility: mail protocols: imap imaps managesieve listen(default): localhost listen(imap): localhost listen(managesieve): localhost:2000 ssl_listen: imap ssl: yes ssl_ca_file: ssl_cert_file: /etc/dovecot/helium-imaps.cert.pem ssl_key_file: /etc/dovecot/helium-imaps.key.pem ssl_key_password: ssl_parameters_regenerate: 0 ssl_cipher_list: ssl_cert_username_field: commonName ssl_verify_client_cert: no disable_plaintext_auth: yes verbose_ssl: no shutdown_clients: yes nfs_check: yes version_ignore: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login login_user: dovecot login_greeting: Dovecot ready. login_log_format_elements: user=<%u> method=%m rip=%r lip=%l %c login_log_format: %$: %s login_process_per_connection: yes login_chroot: yes login_trusted_networks: login_process_size: 64 login_processes_count: 3 login_max_processes_count: 128 login_max_connections: 256 valid_chroot_dirs: mail_chroot: max_mail_processes: 512 mail_max_userip_connections: 10 verbose_proctitle: no first_valid_uid: 500 last_valid_uid: 0 first_valid_gid: 1 last_valid_gid: 0 mail_access_groups: mail_privileged_group: mail mail_uid: mail_gid: mail_location: maildir:~/.mail mail_cache_fields: mail_never_cache_fields: imap.envelope mail_cache_min_mail_count: 0 mailbox_idle_check_interval: 30 mail_debug: no mail_full_filesystem_access: yes mail_max_keyword_length: 50 mail_save_crlf: no mmap_disable: yes dotlock_use_excl: yes fsync_disable: no mail_nfs_storage: no mail_nfs_index: no mailbox_list_index_disable: yes lock_method: fcntl maildir_stat_dirs: no maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: no maildir_very_dirty_syncs: no mbox_read_locks: fcntl mbox_write_locks: fcntl dotlock mbox_lock_timeout: 300 mbox_dotlock_change_timeout: 120 mbox_min_index_size: 0 mbox_dirty_syncs: yes mbox_very_dirty_syncs: no mbox_lazy_writes: yes dbox_rotate_size: 2048 dbox_rotate_min_size: 16 dbox_rotate_days: 1 mail_drop_priv_before_exec: no mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_process_size: 256 mail_plugins: mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve mail_log_prefix: %Us(%u): mail_log_max_lines_per_sec: 10 imap_max_line_length: 65536 imap_capability: imap_client_workarounds: imap_logout_format: bytes=%i/%o imap_id_send: imap_id_log: imap_idle_notify_interval: 120 pop3_no_flag_updates: no pop3_enable_last: no pop3_reuse_xuidl: no pop3_save_uidl: no pop3_lock_session: no pop3_uidl_format: %08Xu%08Xv pop3_client_workarounds: pop3_logout_format: top=%t/%p, retr=%r/%b, del=%d/%m, size=%s dict_db_config: dict_process_count: 1 managesieve_max_line_length: 65536 managesieve_logout_format: bytes=%i/%o managesieve_implementation_string: dovecot lda: postmaster_address: postmaster(a)xxx.xxx mail_plugin_dir: /usr/lib/dovecot/modules/lda mail_plugins: sieve log_path: /dev/stderr log_timestamp: auth default: mechanisms: plain realms: default_realm: cache_size: 0 cache_ttl: 3600 cache_negative_ttl: 3600 executable: /usr/lib/dovecot/dovecot-auth user: root chroot: username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ username_translation: username_format: master_user_separator: anonymous_username: anonymous krb5_keytab: gssapi_hostname: winbind_helper_path: /usr/bin/ntlm_auth failure_delay: 2 verbose: no debug: no debug_passwords: no ssl_require_client_cert: no ssl_username_from_cert: no use_winbind: no count: 1 worker_max_count: 30 process_size: 256 passdb: driver: pam args: deny: no pass: no master: no userdb: driver: passwd args: plugin: sieve_extensions: +imapflags sieve_dir: ~/.mail/sieve

2 2

[Dovecot] Dovecot+Postfix+Maildir with ActiveDirectory userbase
by Wayne Thursby 15 Feb '10

15 Feb '10

> Luigi Rosa <lists(a)luigirosa.com> uttered: > Hi, > has anyone implemented a mail server with maildir, Postfix and Dovecot > using > Active Directory ad userbase and password authentication? > > Do I need Samba to authenticate users? > > Can I use credential caching just like mysql? > > > Ciao, > luigi > > -- > / > +--[Luigi Rosa]-- > \ Hi Luigi, I am running a Postfix/Dovecot setup that is authenticating against a Windows 2000 domain. It is possible to authenticate several different ways, including Kerberos/Winbind and NTLM via Samba. I used the following article as a guide. It is for FreeBSD, but the configuration instructions apply equally to any Postfix/Dovecot setup. I am running mine on Ubuntu 8.10. http://blog.al-shami.net/index.php/freebsd-postfix-dovecot-and-active-direc… The above article covers most of the setup, however I have changed a few things for my purposes. In short, I am using LDAP to communicate with the Active Directory server. I am looking up email addresses and aliases from AD in Postfix and using Dovecot as LDA. Postfix uses Dovecot's SASL for authentication, and Dovecot in turn authenticates against AD. My Setup: Postfix and Dovecot running on the same virtual machine on a Dell 2950 with 1x Xeon E5440 on ESXi 4. Maildirs served up by 10x146GB SCSI drives on RAID-10 via direct attached Dell MD-1000. Serving 600+ users, mailbox sizes up to 14GiB with constant heavy searching (no fts yet) and filtering (sieve and IMAP clients). 80,000 messages delivered per week, 95,000 messages rejected, 17,000 spam. 15 minute Load average stays under .50 all day with 4 CPU cores. IMAP clients include Outlook 2007, Thunderbird 2 and 3, in addition to webmail via Group Office (yuck). dovecot-ldap.conf: # Include multiple domain controllers for redundancy, first one is # at the same facility as the Dovecot server. hosts = 10.0.1.12:389 10.0.2.12:389 10.0.3.12:389 # Specify the full DN of a user to authenticate against dn = cn=Internal, ou=People, dc=example, dc=com dnpass = some_secure_password auth_bind = yes ldap_version = 3 base = ou=People, dc=example, dc=com # Specify the base storage for email here user_attrs = sAMAccountName=home=/var/vmail/example.com/%$ # The following user_filter should all be on a single line. # The ugly userAccountControl param means "Exclude disabled users" user_filter = (&(sAMAccountName=%Ln)(!(userAccountControl:1.2.840.113556.1.4.803:=2))) pass_filter = (&(sAMAccountName=%Ln)(!(userAccountControl:1.2.840.113556.1.4.803:=2))) For the Postfix side of things, the article explains how to set Postfix to use Dovecot's SASL for authentication. It also specifies separate confs for Postfix to lookup users and aliases from. For my purposes, I decided not to modify the AD schema, and to instead utilize an unused field to store aliases in. This field had to be able to store multiple values in a way that I could query easily via LDAP. I chose the "IP Phone" field for this purpose. In the AD GUI it is easily accessible, and allows for multiple values to be stored. Note its usage in the queries that follow. /etc/postfix/ldap-users.cf: # Same list of DCs as in dovecot-ldap.conf server_host = ldap://10.0.1.12:389 ldap://10.0.2.12:389 ldap://10.0.3.12:389 search_base = dc=example, dc=com bind = yes bind_dn = EXAMPLE\internal bind_pw = a_secure_password # Use this lookup for email addresses matching the following domains # corresponds with virtal_mailbox_domains in main.cf domain = example.com, examplelegacy.com, exmple.com # The following query has been modified a bit from the above article. # First, the userAccountControl param specifies the exclusion of # disabled users. # The other change is from objectClass to objectCategory. query_filter = (&(&(objectCategory=person)(sAMAccountName=%u))(!(userAccountControl:1.2.840.113556.1.4.803:=2))) result_attribute = sAMAccountName version = 3 # I was having trouble with referrals not resolving properly due to a # misconfigured domain controller. I turned this option off and have # not had a problem since. chase_referrals = no result_format=example.com/%s/ debuglevel = 0 timeout = 30 /etc/postfix/ldap-aliases.cf: # Same list of DCs as in dovecot-ldap.conf server_host = ldap://10.0.1.12:389 ldap://10.0.2.12:389 ldap://10.0.3.12:389 search_base = dc=example, dc=com bind = yes bind_dn = EXAMPLE\internal bind_pw = a_secure_password domain = example.com, examplelegacy.com, exmple.com # This query looks up aliases from the otherIPPhone field. query_filter = (&(&(objectCategory=person)(|(otherIPPhone=%u)(sAMAccountName=%u)))(!(userAccountControl:1.2.840.113556.1.4.803:=2))) result_attribute = sAMAccountName version = 3 chase_referrals = no result_format=%s(a)example.com debuglevel = 0 timeout = 30 The objectCategory replacement for objectClass is a *huge* performance increase. objectClass is not indexed in Active Directory, forcing a lot of CPU cycles to be wasted for every lookup. The way I have implemented things, this means two of these expensive lookups per incoming email (even if it was undeliverable!). At first this was using around 50% CPU on a dual Xeon domain controller. This was acceptable at the time, but after my optimizations we were able to move authentication to a virtual machine with 640MB of RAM and a single CPU core where it now stays below 20% all day. Changing the lookup to an indexed field probably cut the initial load by 90%. Adding auth_cache settings to dovecot.conf then further reduced load by more than 50%. Well, I suppose that screed should suffice for my first post. I've gained much from just lurking in this list, just trying to contribute where I can. -- Wayne Thursby System Administrator Physicians Group, LLC

2 1

Re: [Dovecot] [RCU] Dovecot managesieve
by Frank Bonnet 15 Feb '10

15 Feb '10

Well I've looked forward into the wiki and did not found my server have only system users ( no virtual domain ) and the wiki says : System users If you wish you use deliver for all system users on a single domain mail host you can do it by editing mailbox_command parameter in /etc/postfix/main.cf (postconf(5)): That's what I did but then I have this problem with DOTLOCK If I chmod 777 /var/mail directory it works but I do not feel comfortable with that kind of access rights ... On 02/12/10 20:39, Joseph Kolb wrote: > Hi Frank, > > the question is, do you use the LDA of dovecot (LocalDeliveryAgent)? > If so, then yes, you must add/modify some stuff in master.cf. This is good > described in dovecot wiki :-). > > Regards, > Joseph > > Am Freitag 12 Februar 2010 15:42:12 schrieben Sie: >> Hi Joseph >> >> Thanks for your answer >> >> did you modify something in Postfix main.cf file ? >> >> >> Frank >> >> On 02/12/10 11:57, Joseph Kolb wrote: >>> On Fri, 12 Feb 2010 10:13:53 +0100, "A.L.E.C"<alec(a)alec.pl> wrote: >>>> Frank Bonnet wrote: >>>>>> protocol lda { >>>>>> >>>>>> # Address to use when sending rejection mails. >>>>>> postmaster_address = postmaster(a)esiee.fr >>>> >>>> In dovecot-1.1 I've got in "protocol lda" section: >>>> >>>> mail_plugins = cmusieve >>> >>> Hi Frank, >>> >>> i use also dovecot 1.2.10 >>> >>> and here is my part of dovecot.conf >>> >>> -->snip<--- >>> protocol imap { >>> >>> ssl_listen = *:993 >>> mail_plugins = quota imap_quota >>> >>> } >>> >>> protocol lda { >>> >>> postmaster_address = postmaster(a)xxxx.xx >>> log_path = /var/log/dovecot-lda.log >>> info_log_path = /var/log/dovecot-info-lda.log >>> mail_plugins = sieve >>> >>> } >>> -->snip<--- >>> >>> Regards, >>> Joseph >>> _______________________________________________ >>> List info: http://lists.roundcube.net/users/ >

2 3

[Dovecot] Capability COMPRESS implemented?
by Ed W 15 Feb '10

15 Feb '10

Hi, I just read that http://tools.ietf.org/html/rfc4978 ..is being implemented in Thunderbird 3. This is great news! It's been a long time since we last discussed this and I don't think it's currently implemented in Dovecot, but since Cyrus just added it, any chance of getting support in Dovecot please? (Fastmail seems to be supporting this effort) On a similar note we had a previous thread about supporting TLS deflate - any chance you could investigate this again and add support on the Dovecot side so at least it would be feasible to submit a bug fix on the Thunderbird side? (It looked like a one liner change back when we last discussed it?) Thanks Ed W (Obviously for those looking for an external high performance compressor you can use our compressing email proxy from http://www.mailasail.com ...)

3 14

[Dovecot] So, what about clustering and load balancing?
by Stefan Foerster 15 Feb '10

15 Feb '10

Hello world, so, with beta2 of Dovecot 2.0 being available, what is the preferred way to achieve load balancing and fault tolerance? As far as I can see it, there are basically two options: 1) Use a HA shared storage, export either a cluster filesystem or NFS, and have your dovecot servers mount that file system. Load balance these servers (Cisco ACE, ldirectord, ...) and there you go. Overall performance is limited by the speed of the shared storage and may further decrease due to locking issues, imperformant cluster filesystems and so on. 2) Set up every dovecot server with local storage and trigger a dsync after each change (pyinotfiy, incron, parse LMTP delivery logs, ...). Load balance these servers as before. Depending on the number of syncs that have to be done, mailbox replication may lag behind. The need to constantly spawn dsync processes may further decrease performance if you don't do it right (stickiness of established connections, replicating at most every xyz seconds). What are your opinions on that matter? Stefan P.S: I've set up option two in a test setup, though the incron/inotify part is still giving me a headache.

6 12

[Dovecot] POP3 with dovecot LDA and sieve
by Arthur Titeica 14 Feb '10

14 Feb '10

Hello. I'm using Postfix with Dovecot as IMAP, POP3 and LDA. With sieve rules I successfully move mails to specific mailboxes and this is great if accessing the account thru IMAP. However, when accessing thru POP3 I only get mails from Inbox. I wonder if it's possible to get real new mails, regardless of the folder they're in. Thanks for any thoughts. Arthur

3 3

[Dovecot] comprehensive info on search capabilities?
by Kārlis Repsons 14 Feb '10

14 Feb '10

People, where to read about how to make the most of the current Dovecot search capabilities, including squat and possibly also the related future plans? There are many aspects like IMAP protocol itself, extensions, current support into Dovecot, support among MUAs... Could you post some good links, please?

2 3

[Dovecot] Virtual plugin and user_query
by Arthur Titeica 14 Feb '10

14 Feb '10

Hello, As per a previous reply regarding POP3 I'm now trying to configure the virtual plugin. However I don't really understand how to modify the sql user_query to properly indicate inbox=yes. I created 3 namespaces as per wiki instructions and I've modified the password and user queries as bellow: password_query = \ SELECT email, password, login , concat('*:bytes=', quota) \ AS userdb_quota_rule, \ CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' \ END AS namespace_1_inbox, \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL \ END AS namespace_2_inbox \ FROM users \ WHERE email = '%u' AND login = 'active' user_query = SELECT \ concat('*:bytes=', quota) AS quota_rule, \ CASE '%s' WHEN 'pop3' THEN NULL ELSE 'yes' \ END AS namespace_1_inbox, \ CASE '%s' WHEN 'pop3' THEN 'yes' ELSE NULL \ END AS namespace_2_inbox \ FROM users \ WHERE email = '%u' After that all I get in the logs is: IMAP(arthur(a)psw.ro): Error: namespace configuration error: inbox=yes namespace missing IMAP(arthur(a)psw.ro): Fatal: Namespace initialization failed As I don't really understand those case sql queries is there something I should add in the mysql tables? Any thoughts will be greatly appreciated. Thanks!

2 1

[Dovecot] deliver and cloning file permissions
by Alexander 'Leo' Bergolth 14 Feb '10

14 Feb '10

Hi! I'd like to use deliver (dovecot-1.2.10) as LDA to deliver mails to my local users on my postfix-system. However, deliver tries to copy the filesystem permissions of the mail-spool-files to the user's index files, which doesn't work, since the files in /var/mail/ belong to the group 'mail'. Feb 13 00:21:19 leo-x61 dovecot: deliver(leo): fchown(/home/leo/mail/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=100(users), group based on /var/mail/leo) I know that dovecot tries to clone the mailbox permissions in order to support shared folders but it would be nice if there was an option to disable this (or to just try it and ignore the fchown error). I could manually change group ownership of all mailboxes from mail to each user's primary group or set the mailbox-modes to 0600 but I'd also have to do that for every new user. (Tools to create users like useradd from shadow-utils use hardcoded file modes and group ownership.) Is there any workaround? Thanks, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria

3 5