- dovecot - dovecot.org

[Dovecot] scalability and arhive ideas
by fernando＠dfcom.com.br 25 Aug '09

25 Aug '09

Hi, I´m reading the past topics related to archive and scalability of dovecot, they are all very interesting. Here, I´m using two dovecot proxies in front of five storages pairs, and we split the domain´s accounts among those servers. So, we can share the i/o load and if one server goes down only few accounts of the domain stops (not all of them). But, we began to have space problems - and the solution would be insert more and more storage servers. So I was searching for some archive solutions (hard links - at S.O level, or some dovecot extension). A friend told me that he knows an ISP that share even the mailbox of the users among many servers - this is very weird and (at same time) very interesting approach. Instead of put all messages into one maildir and this maildir into one server, this "maildir" (?) is spplited among many servers - so, if one servers fails the account is still acessible and they move old/big messages to a new "cheap" storage - archiving transparently. Well, maybe it´s an stupid idea, but couldn´t dovecot imap/pop proxy do the same ? I mean, imagine the following scenario: 1- Proxy does an user account sql - today the return is (among other data), the final server IP, but it could be the storage servers for this account 2- Proxy does parallel connections (instead of today´s one connection with the final server). Retrieve the messages and caches locally where the message is. If the cache is lost, no problem, it just connects again and re-cache them. 3- When a message is deleted/flagged, etc - it has the cache (allowing it to know where send the command to), when a message inserted (as sent folder) - the system can have the 'default storage' where the messages are delivered and saved to - the other storages are just for archiving (or even a round-robin delivery !? - hard to control). Problems: 1- Have the cache at proxy level. 2- Quota calculation (how the delivery process can check this ?) 3- Maybe mixing proxy connection and this new approach is not easy way. 4- the more servers, the more time waiting their answers (imagine one of them answering slowly). Advantages: 1- No local mounts at storage level - (as nfs and other network mounted partitions). 2- Servers independency - if space is at a critical value, insert a pair of servers, move messages there, insert the servers in the sql answer (server1, server2, server3, ... , serverN). 3- Scalability - the more servers, the more i/o load you can share. Well, sorry for the long post...but I hope to collaborate with some ideas (even if they are stupid :( ) Regards, Fernando Bertasso Figaro

3 8

[Dovecot] dovecot-auth failed: Not enough space
by S. A. Woltering 25 Aug '09

25 Aug '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear all, I've been trying to get a Dovecot IMAP service running under HP-UX 11.23, IA64. The system has 6GB of spare space under /var and at least the same amount of RAM free at the current time. I've compiled from the latest source (1.2.4), using the following options: # CC="/opt/aCC/bin/aCC" ./configure --prefix=/opt/dovecot ... usual make && make install My config file follows at the end. In essence, every time I attempt to start dovecot (/opt/dovecot/sbin/dovecot, with no options) I get the following on the command line: Warning: Last died with error (see error log for more information): Auth process died too early - shutting down Info: If you have trouble with authentication failures, enable auth_debug setting. See http://wiki.dovecot.org/WhyDoesItNotWork Looking into /var/adm/syslog/mail.log, the lines associated with this run read: Aug 24 14:46:25 bilbo dovecot: Dovecot v1.2.4 starting up Aug 24 14:46:25 bilbo dovecot: Fatal: execv(/opt/dovecot/libexec/dovecot/dovecot-auth) failed: Not enough space Aug 24 14:46:25 bilbo dovecot: Fatal: Auth process died too early - shutting down I've switched over to using a static passwd file type auth and I still get the same error. Hoping someone has some idea why this is happening... Thanks, Ashley ## Dovecot configuration file ## base_dir = /var/run/dovecot protocols = imap listen = * disable_plaintext_auth = no shutdown_clients = yes ## ## Logging ## log_timestamp = "%b %d %H:%M:%S " syslog_facility = mail ## ## SSL settings ## #ssl_listen = ssl = no ## ## Login processes ## login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ## ## Mailbox locations and namespaces ## mail_location = maildir:/mail%h/Maildir ## ## IMAP specific settings ## protocol imap { login_executable = /opt/dovecot/libexec/dovecot/imap-login mail_executable = /opt/dovecot/libexec/dovecot/imap #imap_max_line_length = 65536 mail_max_userip_connections = 10 #mail_plugins = mail_plugin_dir = /opt/dovecot/lib/dovecot/imap imap_logout_format = bytes=%i/%o #imap_capability = #imap_idle_notify_interval = 120 #imap_id_send = #imap_id_log = #imap_client_workarounds = } ## ## Authentication processes ## auth_executable = /opt/dovecot/libexec/dovecot/dovecot-auth auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ auth default { passdb passwd-file { args = /opt/dovecot/etc/imap-passwd.txt } } - -- Steven-Ashley Woltering, Systems Analyst, ECMWF, Shinfield Park, Reading, RG2 9AX, UK Tel.: +44 1189 499473 (internal x2473) Email: Steven.Woltering(a)ecmwf.int -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4-svn0 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKkpsnh854NVK99FMRAtVZAJ9nOt/HP1ODSTSMkLPNsPOfRK/EsACghPQ7 aFNOxf4hAKV7NKkiZ/4Z7X8= =5Ww9 -----END PGP SIGNATURE-----

2 4

[Dovecot] LDAP: Connection lost to LDAP server, reconnecting
by McShel 25 Aug '09

25 Aug '09

Hi! Sorry by my english. Set Postfix + dovecot + AD + ldap, everything works well, but appears in the logs, every 15 minutes: auth(default): Error: LDAP: Connection lost to LDAP server, reconnecting Standart limit ldap connect to AD 15 minutes. Can somehow fix this problem without increasing the time of connection to the AD?

3 3

Re: [Dovecot] Dovecot and LDAP-Quotas
by south africa 25 Aug '09

25 Aug '09

> Do you have any other userdbs now? What does dovecot-ldap-userdb.conf > now contain? Is it only quota that's now broken or the logins > completely? The dovecot-ldap-userdb.conf is only a symbolic link on the original dovecot-ldap.conf. Only the ldap-quota is broken. The normal ldap-authentification goes like clockwork. >Set auth_debug=yes and log in, what does it say? "login" ------- Aug 25 11:51:18 auth(default): Info: client in: AUTH 1 PLAIN service=pop3 secured lip=123.456.789.72 rip=192.168.1.2 lport=995 rport=1984 Aug 25 11:51:18 auth(default): Info: client out: CONT 1 Aug 25 11:51:18 auth(default): Info: client in: CONT 1 Aug 25 11:51:18 auth(default): Info: client out: OK 1 user=testuser Aug 25 11:51:18 auth(default): Info: master in: REQUEST 1 21237 1 Aug 25 11:51:18 auth(default): Info: master out: USER 1 testuser uid=500 gid=500 home=/home/vmail/testuser Aug 25 11:51:18 pop3-login: Info: Login: user=<testuser>, method=PLAIN, rip=192.168.1.2, lip=123.456.789.72, TLS Aug 25 11:51:18 POP3(testuser): Info: Loading modules from directory: /usr/lib64/dovecot/pop3 Aug 25 11:51:18 POP3(testuser): Info: Module loaded: /usr/lib64/dovecot/pop3/lib10_quota_plugin.so Aug 25 11:51:18 POP3(testuser): Info: Effective uid=500, gid=500, home=/home/vmail/testuser Aug 25 11:51:18 POP3(testuser): Info: Quota root: name=User quota backend=maildir args= Aug 25 11:51:18 POP3(testuser): Info: Quota rule: root=User quota mailbox=* bytes=8388608 messages=0 Aug 25 11:51:18 POP3(testuser): Info: Quota warning: bytes=7549747 (90%) messages=0 command=/usr/local/bin/quota-warning.sh 90 Aug 25 11:51:18 POP3(testuser): Info: Quota warning: bytes=6291456 (75%) messages=0 command=/usr/local/bin/quota-warning.sh 75 Aug 25 11:51:18 POP3(testuser): Info: maildir: data=/home/vmail/testuser/Maildir/ Aug 25 11:51:18 POP3(testuser): Info: maildir++: root=/home/vmail/testuser/Maildir, index=, control=, inbox=/home/vmail/testuser/Maildir Aug 25 11:51:18 POP3(testuser): Info: Namespace : Using permissions from /home/vmail/testuser/Maildir: mode=0700 gid=-1 Aug 25 11:51:18 POP3(testuser): Info: Disconnected: Logged out top=0/0, retr=1/2608, del=1/1, size=2591 Aug 25 11:51:18 auth(default): Info: new auth connection: pid=21247 "email delivery" ---------------- Aug 25 11:53:02 auth(default): Info: new auth connection: pid=21248 Aug 25 11:53:02deliver(testuser@core.uk): Info: Loading modules from directory: /usr/lib64/dovecot/lda Aug 25 11:53:02deliver(testuser@core.uk): Info: Module loaded: /usr/lib64/dovecot/lda/lib10_quota_plugin.so Aug 25 11:53:02 auth(default): Info: master in: USER 1 testuser(a)core.uk service=deliver Aug 25 11:53:02 auth(default): Info: master out: USER 1 testuser uid=500 gid=500 home=/home/vmail/testuser Aug 25 11:53:02deliver(testuser@core.uk): Info: auth input: uid=500 Aug 25 11:53:02deliver(testuser@core.uk): Info: auth input: gid=500 Aug 25 11:53:02deliver(testuser@core.uk): Info: auth input: home=/home/vmail/testuser Aug 25 11:53:02deliver(testuser@core.uk): Info: userdb changed username to testuser Aug 25 11:53:02deliver(testuser): Info: Quota root: name=User quota backend=maildir args= Aug 25 11:53:02deliver(testuser): Info: Quota rule: root=User quota mailbox=* bytes=8388608 messages=0 Aug 25 11:53:02deliver(testuser): Info: Quota warning: bytes=7549747 (90%) messages=0 command=/usr/local/bin/quota-warning.sh 90 Aug 25 11:53:02deliver(testuser): Info: Quota warning: bytes=6291456 (75%) messages=0 command=/usr/local/bin/quota-warning.sh 75 Aug 25 11:53:02deliver(testuser): Info: maildir: data=/home/vmail/testuser/Maildir/ Aug 25 11:53:02deliver(testuser): Info: maildir++: root=/home/vmail/testuser/Maildir, index=, control=, inbox=/home/vmail/testuser/Maildir Aug 25 11:53:02deliver(testuser): Info: Namespace : Using permissions from /home/vmail/testuser/Maildir: mode=0700 gid=-1 Aug 25 11:53:02deliver(testuser): Info: msgid=<E74BF4D485115048AF3A7D77FA7A75610B00F594(a)exc1.three.core.uk>: saved mail to INBOX _____________________________________________________________ Get your FREE TheDoghouseMail email address at http://www.thedoghousemail.com

2 1

[Dovecot] log for each user
by Thomas G. Lau 25 Aug '09

25 Aug '09

Dear all, does anyone did any scripting to log every user activity such as remove message? we got some user keep blaming us about message lost on email server, wondering if any of you did anything about this? Thanks.

4 5

[Dovecot] rawlog not work
by Thomas G. Lau 25 Aug '09

25 Aug '09

Dear all, I am trying to setup dovecot on centos 5.3, dovecot.rawlog is created in user home dir and change to user as owner. But when I try to telnet into pop3 and use list command, nothing is logged and dovecot.rawlog remain empty. should I run rawlog even on login_executable ? Please advice. dovecot --version 1.2.4 # 1.2.4: /etc/dovecot.conf # OS: Linux 2.6.18-128.4.1.el5 x86_64 CentOS release 5.3 (Final) protocols: pop3 login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/pop3-login login_processes_count: 1 mail_debug: yes mail_executable: /usr/libexec/dovecot/rawlog /usr/libexec/dovecot/pop3 mail_plugin_dir: /usr/lib64/dovecot/pop3 lda: postmaster_address: postmaster(a)example.com auth default: passdb: driver: pam userdb: driver: passwd

1 1

[Dovecot] Aborted: Worker is buggy
by David Jonas 25 Aug '09

25 Aug '09

I upgraded from 1.1 to 1.2 a couple months ago and now we are seeing this: Aug 20 05:34:38 kelly dovecot: auth(default): auth workers: Auth request was queued for 7 seconds, 10 left in queue ... Aug 20 05:34:38 kelly dovecot: auth(default): BUG: Worker sent reply with id 53, expected 54 Aug 20 05:34:38 kelly dovecot: auth(default): worker-server(xxx(a)example.com,dd.dd.dd.dd): Aborted: Worker is buggy Aug 20 05:34:38 kelly dovecot: auth(default): passdb(xxx(a)example.com,dd.dd.dd.dd): Fallbacking to expired data from cache When I attempt to restart dovecot there is one dovecot-auth process that won't die and keeps the server from starting. A simple 'killall dovecot-auth' takes care of it and the server is able to start. I'm not using anything exotic, dovecot config below. The most curious fact is that we have identical servers running the same version of dovecot (1.2.3, happened under 1.2.1 as well) talking to the same MySQL database and both develop the problem at the same time. The database is under considerable stress at the time this occurs as the policyd tables are being cleaned. But this happens twice per hour and only sometimes does dovecot have trouble. I'm working to move the policyd database elsewhere, but of course this shouldn't happen in any case. Any help would be much appreciated. Thanks! David # 1.2.3: /etc/dovecot/dovecot.conf # OS: Linux 2.6.9-42.0.10.ELsmp i686 Red Hat Enterprise Linux ES release 4 (Nahant Update 7) protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/dovecot/dovecot.pem ssl_key_file: /etc/dovecot/dovecot.pem ssl_cipher_list: ALL:!LOW:!SSLv2 disable_plaintext_auth: no login_dir: /usr/local/dovecot-1.2/var/run/dovecot/login login_executable(default): /usr/local/dovecot-1.2/libexec/dovecot/imap-login login_executable(imap): /usr/local/dovecot-1.2/libexec/dovecot/imap-login login_executable(pop3): /usr/local/dovecot-1.2/libexec/dovecot/pop3-login login_process_per_connection: no verbose_proctitle: yes first_valid_uid: 89 maildir_stat_dirs: yes mail_executable(default): /usr/local/dovecot-1.2/libexec/dovecot/imap mail_executable(imap): /usr/local/dovecot-1.2/libexec/dovecot/imap mail_executable(pop3): /usr/local/dovecot-1.2/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/dovecot-1.2/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/dovecot-1.2/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/dovecot-1.2/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %f pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: postmaster_address: postmaster(a)no-ip.com hostname: no-ip.com mail_plugins: quota auth_socket_path: /var/run/dovecot/auth-master auth default: mechanisms: login plain cram-md5 cache_size: 5000 cache_ttl: 1800 cache_negative_ttl: 60 user: nobody username_chars: abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ username_translation: %@ username_format: %TLu verbose: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: dict:user:%LTd-%LTn:proxy::quota quote_rule: *:storage=100G dict: quota: mysql:/etc/dovecot/dovecot-dict-sql.conf ## dovecot-sql.conf driver = mysql connect = host=192.168.dd.dd dbname=xxx user=xxx password=xxx user_query=... password_query=... ## dovecot-dict-sql.conf connect = host=192.168.dd.dd dbname=xxx user=xxx password=xxx map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages }

2 2

[Dovecot] Per user namespace
by Ashley M. Kirchner 25 Aug '09

25 Aug '09

I'm in the process of upgrading an old server which is running uw-imap to a new one running Dovecot. With the old machine, unfortunately, my users had the opportunity to store their e-mails in various locations. For example, some have their mail boxes stored in ~/mail/[various mbox files], others ALSO have a ~/mail/clients/[various mbox files] and yet others simply stored them in their ~/ path (thanks to the old IMAP). Is there a way to create per-user namespaces? I don't want to create a "mail/", "mail/clients/", etc., etc. global name space that will end up showing up on everyone's mail client, nor do I want to individually change each user's setup (and rewrite their .subscriptions file). Any solutions here, or am I SOL? AMK4 -- W | It's not a bug - it's an undocumented feature. +-------------------------------------------------------------------- Ashley M. Kirchner <mailto:ashley@pcraft.com> . 303.442.6410 x130 IT Director / SysAdmin / Websmith . 800.441.3873 x130 Photo Craft Imaging . 2901 55th Street http://www.pcraft.com ..... . . . Boulder, CO 80301, U.S.A.

3 3

[Dovecot] xtreemfs
by Robert Schetterer 24 Aug '09

24 Aug '09

Hi @ll, has anybody tried this with dovecot yet ? http://www.xtreemfs.org/ -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

3 5

[Dovecot] quota bug
by Amr Shahin 24 Aug '09

24 Aug '09

Hello i use dovecot for our mail server, it's installed on about 8 servers and it's doing really good, thanks for the great effort :-) however, i have been facing a quota problem recently, dovecot says that some users are using for example (700MB) while they are actually using (10MB), i solve this by removing *maildirsize *which solves it, is there any fix for this bug ? thanks and regards Amr Shahin

3 2