From nam_10 at hotmail.com Tue Nov 1 00:20:56 2011 From: nam_10 at hotmail.com (nuno marques) Date: Mon, 31 Oct 2011 22:20:56 +0000 Subject: [Dovecot] Imap/pop gateway Message-ID: Hello, How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. regards From tss at iki.fi Tue Nov 1 08:24:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Nov 2011 08:24:33 +0200 Subject: [Dovecot] dsync should sync sieve-dirs to! In-Reply-To: <201110290141.41963.p.heinlein@heinlein-support.de> References: <201110290141.41963.p.heinlein@heinlein-support.de> Message-ID: <6589FF32-A0F6-4E75-A6A3-65356164E563@iki.fi> On 29.10.2011, at 2.41, Peer Heinlein wrote: > Having dsync to make backups from existing mail-spaces, it would be nice > to make dsync syncing the sieve-dirs too. -Otherweise backups aren't > complete... dsync was originally meant to be a way to do two-way syncing of mailboxes without losing any changes. That's not really possible to do with Sieve scripts. I guess the best that could be done is to copy the one with the latest timestamp. But if you have multiple Sieve scripts even that doesn't work well, since dsync can't know if a script was added on one side or deleted from another side. Although if scripts were only managed via ManageSieve, the last problem could be solved with some log file. For "dsync backup" I guess it could be useful to have a setting with a list of files/dirs that dsync would always upload to remote.. From tss at iki.fi Tue Nov 1 08:32:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Nov 2011 08:32:47 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: On 30.10.2011, at 13.53, Lukas Wei? wrote: > ive just switched to dovecot imap because dbmail > i used before is not developed any further Oh? It doesn't yet seem dead to me. > and sql storage is difficult > to backup. Ive done that, by setting up dovecot with maildir format and > copy mails from old dbmail to new dovecot imap server with my mail > client (thunderbird). Thunderbird probably changes the INTERNALDATE of the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. From laxlaw at gmx.de Tue Nov 1 10:38:02 2011 From: laxlaw at gmx.de (=?UTF-8?Q?Lukas_Wei=C3=9F?=) Date: Tue, 01 Nov 2011 09:38:02 +0100 Subject: [Dovecot] =?utf-8?q?Bad_creation_timestamp_of_migrated_mails_=28m?= =?utf-8?q?aildir_format=29=3F?= In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: On Tue, 1 Nov 2011 08:32:47 +0200, Timo Sirainen wrote: > On 30.10.2011, at 13.53, Lukas Wei? wrote: > >> ive just switched to dovecot imap because dbmail i used before is not developed any further > Oh? It doesn't yet seem dead to me. > >> and sql storage is difficult to backup. Ive done that, by setting up dovecot with maildir format and copy mails from old dbmail to new dovecot imap server with my mail client (thunderbird). > Thunderbird probably changes the INTERNALDATE of the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. What is INTERNALDATE? The modification timestamp is correct ("ls -l"), and the "Date: ... " line, too. @dbmail: Dead is the wrong designation. But since the last 5 years i only noticed bugfixes, the most "ideas for developement" (STARTTLS for example) mentioned here [1] are still ideas. Links: ------ [1] http://www.dbmail.org/dokuwiki/doku.php/2.3 From taeuber at bbaw.de Tue Nov 1 13:21:49 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Tue, 1 Nov 2011 12:21:49 +0100 Subject: [Dovecot] antispam for dovecot 2.0 Message-ID: <20111101122149.b8028453.taeuber@bbaw.de> Hi! Is there anybody out there who has ported the antispam-storage-1.x.c to antispam-storage-2.0.c? Or is there an alternative to the antispam plugin? Thanks Lars From duihi77 at gmail.com Tue Nov 1 14:34:30 2011 From: duihi77 at gmail.com (Duane Hill) Date: Tue, 1 Nov 2011 07:34:30 -0500 (CDT) Subject: [Dovecot] antispam for dovecot 2.0 In-Reply-To: <20111101122149.b8028453.taeuber@bbaw.de> References: <20111101122149.b8028453.taeuber@bbaw.de> Message-ID: On Tue, 1 Nov 2011, Lars T?uber wrote: > Hi! > > Is there anybody out there who has ported the antispam-storage-1.x.c to > antispam-storage-2.0.c? > > Or is there an alternative to the antispam plugin? Don't know anything about antispam-storage. From searching I assume you mean the antispam plugin for Dovecot 2.x. http://wiki2.dovecot.org/Plugins/Antispam shows where you can get it from and compile. From taeuber at bbaw.de Tue Nov 1 15:58:55 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Tue, 1 Nov 2011 14:58:55 +0100 Subject: [Dovecot] antispam for dovecot 2.0 In-Reply-To: References: <20111101122149.b8028453.taeuber@bbaw.de> Message-ID: <20111101145855.d14e671c.taeuber@bbaw.de> Am Tue, 1 Nov 2011 07:34:30 -0500 (CDT) Duane Hill schrieb: > On Tue, 1 Nov 2011, Lars T?uber wrote: > > > Hi! > > > > Is there anybody out there who has ported the antispam-storage-1.x.c to > > antispam-storage-2.0.c? > > > > Or is there an alternative to the antispam plugin? > > Don't know anything about antispam-storage. From searching I assume you > mean the antispam plugin for Dovecot 2.x. > > http://wiki2.dovecot.org/Plugins/Antispam shows where you can get it from > and compile. Hallo Duane, thank you very much. This is what I was searching for. Thanks Lars From dvstillerfan74 at gmail.com Tue Nov 1 17:55:36 2011 From: dvstillerfan74 at gmail.com (David Varela) Date: Tue, 1 Nov 2011 09:55:36 -0600 Subject: [Dovecot] LDAP w/SASL "Active Directory" authentication failing. Message-ID: Hello, I am running a Dovecot server (version 1.2.17) on FreeBSD 8.2, using LDAP to authenticate Active Directory users. I can successfully bind and authenticate using PLAIN and LDAP without SASL, but obviously passwords for the bind user and the user being authenticated are being passed in plain text. I've attempted to configure my server to us SASL however when I attempt to authenticate a user I see authentication failures. I reviewed the security log on my domain controller and see that the bind user is binding properly, so the issue appears to be orginating from the user authentication, however I cannot determine what the issue is. Here is all the information regarding my configuration, along with the logs from the server: Dovecot -n # 1.2.17: /usr/local/etc/dovecot.conf # OS: FreeBSD 8.2-RELEASE i386 ufs log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot_info.log protocols: imap pop3 imaps pop3s ssl_cert_file: /etc/ssl/dovecot/cert.pem ssl_key_file: /etc/ssl//dovecot/key.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login valid_chroot_dirs: /usr/data/vmail verbose_proctitle: yes first_valid_uid: 1001 last_valid_uid: 1001 first_valid_gid: 1001 last_valid_gid: 1001 mail_privileged_group: mail mail_location: maildir:/usr/data/vmail/%u mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain gssapi ntlm login username_format: %Ln winbind_helper_path: /usr/local/bin/ntlm_auth verbose: yes debug: yes debug_passwords: yes use_winbind: yes passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf userdb: driver: static args: uid=vmail gid=vmail home=/usr/data/vmail/%u socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 dovecot-ldap.conf hosts = 192.168.0.240:3268 dn = cn=PostfixBind,ou=IT,ou=Central Office,ou=LMC,dc=smallmountain,dc=Local dnpass = y0urm0mma sasl_bind = yes #sasl_mech = GSSAPI #sasl_realm = smallmountain.local #sasl_authz_id = debug_level = -1 #auth_bind = yes ldap_version = 3 base = dc=smallmountain,dc=Local deref = never scope = subtree user_filter = (&(objectClass=person)(mail=%u)) pass_attrs = mail=user pass_filter = (&(objectClass=person)(sAMAccountName=%n)) dovecot.log Nov 01 09:09:48 dovecot: Warning: Killed with signal 15 (by pid=99758 uid=0 code=kill) Nov 01 09:09:48 auth(default): Error: ldap_pvt_sasl_getmech Nov 01 09:09:48 auth(default): Error: ldap_search Nov 01 09:09:48 auth(default): Error: put_filter: "(objectclass=*)" Nov 01 09:09:48 auth(default): Error: put_filter: simple Nov 01 09:09:48 auth(default): Error: put_simple_filter: "objectclass=*" Nov 01 09:09:48 auth(default): Error: ldap_build_search_req ATTRS: supportedSASLMechanisms Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_new_connection 1 1 0 Nov 01 09:09:48 auth(default): Error: ldap_int_open_connection Nov 01 09:09:48 auth(default): Error: ldap_connect_to_host: TCP 192.168.0.240:3268 Nov 01 09:09:48 auth(default): Error: ldap_new_socket: 11 Nov 01 09:09:48 auth(default): Error: ldap_prepare_socket: 11 Nov 01 09:09:48 auth(default): Error: ldap_connect_to_host: Trying 192.168.0.240:3268 Nov 01 09:09:48 auth(default): Error: ldap_pvt_connect: fd: 11 tm: -1 async: 0 Nov 01 09:09:48 auth(default): Error: ldap_open_defconn: successful Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 1 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 1, origid 1, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 message type search-entry Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 1, origid 1, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: * msgid 1, type 100 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 message type search-result Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 1, msgid 1) Nov 01 09:09:48 auth(default): Error: adding response ld 0x18611700 msgid 1 type 101: Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: ldap_get_values Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_sasl_interactive_bind: server supports: GSSAPI GSS-SPNEGO EXTERNAL DIGEST-MD5 Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: GSSAPI GSS-SPNEGO EXTERNAL DIGEST-MD5 Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_open: host=adminserver-1.smallmountain.local Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 2 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 2, origid 2, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 2 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: res_errno: 14, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 2, msgid 2) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: sasl_client_step: 2 Nov 01 09:09:48 auth(default): Error: sasl_client_step: 1 Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 3 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 3, origid 3, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 3 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: res_errno: 14, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 3, msgid 3) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: sasl_client_step: 0 Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 4 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 4, origid 4, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 4 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 4, msgid 4) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: ldap_pvt_sasl_generic_install Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:10:13 auth(default): Error: ldap_search Nov 01 09:10:13 auth(default): Error: put_filter: "(&(objectClass=person)(sAMAccountName=davidv))" Nov 01 09:10:13 auth(default): Error: put_filter: AND Nov 01 09:10:13 auth(default): Error: put_filter_list "(objectClass=person)(sAMAccountName=davidv)" Nov 01 09:10:13 auth(default): Error: put_filter: "(objectClass=person)" Nov 01 09:10:13 auth(default): Error: put_filter: simple Nov 01 09:10:13 auth(default): Error: put_simple_filter: "objectClass=person" Nov 01 09:10:13 auth(default): Error: put_filter: "(sAMAccountName=davidv)" Nov 01 09:10:13 auth(default): Error: put_filter: simple Nov 01 09:10:13 auth(default): Error: put_simple_filter: "sAMAccountName=davidv" Nov 01 09:10:13 auth(default): Error: ldap_build_search_req ATTRS: mail Nov 01 09:10:13 auth(default): Error: ldap_send_initial_request Nov 01 09:10:13 auth(default): Error: ldap_send_server_request Nov 01 09:10:13 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:10:13 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:10:13 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:10:13 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:10:13 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:10:13 auth(default): Error: last used: Tue Nov 1 09:10:13 2011 Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:10:13 auth(default): Error: * msgid 5, origid 5, status InProgress Nov 01 09:10:13 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:10:13 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:10:13 auth(default): Error: ldap_int_select Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid 5 message type search-entry Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid 5 message type search-result Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:10:13 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 5 Nov 01 09:10:13 auth(default): Error: request done: ld 0x18611700 msgid 5 Nov 01 09:10:13 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:10:13 auth(default): Error: ldap_free_request (origid 5, msgid 5) Nov 01 09:10:13 auth(default): Error: adding response ld 0x18611700 msgid 5 type 101: Nov 01 09:10:13 auth(default): Error: ldap_parse_result Nov 01 09:10:13 auth(default): Error: ldap_first_attribute Nov 01 09:10:13 auth(default): Error: ldap_get_values Nov 01 09:10:13 auth(default): Error: ldap_next_attribute Nov 01 09:10:13 auth(default): Error: ldap_msgfree Nov 01 09:10:13 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:10:13 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:10:13 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:10:13 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:10:13 auth(default): Error: refcnt: 1 status: Connected Nov 01 09:10:13 auth(default): Error: last used: Tue Nov 1 09:10:13 2011 Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 request count 0 (abandoned 0) Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:10:13 auth(default): Error: ldap_int_select Nov 01 09:13:26 auth(default): Error: ldap_search Nov 01 09:13:26 auth(default): Error: put_filter: "(&(objectClass=person)(sAMAccountName=davidv))" Nov 01 09:13:26 auth(default): Error: put_filter: AND Nov 01 09:13:26 auth(default): Error: put_filter_list "(objectClass=person)(sAMAccountName=davidv)" Nov 01 09:13:26 auth(default): Error: put_filter: "(objectClass=person)" Nov 01 09:13:26 auth(default): Error: put_filter: simple Nov 01 09:13:26 auth(default): Error: put_simple_filter: "objectClass=person" Nov 01 09:13:26 auth(default): Error: put_filter: "(sAMAccountName=davidv)" Nov 01 09:13:26 auth(default): Error: put_filter: simple Nov 01 09:13:26 auth(default): Error: put_simple_filter: "sAMAccountName=davidv" Nov 01 09:13:26 auth(default): Error: ldap_build_search_req ATTRS: mail Nov 01 09:13:26 auth(default): Error: ldap_send_initial_request Nov 01 09:13:26 auth(default): Error: ldap_send_server_request Nov 01 09:13:26 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:13:26 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:13:26 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:13:26 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:13:26 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:13:26 auth(default): Error: last used: Tue Nov 1 09:13:26 2011 Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:13:26 auth(default): Error: * msgid 6, origid 6, status InProgress Nov 01 09:13:26 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:13:26 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:13:26 auth(default): Error: ldap_int_select Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid 6 message type search-entry Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid 6 message type search-result Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:13:26 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 6 Nov 01 09:13:26 auth(default): Error: request done: ld 0x18611700 msgid 6 Nov 01 09:13:26 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:13:26 auth(default): Error: ldap_free_request (origid 6, msgid 6) Nov 01 09:13:26 auth(default): Error: adding response ld 0x18611700 msgid 6 type 101: Nov 01 09:13:26 auth(default): Error: ldap_parse_result Nov 01 09:13:26 auth(default): Error: ldap_first_attribute Nov 01 09:13:26 auth(default): Error: ldap_get_values Nov 01 09:13:26 auth(default): Error: ldap_next_attribute Nov 01 09:13:26 auth(default): Error: ldap_msgfree Nov 01 09:13:26 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:13:26 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:13:26 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:13:26 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:13:26 auth(default): Error: refcnt: 1 status: Connected Nov 01 09:13:26 auth(default): Error: last used: Tue Nov 1 09:13:26 2011 Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 request count 0 (abandoned 0) Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:13:26 auth(default): Error: ldap_int_select dovecot_info.log Nov 01 09:09:48 dovecot: Info: Dovecot v1.2.17 starting up Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99773 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99774 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99775 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99776 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99777 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99778 Nov 01 09:10:09 auth(default): Info: new auth connection: pid=99782 Nov 01 09:10:13 auth(default): Info: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=993 rport=55010 resp=AGRhdmlkdgBlbGkqNDJ0dW4= Nov 01 09:10:13 auth(default): Info: ldap(davidv,127.0.0.1): pass search: base=dc=smallmountain,dc=Local scope=subtree filter=(&(objectClass=person)(sAMAccountName=davidv)) fields=mail Nov 01 09:10:13 auth(default): Info: auth(davidv,127.0.0.1): username changed davidv -> davidv at smallmountain.net Nov 01 09:10:13 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): result: mail(user)=davidv at smallmountain.net Nov 01 09:10:13 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): No password returned (and no nopassword) Nov 01 09:10:15 auth(default): Info: client out: FAIL 1 user=davidv at smallmountain.net Nov 01 09:13:13 imap-login: Info: Disconnected: Inactivity (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Nov 01 09:13:21 auth(default): Info: new auth connection: pid=99797 Nov 01 09:13:26 auth(default): Info: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=993 rport=61080 resp=AGRhdmlkdgBlbGkqNDJ0dW4= Nov 01 09:13:26 auth(default): Info: ldap(davidv,127.0.0.1): pass search: base=dc=smallmountain,dc=Local scope=subtree filter=(&(objectClass=person)(sAMAccountName=davidv)) fields=mail Nov 01 09:13:26 auth(default): Info: auth(davidv,127.0.0.1): username changed davidv -> davidv at smallmountain.net Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): result: mail(user)=davidv at smallmountain.net Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): No password returned (and no nopassword) Nov 01 09:13:28 auth(default): Info: client out: FAIL 1 user=davidv at smallmountain.net Nov 01 09:13:36 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Your assistance is greatly appreciated, David From leo at strike.wu.ac.at Tue Nov 1 22:12:14 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Tue, 01 Nov 2011 21:12:14 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EAEDDDD.40206@gmx.de> References: <4EAED444.70608@strike.wu.ac.at> <4EAEDDDD.40206@gmx.de> Message-ID: <4EB0529E.2090808@strike.wu.ac.at> On 10/31/2011 06:41 PM, e-frog wrote: > On 31.10.2011 18:00, wrote Alexander 'Leo' Bergolth: >> Have environment based namespace settings been abandoned in 2.0? >> > I think you need to add them to USERDB_KEYS now... > > http://wiki2.dovecot.org/PostLoginScripting#Running_environment I have added NAMESPACE_1 NAMESPACE_1_SEP NAMESPACE_1_TYPE NAMESPACE_1_LIST NAMESPACE_1_PREFIX to USERDB_KEYS. Unfortunately the namespace still isn't used. :-( --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From pgnet.dev+rich-dovecot at gmail.com Tue Nov 1 22:53:47 2011 From: pgnet.dev+rich-dovecot at gmail.com (Rich) Date: Tue, 1 Nov 2011 13:53:47 -0700 Subject: [Dovecot] looking for Dovecot-code + SQL consultants Message-ID: Hi, We're using Dovecot2. Trying, given our own spread-too-thin bandwidth, to make it work within our evolving SQL application environment. When there's a problem, we post to this list (e.g., http://www.dovecot.org/list/dovecot/2011-October/061609.html), but aren't getting any/timely responses. We've decided to look for a consultant (hourly or retainer) that can be available for working with our in-house staff to straighten these issues out -- by helping us identify & fix our own mess, and by working to get fixes pushed to Dovecot project code, where appropriate. If you provide these services, rather than simply deployment or hosting, and are available, please drop me a line *offlist*. We're in the San Francisco area, and local is best, but remote work is certainly an option. Thanks, Rich From ghandidrivesahumvee at rocketfish.com Tue Nov 1 22:57:05 2011 From: ghandidrivesahumvee at rocketfish.com (Dovecot-GDH) Date: Tue, 1 Nov 2011 13:57:05 -0700 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EA916D1.30409@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: If I/O performance is a concern, you may be interested in ZFS and Flashcache. Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) ZFS does run on Linux http://zfs-fuse.net Flashcache: https://github.com/facebook/flashcache/ Both of these techniques can use a pair of SSDs in RAID1 rather than a single SSD. On Oct 27, 2011, at 1:31 AM, Ed W wrote: > On 27/10/2011 03:36, Stan Hoeppner wrote: >> On 10/26/2011 4:13 PM, Patrick Westenberg wrote: >>> Hi all, >>> >>> is anyone on this list who dares/dared to store his index files on a >>> MLC-SSD? >> I have not. But I can tell you that a 32GB Corsair MLC SSD in my >> workstation died after 4 months of laughably light duty. It had nothing >> to do with cell life but low product quality. This was my first foray >> into SSD. The RMA replacement is still kickin after 2 months, >> thankfully. I'm holding my breath... >> >> Scanning the reviews on Newegg shows early MLC SSD failures across most >> brands, early being a year or less. Some models/sizes are worse than >> others. OCZ has a good reputation overall, but reviews show some of >> their models to be grenades. >> >> Thus, if you were to put indexes on SSD, you should strongly consider >> using a mirrored pair. >> > > I don't think you are saying that the advice varies here compared with > HDDs? I do agree that some SSDs are showing very early failures, but > it's only a tweak to the probability parameter compared with any other > storage medium. They ALL fail at some point, and generally well within > the life of the rest of the server. Some kind of failure planning is > necessary > > Caveat the potentially higher failures vs HDDs I don't see any reason > why an SSD shouldn't work well? (even more so if you are using maildir > where indexes can be regenerated). > > More interestingly: for small sizes like 32GB, has anyone played with > the "compressed ram with backing store" thing in newer kernels (that I > forget the name of now). I think it's been marketed for swap files, but > assuming I got the theory it could be used as a ram drive with slow > writeback to permanent storage? > > Good luck > > Ed W From pw at wk-serv.de Tue Nov 1 23:19:11 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 01 Nov 2011 22:19:11 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: <4EB0624F.90307@wk-serv.de> Dovecot-GDH schrieb: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was thinking about a SSD based LUN for the indexes. As I'm using multiple servers this LUN will use OCFS2. From dswartz at druber.com Tue Nov 1 23:40:39 2011 From: dswartz at druber.com (Dan Swartzendruber) Date: Tue, 1 Nov 2011 17:40:39 -0400 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB0624F.90307@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com><4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> Message-ID: <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> I can't imagine running any kind of performance critical app on linux using fuse! There is a native ZFS port going on, but I don't know how stable it is yet. -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Patrick Westenberg Sent: Tuesday, November 01, 2011 5:19 PM To: dovecot at dovecot.org Subject: Re: [Dovecot] Indexes to MLC-SSD Dovecot-GDH schrieb: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was thinking about a SSD based LUN for the indexes. As I'm using multiple servers this LUN will use OCFS2. From janfrode at tanso.net Wed Nov 2 10:57:15 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 2 Nov 2011 09:57:15 +0100 Subject: [Dovecot] doveadm delete-user ? Message-ID: <20111102085715.GA4089@dibs.tanso.net> IMHO it would be great if we could both provision and un-provision users without having the provisioning system mess with files in the filesystem. Provisioning new users works fine since dovecot will create all files/directories on first login, but I haven?t found the tools to un-provison them. I see "doveadm mailbox delete" can delete individual mailboxes, but what about indexes, sieve-scripts and home-directories? Are there any tools to clean out these ? -jf From goetz.reinicke at filmakademie.de Wed Nov 2 15:59:21 2011 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Wed, 02 Nov 2011 14:59:21 +0100 Subject: [Dovecot] Upgrading dovecot 1.0.7 -> 2.0.14 / Redhat EL 5.7 Message-ID: <4EB14CB9.9030708@filmakademie.de> Hi, recently I debugged some load and performance and 'log out' problems and found an error in the dovecot logs: Error: pipe() failed: Too many open files Googling around and reading some posts led to the conclusion, that my problem is a bug, which will not be removed. So I'm thinking of upgrading to the latest prebuild rpm from here: http://packages.atrpms.net/dist/el5/dovecot/ As far as I understand the upgrading instructions, I'll be save as we dont use special setting, but I'll check all changes. My question: Dose anybody has done such an upgrade yet? Or are there any suggestion or hints? Thanks a lot and best regards. G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From arif at team.de.inter.net Wed Nov 2 16:28:05 2011 From: arif at team.de.inter.net (Kai Arif) Date: Wed, 02 Nov 2011 15:28:05 +0100 Subject: [Dovecot] Different default realm per listener Message-ID: <4EB15375.30106@team.de.inter.net> Hi all! I want create a large IMAP server setup where dovecot is listening on multiple interfaces and needs a different default realm on each interface. I'm using dovecot 2.0.5 on FreeBSD9 and authenticate against a LDAP database. I found an old mail thread from 2007 that said I'm basically screwed. Has that change since than? Is there any way to use different default realms per listener? Best regards Kai -- Kai Arif - System Administrator Inter.net Germany GmbH Knesebeckstra?e 59-61 10719 Berlin Germany Fon +49 30 25430 0 Fax +49 30 25430 499 arif at team.de.inter.net www.de.inter.net Legal-Information: Inter.net Germany GmbH, HRB #79136, Amtsgericht Berlin Charlottenburg, UST-IdNr:: DE 813 165 159 FA f?r K?rperschaften I Berlin, Gesch?ftsf?hrer: J?rn Lubkoll Zust?ndige Regulierungsbeh?rde: Bundesnetzagentur Chemnitz, Liselotte-Herrmann-Str. 20a, 09127 Chemnitz, Registriernummer: 06/164 From rick at havokmon.com Wed Nov 2 16:37:47 2011 From: rick at havokmon.com (Rick Romero) Date: Wed, 02 Nov 2011 09:37:47 -0500 Subject: [Dovecot] IMP, Dovecot and multiple namespaces Message-ID: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> Hey guys, I'm wondering what the best way is to be backwards compatible with Courier mailbox formats and not duplicate mailbox trees with Dovecot. Is anyone doing this right now? My Dovecot 2.0.13 is configured as follows for Namespaces: namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } So I should have an INBOX, plus additional root folders - rather than the Courier default of all folders being subfolders of the INBOX. So when I run LIST in IMAP I get - 08 LIST "" % * LIST (\HasChildren) "." "INBOX" * LIST (\HasChildren) "." "Spam" * LIST (\HasNoChildren) "." "Sent" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "Junk" * LIST (\HasNoChildren) "." "Drafts" Which looks fine. The problem is, IMP (old and new) show two sets of mailboxes (one for each namespace) like so: INBOX Spam Spam.Over10 Sent INBOX.Spam INBOX.Spam.Over10 INBOX.Sent (etc) I assume IMP is walking the INBOX because it says it has Children, and displaying those Children - The Horde team is telling me that's the root of the problem. It makes sense, but I'm not sure what the real solution is. Is it possible to disable Children for a mailbox when an alternate hidden namespace matches the mailbox name? Does that make sense? Or, is there another option I change change in the dovecot config to accomplish the same? Rick From jtam.home at gmail.com Thu Nov 3 02:43:04 2011 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 2 Nov 2011 17:43:04 -0700 (PDT) Subject: [Dovecot] doveadm delete-user ? In-Reply-To: References: Message-ID: Jan-Frode Myklebust writes: > IMHO it would be great if we could both provision and un-provision users > without having the provisioning system mess with files in the > filesystem. Provisioning new users works fine since dovecot will create > all files/directories on first login, but I haven?t found the tools to > un-provison them. > I see "doveadm mailbox delete" can delete individual > mailboxes, but what about indexes, sieve-scripts and home-directories? > Are there any tools to clean out these ? I coundn't find any, so I wrote my own executable cron job that scans the index cache and checks if the corresponding mailbox exists. If not, it echo the commands to remove the stale indices. From there, you can review it, or pipe it into sh. I can post it if there's any interest, but it's not that hard to write. This is only necessary, of course, if you store the indices in a different place than the mail folder. Joseph Tam From scott_the_musician at yahoo.com.au Thu Nov 3 07:31:54 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Wed, 2 Nov 2011 22:31:54 -0700 (PDT) Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? Message-ID: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Hi all, I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. Here's what happens: [root at mail ~]# whereis dovecot dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot ... checking whether to build static libraries... yes dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH to give path to compiled Dovecot sources or to a directory with the installed dovecot-config file. configure: error: dovecot-config not found -- I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: Avelsieve plugin details: backend = ManageSieve ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? thanks in advance! From lists at wildgooses.com Thu Nov 3 10:29:32 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 08:29:32 +0000 Subject: [Dovecot] Thunderbird slow in talking with dovecot IMAP AND to sendmail In-Reply-To: <4EA68C11.5060102@tlinx.org> References: <4EA68C11.5060102@tlinx.org> Message-ID: <4EB250EC.90204@wildgooses.com> On 25/10/2011 11:14, Linda Walsh wrote: > > > I'm trying to find out what's causing this slowdown -- it's > INTOLERABLE.... > > over 1 minute and less than 1% done. (400MB file)... > > After trying 3 times, I gave up and logged in using X to the server > and ran Tbird from there.... > > Mail sent out in < 1 minute, though the copy to dovecot took about 50% > longer. > > So... > > I looked at the network trace. > > and everyfrackin' body was using 4K packet sizes (at the application > level!, the window size on TCP was over 64K...but no one was using > it)....especially galling with my network's MTU at 9K, BTW, because > small packets are really bad on a 1Gb network. > Although larger packets might be helpful, I don't see that you shouldn't be getting much faster speed without it? Even the 64K window, whilst it looks too small, might be ok if your ping times are very low? Something else is limiting your performance I think? Ed W From maria_arrea at gmx.com Thu Nov 3 11:54:24 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 10:54:24 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103095424.67160@gmx.com> Hello. We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 Nov 3 09:23:07 buzon dovecot: imap(delolmo at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/15/77/delolmo/subscriptions Nov 3 09:23:07 buzon dovecot: imap(delolmo at mydomain): Connection closed bytes=0/295 I have been googling but I only see problems with remote NFS, our setup does not use NFS. I give you doveconf -n output & mount options, if more info es needed, please ask. doveconf -n output # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.el5 x86_64 Red Hat Enterprise Linux Server release 5.7 (Tikanga) ext4 auth_cache_negative_ttl = 10 secs auth_cache_size = 10 M auth_cache_ttl = 2 mins auth_master_user_separator = * auth_mechanisms = plain login auth_worker_max_count = 3500 base_dir = /var/run/dovecot/ default_client_limit = 5000 default_process_limit = 6500 disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep delay-newmail tb-lsub-flags lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_fsync = never mail_gid = entrega mail_home = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/ mail_location = mdbox:/buzones/mydomain/%2.26Hn/%2.200Hn/%n:INDEX=/indices_dovecot/indices/%2.26Hn/%2.200Hn/%n mail_max_userip_connections = 15000 mail_plugins = " zlib acl quota autocreate" mail_uid = entrega managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mdbox_rotate_interval = 1 days mdbox_rotate_size = 60 M passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/usuario_maestro.txt driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile autocreate = SPAM autocreate2 = Sent autocreate3 = Drafts autocreate4 = Trash autosubscribe = SPAM autosubscribe2 = Sent autosubscribe3 = Drafts autosubscribe4 = Trash lda_mailbox_autosubscribe = yes quota = dict:Cuota de usuario::file:/buzones/cuotas/%n quota_rule2 = Trash:storage=+10%% quota_warning = storage=90%% aviso_cuota 90 %u sieve = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/dovecot.sieve sieve_dir = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/sieve/ zlib_save = gz zlib_save_level = 9 } pop3_no_flag_updates = yes protocols = pop3 imap sieve service anvil { client_limit = 25000 } service auth { client_limit = 28000 unix_listener auth-master { user = entrega } unix_listener auth-userdb { user = entrega } user = root } service aviso_cuota { executable = script /usr/local/bin/quota-warning.sh unix_listener aviso_cuota { mode = 0666 } user = entrega } service imap-login { executable = /usr/libexec/dovecot/imap-login group = dovenull service_count = 0 } service imap { executable = /usr/libexec/dovecot/imap process_limit = 6000 } service managesieve-login { executable = /usr/libexec/dovecot/managesieve-login inet_listener sieve { port = 2000 } process_limit = 2000 } service managesieve { executable = /usr/libexec/dovecot/managesieve process_limit = 5000 } service pop3-login { executable = /usr/libexec/dovecot/pop3-login process_limit = 4000 service_count = 0 } service pop3 { executable = /usr/libexec/dovecot/pop3 process_limit = 4000 } ssl_ca = References: <20111103095424.67160@gmx.com> Message-ID: <4EB26C2F.1050505@wildgooses.com> Hi > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:30:09 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:30:09 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103103009.67160@gmx.com> We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:25 AM To: dovecot at dovecot.org Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Hi > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W From lists at wildgooses.com Thu Nov 3 12:31:21 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:31:21 +0000 Subject: [Dovecot] Imap/pop gateway In-Reply-To: References: Message-ID: <4EB26D79.1010206@wildgooses.com> On 31/10/2011 22:20, nuno marques wrote: > > > > Hello, > How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. > The question isn't entirely clear, but I *think* you just want to use the normal "proxy" feature of dovecot. This accepts connections on one machine, examines them until the end of the auth stage and passes them onto some other machine based on the results of the auth process Also there are other imap/pop proxies such as nginx That said I'm not sure how much security this really buys you versus port forwarding POP/IMAP ports to your real server? If the proxy machine were to get hacked (over imap?) then the same hack can jump from the proxy to the real server. Also your only exposure in each case is via POP/IMAP, which means you would be mainly chasing buffer overflow vulnerabilities and the like. These can also be mitigated by chrooting the server machine (please consider virtualisation options, it's usually simpler/faster/saner, eg see my favourite: linux-vservers), MAC controls on the dovecot process (grsec/selinux, etc), and compiler extensions (gcc hardened) Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:35:46 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:35:46 +0100 Subject: [Dovecot] Imap/pop gateway Message-ID: <20111103103546.67180@gmx.com> If you are going to use an imap proxy for security reasons, consider using a software DIFFERENT than in your real mailboxes. If you use dovecot in your backend, you could use perdition in the frontend. Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:31 AM To: Dovecot Mailing List Subject: Re: [Dovecot] Imap/pop gateway On 31/10/2011 22:20, nuno marques wrote: > > > > Hello, > How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. > The question isn't entirely clear, but I *think* you just want to use the normal "proxy" feature of dovecot. This accepts connections on one machine, examines them until the end of the auth stage and passes them onto some other machine based on the results of the auth process Also there are other imap/pop proxies such as nginx That said I'm not sure how much security this really buys you versus port forwarding POP/IMAP ports to your real server? If the proxy machine were to get hacked (over imap?) then the same hack can jump from the proxy to the real server. Also your only exposure in each case is via POP/IMAP, which means you would be mainly chasing buffer overflow vulnerabilities and the like. These can also be mitigated by chrooting the server machine (please consider virtualisation options, it's usually simpler/faster/saner, eg see my favourite: linux-vservers), MAC controls on the dovecot process (grsec/selinux, etc), and compiler extensions (gcc hardened) Good luck Ed W From giulio at dsi.unimi.it Thu Nov 3 12:38:17 2011 From: giulio at dsi.unimi.it (Giulio Casella) Date: Thu, 03 Nov 2011 11:38:17 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103103009.67160@gmx.com> References: <20111103103009.67160@gmx.com> Message-ID: <4EB26F19.1080105@dsi.unimi.it> This could be the problem. Double check the time also on your host system(s), not only on guest. Bye, gc Il 03/11/2011 11.30, Maria Arrea ha scritto: > We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:25 AM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > Hi> We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W > -- Giulio Casella giulio at dsi.unimi.it System and network manager Computer Science Dept. - University of Milano From lists at wildgooses.com Thu Nov 3 12:42:36 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:42:36 +0000 Subject: [Dovecot] How can we horizontally scale Dovecot across multiple servers? In-Reply-To: References: <003001cc97a9$addbeb80$0993c280$@mvs.us> <4EAE62AA.6080203@schetterer.org> <000301cc97b1$81100ca0$833025e0$@mvs.us> <4EAE71C1.7060400@schetterer.org> Message-ID: <4EB2701C.6060706@wildgooses.com> On 31/10/2011 11:28, Felipe Scarel wrote: > Quick question about the usage of DRBD: I'm thinking of a setup on my > organization here (15k+ users, 4TB of email data), but I'm holding back on > the clusterization due to the high volume of data. > > Using DRBD would implicate mirroring those 4TB of data across all cluster > nodes? If yes, I might go with a SAN-based solution, though I haven't I don't the technique with DRBD is something like having pairs of machines, each of which is a backup for the other. There were some old notes on the Dovecot website about such a setup? Roughly I seem to recall that each pair of machines ran two virtual machines, each of which ran active on one of the nodes each, but could migrate to the other if needed. Add a bunch of such paired nodes to get to the performance you require and put a dovecot proxy instance in front of the whole lot In contrast the SAN solution uses a clustered filesystem (opinion varies on which performs best) and then in theory every machine has access to every mailbox. In practice access to the SAN is relatively slow compared with local storage, so the technique seems to be to store indexes on the local machine and then using the front end proxy to be somewhat "sticky" in returning users to the same backend node so that the indexes can be re-used and not rebuilt The DRBD solution offers local disk access speed to the node and would on the surface give far faster performance (if disk were the limiting issue). However, it's likely to be more complex to maintain and manage and without buying licences you get only failover between pairs of machines. The SAN solution in theory looks like perfect scale up, big backend and just add more backend IMAP nodes as you need them, and all the clever stuff moves to the frontend load balancer to be "sticky" and obviously that's your main maintenance problem. However, based on evidence from users of big systems, IO is likely to be your main bottleneck and so just theoretically, the SAN will only scale as far as it doesn't run out of IOs... Using local disk for indexes would tend to reduce the amount of IOs needed (from the SAN) very dramatically, but you still have some limit out there and it's a question of whether you will reach it? DRBD has theoretical infinite scale out because each time you add another pair you get more IO as well as more CPU I don't have the fortune to have anything like the volume of users you have so I have no opinion to offer... However, I think the above accurately summarises your options. Others might help clarify the likely bounds on performance of each solution and maintenance headaches (eg some have had problems with maildir mounted on OCFS/GFS2 and fixed that by moving to dbox, etc) Please report on your results! Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:49:04 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:49:04 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103104904.67150@gmx.com> All the ESXs hosts and all the VM use the same NTP server. Any other idea? Regards Maria ----- Original Message ----- From: Giulio Casella Sent: 11/03/11 11:38 AM To: dovecot at dovecot.org Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube This could be the problem. Double check the time also on your host system(s), not only on guest. Bye, gc Il 03/11/2011 11.30, Maria Arrea ha scritto: > We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:25 AM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > Hi> We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W > -- Giulio Casella giulio at dsi.unimi.it System and network manager Computer Science Dept. - University of Milano From stephan at rename-it.nl Thu Nov 3 12:54:13 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 03 Nov 2011 11:54:13 +0100 Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Message-ID: <4EB272D5.1090403@rename-it.nl> Op 3-11-2011 6:31, Scott Lewis schreef: > Hi all, > > I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. > > Here's what happens: > > [root at mail ~]# whereis dovecot > dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz > > [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot > > ... > > checking whether to build static libraries... yes > dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH > to give path to compiled Dovecot sources or to a directory with the > installed dovecot-config file. configure: error: dovecot-config not found > > -- > > I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I'm not familiar with CentOS, but there usually is a separate package containing the Dovecot development headers and the dovecot-config file you need. By the looks of things, that is not installed at your end. Point the --with-dovecot to wherever the dovecot-config file is installed. Regards, Stephan. From lists at wildgooses.com Thu Nov 3 12:57:37 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:57:37 +0000 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103104904.67150@gmx.com> References: <20111103104904.67150@gmx.com> Message-ID: <4EB273A1.4030106@wildgooses.com> On 03/11/2011 10:49, Maria Arrea wrote: > All the ESXs hosts and all the VM use the same NTP server. > > Any other idea? > Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 13:03:36 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 12:03:36 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103110336.67190@gmx.com> We follow the guidelines about timekeeping RHEL in vmware vsphere located here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 These problems happens in peak hours. Any dovecot config parameter I could set to mitigate this problem? Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:57 AM To: Maria Arrea, Dovecot Mailing List Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube On 03/11/2011 10:49, Maria Arrea wrote: > All the ESXs hosts and all the VM use the same NTP server. > > Any other idea? > Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W From fbscarel at gmail.com Thu Nov 3 13:32:34 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Thu, 3 Nov 2011 09:32:34 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> Message-ID: I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ users, over 2TB of mail data) with little issues. Dedup and compression disabled, mind that. Dedup especially is a major source of trouble, I wouldn't recommend it for production just yet. Cheers, fbscarel On Tue, Nov 1, 2011 at 19:40, Dan Swartzendruber wrote: > > I can't imagine running any kind of performance critical app on linux using > fuse! There is a native ZFS port going on, but I don't know how stable it > is yet. > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On > Behalf Of Patrick Westenberg > Sent: Tuesday, November 01, 2011 5:19 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Indexes to MLC-SSD > > Dovecot-GDH schrieb: > > If I/O performance is a concern, you may be interested in ZFS and > Flashcache. > > > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive > Read Cache) > > ZFS does run on Linux http://zfs-fuse.net > > I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was > thinking about a SSD based LUN for the indexes. As I'm using multiple > servers this LUN will use OCFS2. > > From lists at wildgooses.com Thu Nov 3 16:27:16 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 14:27:16 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> Message-ID: <4EB2A4C4.2080100@wildgooses.com> On 03/11/2011 11:32, Felipe Scarel wrote: > I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ > users, over 2TB of mail data) with little issues. Dedup and compression > disabled, mind that. > OT: but what were the rough criteria that led you to using ZFS over say LVM with EXT4/XFS/btrfs? I can think of plenty for/against reasons for each, just wondering what criteria affected *your* situation? I'm guessing some kind of manageability reason is at the core, but perhaps you can expand on how it's all worked out for you? I have a fairly static server setup here so I have been "satisfied" with LVM, software raid and mainly ext4. The main thing I miss is simple to use snapshots Cheers Ed W From lists at wildgooses.com Thu Nov 3 17:21:01 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 15:21:01 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB0624F.90307@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> Message-ID: <4EB2B15D.1010005@wildgooses.com> > I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was > thinking about a SSD based LUN for the indexes. As I'm using multiple > servers this LUN will use OCFS2. Given that the SAN always has the network latency behind it, might you be better to look at putting the SSDs in the frontend machines? Obviously this then needs some way to make users "sticky" to one machine (or some few machines) where the indexes are stored? This seems theoretically likely to give you higher IOPs to the index than having them on the OCFS2 storage? (At a trade off with more complexity for the load balancer front end...) Ed W From alietsantiesteban at gmail.com Thu Nov 3 17:52:24 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Thu, 3 Nov 2011 11:52:24 -0400 Subject: [Dovecot] How to define ldap connection idle Message-ID: I'm having a problem with dovecot ldap connection when ldap server is in another firewall zone, firewall kills the ldap connection after a determined period of inactivity, this is good from the firewall point of view but is bad for dovecot because it never knows the connections has been dropped, this creates longs timeouts in dovecot and finally it reconnects, meanwhile many users fails to authenticate, I have seen this kind of post in the list for a while but can't find a solution for it, so my question is how to define a idle ldap time in dovecot so it can reconnect before the firewall has dropped the connection or just close the connection under inactivity so when a user authenticate doesn't fails for a while until dovecot detects that the connection has hanged. Is this a feature request or there is already a configuration for this??? Thank's in advance and congrats tu Timo for this great app. From pgnet.dev+rich-dovecot at gmail.com Thu Nov 3 18:37:13 2011 From: pgnet.dev+rich-dovecot at gmail.com (Rich) Date: Thu, 3 Nov 2011 09:37:13 -0700 Subject: [Dovecot] looking for Dovecot-code + SQL consultants In-Reply-To: References: Message-ID: Hi, I've already received a number of replies from providers offering to help out. I'll be in touch with each, and am certain we'll be able to find the right solution from among them. Thanks for the responses, Rich On Tue, Nov 1, 2011 at 1:53 PM, Rich wrote: > Hi, > > We're using Dovecot2. ?Trying, given our own spread-too-thin > bandwidth, to make it work within our evolving SQL application > environment. > > When there's a problem, we post to this list (e.g., > http://www.dovecot.org/list/dovecot/2011-October/061609.html), but > aren't getting any/timely responses. > > We've decided to look for a consultant (hourly or retainer) that can > be available for working with our in-house staff to straighten these > issues out -- by helping us identify & fix our own mess, and by > working to get fixes pushed to Dovecot project code, where > appropriate. > > If you provide these services, rather than simply deployment or > hosting, and are available, please drop me a line *offlist*. ?We're in > the San Francisco area, and local is best, but remote work is > certainly an option. > > Thanks, > > Rich > From pw at wk-serv.de Thu Nov 3 18:53:24 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 03 Nov 2011 17:53:24 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2B15D.1010005@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> Message-ID: <4EB2C704.1010107@wk-serv.de> Ed W schrieb: >> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >> thinking about a SSD based LUN for the indexes. As I'm using multiple >> servers this LUN will use OCFS2. > > Given that the SAN always has the network latency behind it, might you > be better to look at putting the SSDs in the frontend machines? > Obviously this then needs some way to make users "sticky" to one machine > (or some few machines) where the indexes are stored? Storing the indexes on several machines? In this case I have to synchronize them. From dswartz at druber.com Thu Nov 3 18:56:45 2011 From: dswartz at druber.com (Dan Swartzendruber) Date: Thu, 03 Nov 2011 12:56:45 -0400 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2C704.1010107@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> Message-ID: <4EB2C7CD.2090707@druber.com> Patrick Westenberg wrote: > Ed W schrieb: > >>> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >>> thinking about a SSD based LUN for the indexes. As I'm using multiple >>> servers this LUN will use OCFS2. >> >> Given that the SAN always has the network latency behind it, might you >> be better to look at putting the SSDs in the frontend machines? >> Obviously this then needs some way to make users "sticky" to one machine >> (or some few machines) where the indexes are stored? > > Storing the indexes on several machines? > In this case I have to synchronize them. > maybe i am missing something. if a client has to fetch the index, the server has to read the index from disk and pass it back. the network latency is unavoidable, but i don't see why putting the fastest possible SSD on the server isn't a win. possibly i am misunderstanding something? From thierry at mailhub.co.za Thu Nov 3 20:13:18 2011 From: thierry at mailhub.co.za (Thierry de Montaudry) Date: Thu, 3 Nov 2011 19:13:18 +0100 Subject: [Dovecot] Restricting IMAP access Message-ID: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Hi list, I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). Anybody has a trick for that? Running dovecot 2.0.13. I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. Regards, Thierry From fbscarel at gmail.com Thu Nov 3 20:24:06 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Thu, 3 Nov 2011 16:24:06 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2A4C4.2080100@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> Message-ID: Reasons to choose ZFS were snapshots, and mainly dedup and compression capabilities. I know, it's ironic since I'm not able to use them now due to severe performance issues with them (mostly dedup) turned on. I do like the emphasis on data integrity and fast on-the-fly configurability of ZFS to an extent, but I wouldn't recommend it highly for new users, especially for production. It works (in fact it's working right now), but has its fair share of troubles. We've started implementations to move our mail system to a more modular enviroment and we'll probably move away from ZFS. Was a nice experiment nonetheless, I learned quite a bit from it. On Thu, Nov 3, 2011 at 12:27, Ed W wrote: > On 03/11/2011 11:32, Felipe Scarel wrote: > > I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ > > users, over 2TB of mail data) with little issues. Dedup and compression > > disabled, mind that. > > > > OT: but what were the rough criteria that led you to using ZFS over say > LVM with EXT4/XFS/btrfs? I can think of plenty for/against reasons for > each, just wondering what criteria affected *your* situation? I'm > guessing some kind of manageability reason is at the core, but perhaps > you can expand on how it's all worked out for you? > > I have a fairly static server setup here so I have been "satisfied" with > LVM, software raid and mainly ext4. The main thing I miss is simple to > use snapshots > > Cheers > > Ed W > From robert at schetterer.org Thu Nov 3 20:46:16 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 03 Nov 2011 19:46:16 +0100 Subject: [Dovecot] Restricting IMAP access In-Reply-To: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> References: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Message-ID: <4EB2E178.8060103@schetterer.org> Am 03.11.2011 19:13, schrieb Thierry de Montaudry: > Hi list, > > I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. > I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). > Anybody has a trick for that? Running dovecot 2.0.13. > > I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. > > Regards, > > Thierry i am shot in time , with a databse this should be get to work i have it as flag for all users, so i can forbid imap to special ones as far i remember ther should be examples on the dovecot site , and it was written about here on the list before -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From simon.brereton at buongiorno.com Thu Nov 3 22:42:40 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 3 Nov 2011 16:42:40 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN Message-ID: Hi Could someone explain to me the difference between LOGIN and PLAIN? I've been googling for a while, but haven't found anything. Thanks. Simon From p at state-of-mind.de Thu Nov 3 23:00:24 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 3 Nov 2011 22:00:24 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <20111103210023.GB2312@state-of-mind.de> * Simon Brereton : > Could someone explain to me the difference between LOGIN and PLAIN? In SMTP these are: Both - are plaintext mechanisms. - base64 encode identification data before they send it over the wire - do not encrypt the indentification data and should therefore only be offered over an encrypted transport layer PLAIN - is an open standard supported by most clients - sends identification data as one string - sends an authentication ID, an authorization ID and the password LOGIN - is a proprietary standard supported by Microsofts clients - sends LOGIN, login name, password and optionally the domain name one after another I guess they are basically the same in IMAP, but others will know better. p at rick > I've been googling for a while, but haven't found anything. > > Thanks. > > Simon -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From stephan at rename-it.nl Thu Nov 3 23:01:17 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 03 Nov 2011 22:01:17 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <4EB3011D.80108@rename-it.nl> On 11/3/2011 9:42 PM, Simon Brereton wrote: > Hi > > Could someone explain to me the difference between LOGIN and PLAIN? > I've been googling for a while, but haven't found anything. The LOGIN SASL mechanism is an obsolete plain text mechanism. It is documented here: http://tools.ietf.org/html/draft-murchison-sasl-login-00 Some clients still support it, but I would not recommend using it when PLAIN or a better SASL mechanism is also available at both ends. The PLAIN mechanism is documented here: http://tools.ietf.org/html/rfc4616 The main technical difference between the two is that the PLAIN mechanism transfers both username and password in a single SASL interaction, where LOGIN needs two. The PLAIN mechanism also provides support for having an authorization id different from the authentication id, allowing for master user login for example. Regards, Stephan. From dovecot.user at seibercom.net Thu Nov 3 23:17:19 2011 From: dovecot.user at seibercom.net (Jerry) Date: Thu, 3 Nov 2011 17:17:19 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <20111103171719.55e0a7b9@scorpio> On Thu, 3 Nov 2011 16:42:40 -0400 Simon Brereton articulated: > Hi > > Could someone explain to me the difference between LOGIN and PLAIN? > I've been googling for a while, but haven't found anything. You could start here for some basic information: http://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer http://wiki.dovecot.org/Authentication/Mechanisms -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From simon.brereton at buongiorno.com Thu Nov 3 23:28:28 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 3 Nov 2011 17:28:28 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: <4EB3011D.80108@rename-it.nl> References: <4EB3011D.80108@rename-it.nl> Message-ID: On 3 November 2011 17:01, Stephan Bosch wrote: > On 11/3/2011 9:42 PM, Simon Brereton wrote: >> >> Hi >> >> Could someone explain to me the difference between LOGIN and PLAIN? >> I've been googling for a while, but haven't found anything. > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > documented here: > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > Some clients still support it, but I would not recommend using it when PLAIN > or a better SASL mechanism is also available at both ends. The PLAIN > mechanism is documented here: > > http://tools.ietf.org/html/rfc4616 > > The main technical difference between the two is that the PLAIN mechanism > transfers both username and password in a single SASL interaction, where > LOGIN needs two. The PLAIN mechanism also provides support for having an > authorization id different from the authentication id, allowing for master > user login for example. Thanks to both of you. Can I bet that Outlook doesn't support anything but plain? I'm not sure I've ever heard of a client supporting other than Evolution supporting MD5 passwords.. Simon From stan at hardwarefreak.com Fri Nov 4 01:01:29 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 03 Nov 2011 18:01:29 -0500 Subject: [Dovecot] How can we horizontally scale Dovecot across multiple servers? In-Reply-To: <003b01cc9565$c3b86960$4b293c20$@mvs.us> References: <003b01cc9565$c3b86960$4b293c20$@mvs.us> Message-ID: <4EB31D49.9090009@hardwarefreak.com> On 10/28/2011 6:35 AM, Arlin wrote: > How can we horizontally scale Dovecot across multiple servers? Do we require > to install independent instances of Dovecot on each server? The performance limiting factors for IMAP servers are disk IO, memory size, network bandwidth/latency, and processor power, usually in that order. To scale Dovecot horizontally would imply multiple servers each with independent storage handling a subset of the total concurrent user load. > We are planning to use a NAS/SAN device using ZFS or EFS for email storage. > Each logical unit will be of 10TB and similarly as the no: of user increases > we are planning to add multiple 10TB units. This does not describe horizontal maildir performance scaling. This simply describes vertical scaling of the IO, to a point, as the nodes are all attached to one controller with finite IOPS performance, which is dictated by the speed of the IO ASIC and cache memory bandwidth WRT writes, and aggregate spindle IOPS WRT reads. With this setup, you simply gain multi-node redundancy of the front end servers. However, you will *lose* performance compared to a single equivalent large host due to NFS/cluster filesystem overhead, latency, locking, etc, and your management/troubleshooting burden will be increased. If your goal is simply increased performance and storage expandability, go with a single large host system. I'd only go the cluster route if you believe you need the redundancy. -- Stan From stan at hardwarefreak.com Fri Nov 4 04:16:09 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 03 Nov 2011 21:16:09 -0500 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> Message-ID: <4EB34AE9.6070002@hardwarefreak.com> On 11/3/2011 1:24 PM, Felipe Scarel wrote: > Reasons to choose ZFS were snapshots, and mainly dedup and compression > capabilities. I know, it's ironic since I'm not able to use them now due to > severe performance issues with them (mostly dedup) turned on. > > I do like the emphasis on data integrity and fast on-the-fly > configurability of ZFS to an extent, but I wouldn't recommend it highly for > new users, especially for production. It works (in fact it's working right > now), but has its fair share of troubles. > > We've started implementations to move our mail system to a more modular > enviroment and we'll probably move away from ZFS. Was a nice experiment > nonetheless, I learned quite a bit from it. I find this all very interesting... "Please keep in mind the current 0.5.2 stable release does not yet support a mountable filesystem. This functionality is currently available only in the 0.6.0-rc6 release candidate." https://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz "Uploaded October 14, 2011" So in the past ~two weeks, you converted your 15K+ user production server to ZFS on Linux, as an experiment, and have now decided to change to another filesystem solution, a mere two weeks later? Or am I misinterpreting the date given that 0.6.0-rc6 was released? -- Stan From p at state-of-mind.de Fri Nov 4 00:56:11 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 3 Nov 2011 23:56:11 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: <4EB3011D.80108@rename-it.nl> Message-ID: <20111103225611.GA14437@state-of-mind.de> * Simon Brereton : > On 3 November 2011 17:01, Stephan Bosch wrote: > > On 11/3/2011 9:42 PM, Simon Brereton wrote: > >> > >> Hi > >> > >> Could someone explain to me the difference between LOGIN and PLAIN? > >> I've been googling for a while, but haven't found anything. > > > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > > documented here: > > > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > > > Some clients still support it, but I would not recommend using it when PLAIN > > or a better SASL mechanism is also available at both ends. The PLAIN > > mechanism is documented here: > > > > http://tools.ietf.org/html/rfc4616 > > > > The main technical difference between the two is that the PLAIN mechanism > > transfers both username and password in a single SASL interaction, where > > LOGIN needs two. The PLAIN mechanism also provides support for having an > > authorization id different from the authentication id, allowing for master > > user login for example. > > Thanks to both of you. Can I bet that Outlook doesn't support > anything but plain? Outlook > 2007 LOGIN, NTLM Outlook 2010 > LOGIN, NTLM2, DIGEST-MD5 > I'm not sure I've ever heard of a client supporting other than > Evolution supporting MD5 passwords.. Two come to mind: mutt, Thunderbird However DIGEST-MD5 has been marked deprecated this summer. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From javierdemiguel at us.es Fri Nov 4 11:52:22 2011 From: javierdemiguel at us.es (=?UTF-8?B?SmF2aWVyIGRlIE1pZ3VlbCBSb2Ryw61ndWV6?=) Date: Fri, 04 Nov 2011 10:52:22 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103110336.67190@gmx.com> References: <20111103110336.67190@gmx.com> Message-ID: <4EB3B5D6.10102@us.es> Same problem here, any hint about a fix or workaround? Regards Javier > We follow the guidelines about timekeeping RHEL in vmware vsphere located here > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 > > These problems happens in peak hours. Any dovecot config parameter I could set to mitigate this problem? > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:57 AM > To: Maria Arrea, Dovecot Mailing List > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > On 03/11/2011 10:49, Maria Arrea wrote:> All the ESXs hosts and all the VM use the same NTP server.> > Any other idea?> Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W > From dovecot.user at seibercom.net Fri Nov 4 01:24:43 2011 From: dovecot.user at seibercom.net (Jerry) Date: Thu, 3 Nov 2011 19:24:43 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: <4EB3011D.80108@rename-it.nl> Message-ID: <20111103192443.4e239384@scorpio> On Thu, 3 Nov 2011 17:28:28 -0400 Simon Brereton articulated: > On 3 November 2011 17:01, Stephan Bosch wrote: > > On 11/3/2011 9:42 PM, Simon Brereton wrote: > >> > >> Hi > >> > >> Could someone explain to me the difference between LOGIN and PLAIN? > >> I've been googling for a while, but haven't found anything. > > > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > > documented here: > > > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > > > Some clients still support it, but I would not recommend using it > > when PLAIN or a better SASL mechanism is also available at both > > ends. The PLAIN mechanism is documented here: > > > > http://tools.ietf.org/html/rfc4616 > > > > The main technical difference between the two is that the PLAIN > > mechanism transfers both username and password in a single SASL > > interaction, where LOGIN needs two. The PLAIN mechanism also > > provides support for having an authorization id different from the > > authentication id, allowing for master user login for example. > > Thanks to both of you. Can I bet that Outlook doesn't support > anything but plain? > > I'm not sure I've ever heard of a client supporting other than > Evolution supporting MD5 passwords.. You would lose the bet. In addition, Claws-Mail supports CRAM-MD5. -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From taeuber at bbaw.de Fri Nov 4 13:04:22 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 12:04:22 +0100 Subject: [Dovecot] doveadm altmove -r Message-ID: <20111104120422.85c93004.taeuber@bbaw.de> Hi there, there is the ubuntu version 2.0.13 ov dovecot installed on our server. But in contrast to http://wiki2.dovecot.org/Tools/Doveadm/Altmove the option -r isn't supported in this version. altmove: invalid option -- 'r' doveadm altmove [-u |-A] [-S ] In which version of dovecot is this option supported? Thanks Lars From taeuber at bbaw.de Fri Nov 4 13:57:31 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 12:57:31 +0100 Subject: [Dovecot] doveadm altmove -r In-Reply-To: <20111104120422.85c93004.taeuber@bbaw.de> References: <20111104120422.85c93004.taeuber@bbaw.de> Message-ID: <20111104125731.5f05d91f.taeuber@bbaw.de> Ok, I've found this option to be introduce in version 2.0.14. Maybe someone (with the permissions) should add this info to the wiki page. Best wishes Lars Am Fri, 4 Nov 2011 12:04:22 +0100 Lars T?uber schrieb: > Hi there, > > there is the ubuntu version 2.0.13 ov dovecot installed on our server. > But in contrast to http://wiki2.dovecot.org/Tools/Doveadm/Altmove > the option -r isn't supported in this version. > > altmove: invalid option -- 'r' > doveadm altmove [-u |-A] [-S ] > > In which version of dovecot is this option supported? > > Thanks > Lars From fbscarel at gmail.com Fri Nov 4 13:57:45 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Fri, 4 Nov 2011 09:57:45 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB34AE9.6070002@hardwarefreak.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> <4EB34AE9.6070002@hardwarefreak.com> Message-ID: I'm using the GIT version, that 0.5 version is quite a bit outdated. I was not all that worried about using ZFS on this experiment because we do have the old mail storage on ext3 synchronized and ready to switch back, and I could disable dedup and compression on-the-fly if needed (which eventually was). On Fri, Nov 4, 2011 at 00:16, Stan Hoeppner wrote: > On 11/3/2011 1:24 PM, Felipe Scarel wrote: > > Reasons to choose ZFS were snapshots, and mainly dedup and compression > > capabilities. I know, it's ironic since I'm not able to use them now due > to > > severe performance issues with them (mostly dedup) turned on. > > > > I do like the emphasis on data integrity and fast on-the-fly > > configurability of ZFS to an extent, but I wouldn't recommend it highly > for > > new users, especially for production. It works (in fact it's working > right > > now), but has its fair share of troubles. > > > > We've started implementations to move our mail system to a more modular > > enviroment and we'll probably move away from ZFS. Was a nice experiment > > nonetheless, I learned quite a bit from it. > > I find this all very interesting... > > "Please keep in mind the current 0.5.2 stable release does not yet > support a mountable filesystem. This functionality is currently > available only in the 0.6.0-rc6 release candidate." > > https://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz > > "Uploaded October 14, 2011" > > So in the past ~two weeks, you converted your 15K+ user production > server to ZFS on Linux, as an experiment, and have now decided to change > to another filesystem solution, a mere two weeks later? Or am I > misinterpreting the date given that 0.6.0-rc6 was released? > > -- > Stan > From duihi77 at gmail.com Fri Nov 4 14:17:56 2011 From: duihi77 at gmail.com (Duane Hill) Date: Fri, 4 Nov 2011 07:17:56 -0500 (CDT) Subject: [Dovecot] doveadm altmove -r In-Reply-To: <20111104125731.5f05d91f.taeuber@bbaw.de> References: <20111104120422.85c93004.taeuber@bbaw.de> <20111104125731.5f05d91f.taeuber@bbaw.de> Message-ID: On Fri, 4 Nov 2011, Lars T?uber wrote: > Ok, I've found this option to be introduce in version 2.0.14. > Maybe someone (with the permissions) should add this info to the wiki page. It's in the release notes for v2.0.15 (http://www.dovecot.org/doc/NEWS-2.0): v2.0.15 2011-09-16 Timo Sirainen + doveadm altmove: Added -r parameter to move mails back to primary storage. ... From taeuber at bbaw.de Fri Nov 4 14:41:08 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 13:41:08 +0100 Subject: [Dovecot] doveadm altmove -r In-Reply-To: References: <20111104120422.85c93004.taeuber@bbaw.de> <20111104125731.5f05d91f.taeuber@bbaw.de> Message-ID: <20111104134108.efa5c27e.taeuber@bbaw.de> Am Fri, 4 Nov 2011 07:17:56 -0500 (CDT) Duane Hill schrieb: > On Fri, 4 Nov 2011, Lars T?uber wrote: > > > Ok, I've found this option to be introduce in version 2.0.14. > > Maybe someone (with the permissions) should add this info to the wiki > > page. > > It's in the release notes for v2.0.15 (http://www.dovecot.org/doc/NEWS-2.0): > > v2.0.15 2011-09-16 Timo Sirainen > > + doveadm altmove: Added -r parameter to move mails back to > primary storage. ... Oh yes this was a typo. Lars From leo at strike.wu.ac.at Fri Nov 4 19:36:16 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Fri, 04 Nov 2011 18:36:16 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EB0529E.2090808@strike.wu.ac.at> References: <4EAED444.70608@strike.wu.ac.at> <4EAEDDDD.40206@gmx.de> <4EB0529E.2090808@strike.wu.ac.at> Message-ID: <4EB42290.8060806@strike.wu.ac.at> On 11/01/2011 09:12 PM, Alexander 'Leo' Bergolth wrote: > On 10/31/2011 06:41 PM, e-frog wrote: >> On 31.10.2011 18:00, wrote Alexander 'Leo' Bergolth: >>> Have environment based namespace settings been abandoned in 2.0? >>> >> I think you need to add them to USERDB_KEYS now... >> >> http://wiki2.dovecot.org/PostLoginScripting#Running_environment > > I have added NAMESPACE_1 NAMESPACE_1_SEP NAMESPACE_1_TYPE > NAMESPACE_1_LIST NAMESPACE_1_PREFIX to USERDB_KEYS. > > Unfortunately the namespace still isn't used. :-( Can anyone confirm that setting namespaces via environment isn't supported in 2.0 anymore? Is there another way to dynamically add namespaces? Thanks, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From apm at one.com Fri Nov 4 21:23:36 2011 From: apm at one.com (Peter Mogensen) Date: Fri, 04 Nov 2011 20:23:36 +0100 Subject: [Dovecot] Corrupted transaction log file Message-ID: <4EB43BB8.6090803@one.com> Hi, I'm experiencing a problem I need some pointers to debug. I'm running Dovecot 2.0.15 and have a client which keeps causing log-entries like: Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: indexid changed 1320419300 -> 1320419441 (sync_offset=0) Nov 4 15:10:42 mail dovecot: imap(test at aaaone.net): Error: Corrupted transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: Invalid transaction log size (4476 vs 4360): /mail/3340444/. TestMails/dovecot.index.log (sync_offset=4476) Nov 4 15:10:42 mail dovecot: imap(test at aaaone.net): Disconnected: IMAP session state is inconsistent, please relogin. bytes=549/100474 The client issues a lot of IMAP connections like this: A1_0 LOGIN "test at aaaone.net" "password" A1_1 ENABLE QRESYNC CONDSTORE * OK Waiting for authentication process to respond.. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. A1_0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in * ENABLED QRESYNC CONDSTORE A1_1 OK Enabled. A2_0 CREATE "INBOX.folder" A2_0 OK Create completed. A3_0 SUBSCRIBE "INBOX.folder" A3_0 OK Subscribe completed. A4_0 APPEND "INBOX.folder" ($CreatedByImapClient-test) {20487+} ... more similar appends. But it only seems to be reproducible, when there's another client involved. Starting up Thunderbird with a login to the account seems to be enough to cause this. (To my knowledge this causes a series of STATUS command and then IDLE). The installation is using Maildir and lock_method = fcntl on local disks. /Peter From tss at iki.fi Fri Nov 4 21:30:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:30:29 +0200 Subject: [Dovecot] Auth Worker failures In-Reply-To: References: <1318948640.21919.103.camel@hurina> Message-ID: <1320435030.21919.142.camel@hurina> On Thu, 2011-10-20 at 16:24 -0400, Simon Brereton wrote: > Oct 20 06:25:14 mail dovecot: auth-worker(default): > sql(simon at example.net,127.0.0.1): CRYPT() != 'RaNDomsTRinG' CRYPT() means that it attempted to log in with an empty password. > I'm still sure this is the webmail trying to log in though.. Yeah, could be. But why it tries it with an empty password, I've no idea. From tss at iki.fi Fri Nov 4 21:34:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:34:03 +0200 Subject: [Dovecot] Accessing a strange mailbox In-Reply-To: <8762jie6ut.fsf@algae.riseup.net> References: <8762jie6ut.fsf@algae.riseup.net> Message-ID: <1320435243.21919.144.camel@hurina> On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote: > I have a user who has a mailbox called: > > A----->B > > It seemed to work in courier fine, they managed to create it, and there > are mails in it. However, dovecot is not letting the user access it, the > IMAP server gives an error. I tried to rename it but I would also get an > error: > > # doveadm mailbox rename 'A----->B' AtoB > doveadm(root): Error: Can't rename mailbox A----->B to AtoB: Mailbox doesn't exist: A----->B Maybe it's not named that? What does it look like in filesystem? Worked fine with me: # doveadm mailbox list 'A*' A----->B # doveadm mailbox rename 'A----->B' AtoB # doveadm mailbox list 'A*' AtoB From tss at iki.fi Fri Nov 4 21:38:27 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:38:27 +0200 Subject: [Dovecot] Getting the SMTP envelope through IMAP? In-Reply-To: <4EA28848.6070303@msgid.danisch.de> References: <4EA19119.4020803@msgid.danisch.de> <20111021180241.GA6924@PC211.ikt.de> <4EA1EB82.10807@msgid.danisch.de> <20111022081510.GA26132@laptop-dg.leere.eu> <4EA28848.6070303@msgid.danisch.de> Message-ID: <1320435508.21919.147.camel@hurina> On Sat, 2011-10-22 at 11:09 +0200, Hadmut Danisch wrote: > And neither the Return-Path nor the Received-Lines reveal the precise > date of delivery as the From line contains. The delivery date in From_ line is available in IMAP INTERNALDATE. The address part isn't available without patching Dovecot (mail_get_special(mail, MAIL_FETCH_FROM_ENVELOPE, &value)) From tss at iki.fi Fri Nov 4 21:43:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:43:31 +0200 Subject: [Dovecot] Dovecot crashes totally In-Reply-To: <4EA317B5.3090209@uni-greifswald.de> References: <4EA317B5.3090209@uni-greifswald.de> Message-ID: <1320435812.21919.150.camel@hurina> On Sat, 2011-10-22 at 21:21 +0200, Gordon Grubert wrote: > Hello, > > our dovecot server crashes totally without any really useful > log messages. The error log can be found in the attachment. > The only way to get dovecot running again is a complete > system restart. How often does it break? If really a "complete system restart" is needed to fix it, it doesn't sound like a Dovecot problem. Check if it's enough to stop dovecot and then make sure there aren't any dovecot processes lying around afterwards. > Oct 11 09:55:23 mailserver2 dovecot: master: Error: service(imap): > Initial status notification not received in 30 seconds, killing the > process > Oct 11 09:56:23 mailserver2 dovecot: imap-login: Error: master(imap): > Auth request timed out (received 0/12 bytes) Kind of looks like auth process is hanging. You could see if stracing it shows anything useful. Also are any errors logged about LDAP? Is LDAP running on the same server? From tss at iki.fi Fri Nov 4 21:47:39 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:47:39 +0200 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <4EA2C213.7030204@mike2k.de> References: <4EA2C213.7030204@mike2k.de> Message-ID: <1320436059.21919.152.camel@hurina> On Sat, 2011-10-22 at 15:16 +0200, Michael Stilkerich wrote: > dovecot: imap(michael): Error: > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file > or directory This is some weird filesystem? fstat() isn't supposed to ever fail with ENOENT. > Another thing that irritates me is that dovecot seems to use the dotlock > locking method, although I explicitly set lock_method to > fcntl (also tried flock, same behavior). The lock_method setting only affects index files. There are several others that always use dotlocks, because they're rarely enough modified that the dotlock performance doesn't matter. That's also why I haven't bothered to set use_excl_lock for them. From tss at iki.fi Fri Nov 4 21:50:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:50:13 +0200 Subject: [Dovecot] iterate_query not working In-Reply-To: <4EA5BBC8.5090901@netexpert.pl> References: <4EA5BBC8.5090901@netexpert.pl> Message-ID: <1320436213.21919.153.camel@hurina> On Mon, 2011-10-24 at 21:26 +0200, Piotr NetExpert wrote: > Hi, > > I have got iterate_query defined in dovecot-sql.conf.ext. However when I > run a command like: > doveadm search -A BEFORE 10d > I receive an error message (Failed to iterate ...) > > There is an error message in a log file too: > Error: Trying to iterate users, but userdbs don't support it dovecot -n output? Maybe you're using only passdb sql, but userdb static? Maybe you have more than one userdb? From piotr-l at netexpert.pl Fri Nov 4 21:53:41 2011 From: piotr-l at netexpert.pl (Piotr NetExpert) Date: Fri, 04 Nov 2011 20:53:41 +0100 Subject: [Dovecot] iterate_query not working In-Reply-To: <1320436213.21919.153.camel@hurina> References: <4EA5BBC8.5090901@netexpert.pl> <1320436213.21919.153.camel@hurina> Message-ID: <4EB442C5.9000709@netexpert.pl> Thanks for a reply. In the meantime I have found the problem. Userdb was static as you say. I believe docs do not tell that iterate query is based on user query. -- Tre?? oryginalnej wiadomo?ci -- > On Mon, 2011-10-24 at 21:26 +0200, Piotr NetExpert wrote: >> Hi, >> >> I have got iterate_query defined in dovecot-sql.conf.ext. However when I >> run a command like: >> doveadm search -A BEFORE 10d >> I receive an error message (Failed to iterate ...) >> >> There is an error message in a log file too: >> Error: Trying to iterate users, but userdbs don't support it > > dovecot -n output? Maybe you're using only passdb sql, but userdb > static? Maybe you have more than one userdb? > > -- pozdrawiam Piotr Szafarczyk http://www.netexpert.pl From tss at iki.fi Fri Nov 4 21:55:00 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:55:00 +0200 Subject: [Dovecot] Expire plugin not executing for sieve plugin? In-Reply-To: <4EA5E5DF.1050508@rename-it.nl> References: <4EA03F1C.5010500@druber.com> <4EA5A5E1.3000202@druber.com> <4EA5E5DF.1050508@rename-it.nl> Message-ID: <1320436501.21919.155.camel@hurina> On Tue, 2011-10-25 at 00:25 +0200, Stephan Bosch wrote: > On 10/24/2011 7:52 PM, Dan Swartzendruber wrote: > > > > No ideas? I googled some more and eventually turned up this posting: > > > > http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-01/msg00716.html > > > > > > guy seems to be having the exact problem I am, unfortunately, no-one > > replied to his post either :( > > Hmm, wonder why no one answered that one. In theory, those plugins > should interact gracefully. Right. There shouldn't be anything special with expire plugin and Sieve. > Running dovecot 1.2.12 on ubuntu 10.10. This very likely works fine with v2.0 though, and I don't really have time to debug noncritical v1.x bugs anymore. From tss at iki.fi Fri Nov 4 22:01:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:01:23 +0200 Subject: [Dovecot] SSL renegotiation vulnerability In-Reply-To: References: <23D958A8-91DE-42EA-BDC5-B0E2050EEC9F@iki.fi> <90232072-D5CF-42E7-8B5A-3DFBF75C6B3E@iki.fi> <7CAD9FBB-73BF-4D68-939B-1055AFED4C79@iki.fi> <3E17898B-B716-413F-9582-619B472D7C2B@iki.fi> Message-ID: <1320436883.21919.158.camel@hurina> http://vincent.bernat.im/en/blog/2011-ssl-dos-mitigation.html -> "Things get worse" shows that it's easier to DoS the server with multiple connections than with renegotiations, so I don't know if there's much point in disabling renegotiations. Perhaps Dovecot could allow e.g. one renegotiation per minute, but is that really worth the trouble?.. Perhaps there even are some clients that do renegotiations and Dovecot would break them. From tss at iki.fi Fri Nov 4 22:02:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:02:46 +0200 Subject: [Dovecot] GSSAPI and deny=yes passdb In-Reply-To: <4E8ACB70.7060004@telros.ru> References: <4E8ACB70.7060004@telros.ru> Message-ID: <1320436966.21919.160.camel@hurina> On Tue, 2011-10-04 at 13:01 +0400, Sergey Urushkin wrote: > Hi. Is it possible to use GSSAPI authentication and deny passdb > together? Seems it doesn't work as I expect: GSSAPI doesn't check deny > passdb, so I'm not able to restrict access to GSSAPI-users. GSSAPI doesn't use any passdbs, so it doesn't work. It would be nice to change this some day (so e.g. proxying would also be possible with GSSAPI auth), but for now it's a rather low priority for me. From tss at iki.fi Fri Nov 4 22:04:54 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:04:54 +0200 Subject: [Dovecot] POP server migration and duplicate messages. In-Reply-To: References: Message-ID: <1320437094.21919.162.camel@hurina> On Thu, 2011-10-27 at 12:16 -0700, Ernie Dunbar wrote: > From a > random sampling, the UIDL of the messages checks out between both servers, Completely identical UIDL output with the same order of messages? If so, there's nothing you can do about it from Dovecot's side. > so I really don't understand what's going on here. If this is just a bug > in Evolution, then that means I can just go ahead with the full migration > and hope that people aren't going to be stuck with re-downloading their > entire mailbox which in some cases, means hundreds or thousands of > messages. Some clients redownload mails if you change the hostname. But I don't know why Evolution would redownload only some of the messages but not all. From tss at iki.fi Fri Nov 4 22:08:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:08:59 +0200 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> Message-ID: <1320437339.21919.165.camel@hurina> On Fri, 2011-10-28 at 00:55 +0200, Marco Carcano wrote: > I'm struggling against what seems a really serious trouble: sometimes > mail get lost > > I did all the possible checks, but they seems really lost - I know is > absurd, but it seems so > > it has been noticed by two users on my server: intially I tought that > was their mistake (mail deleted, mail into spam, ecc), but it wasn't Enable mail_log plugin to make sure of this. http://wiki2.dovecot.org/Plugins/MailLog > Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid= >: saved mail to INBOX If Dovecot logs this, then the message definitely was saved to INBOX. > maybe is there some kind of bug in the maildrop version I use? by the > way, I had this issue also before installing amavisd-new Maildrop? From tss at iki.fi Fri Nov 4 22:19:43 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:19:43 +0200 Subject: [Dovecot] doveadm 'proxy list' or 'who' not working ? In-Reply-To: <7A7D318C-0BA3-4FDB-AB7F-F36A540C8472@aquaray.com> References: <7A7D318C-0BA3-4FDB-AB7F-F36A540C8472@aquaray.com> Message-ID: <1320437984.21919.170.camel@hurina> On Fri, 2011-10-28 at 11:49 +0200, de Lafond Guillaume wrote: > Hi, > > I switched from dovecot 1.2.* to 2.0.* for a mail proxy. > I try to play with the new cool utility doveadm, but I don't manage to get results from it. > > # /usr/local/dovecot-2.0.15/bin/doveadm proxy list > username proto src ip dest ip port > # This should work. A quick test shows it working with me. Check if you have "ipc" process running? Check that you have /login/ipc-proxy fifo. Maybe try straceing the imap-login processes and see if any ipc connection arrives there when running the doveadm command. > As 'proxy list' does not return anything, I tried with 'who' > # /usr/local/dovecot-2.0.15/bin/doveadm who -a /var/run/dovecot-socket-auth/anvil > username # proto (pids) (ips) > # This isn't supposed to show anything. Although why do you give such a socket path parameter? From tss at iki.fi Fri Nov 4 22:22:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:22:31 +0200 Subject: [Dovecot] What owner/permissions for virtual homes? In-Reply-To: References: Message-ID: <1320438151.21919.172.camel@hurina> On Fri, 2011-10-28 at 16:04 -0400, sean darcy wrote: > On 10/28/2011 03:34 PM, sean darcy wrote: > > VirtualUsers.Home.txt: > > > > The directory layouts for home and mail directories could look like one of > > these (in the preferred order): > > > > 1. Mail directory under home, for example: home='/var/vmail/domain/user/' > > mail='/var/vmail/domain/user/mail/' > > > > Who should be the owner of these directories, with that permissions? Typically 0700 permissions, owned by the "mail user", whichever you've configured for that. http://wiki2.dovecot.org/UserIds > > Does it make a difference if it's /home/vmail rather than /var/vmail? Either one is fine. > it suggests the following configuration: > > mail_home = /var/vmail/%d/%n > mail_location = maildir:~/mail > > but where does mail_home appear? Also in 10-mail.conf? Doesn't matter. For example there. From tss at iki.fi Fri Nov 4 22:28:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:28:28 +0200 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111025110238.GB8900@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> Message-ID: <1320438508.21919.177.camel@hurina> On Tue, 2011-10-25 at 13:02 +0200, Edgar Fu? wrote: > We have two dovecot 1.2 instances sharing Maildirs on NFS. Indexes are local to the individual servers. > Occasionally (no idea how to trigger this), the Seen flag gets lost on some messages. I've verified that actually the ``S'' is missing from the filename. > I suspect something like server A caching the flags, server B setting Seen, and then server A flushing its cache for another change so overwriting what B changed. That shouldn't happen because of Dovecot's indexes. Dovecot doesn't force Maildir to match its indexes. Whenever doing flag changes, it stores them as "add \Seen flag to UID 1234". Then the 'S' flag is added to the current Maildir filename without losing any other changes. From ms at mike2k.de Fri Nov 4 22:29:16 2011 From: ms at mike2k.de (Michael Stilkerich) Date: Fri, 04 Nov 2011 21:29:16 +0100 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320436059.21919.152.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> Message-ID: <4EB44B1C.90505@mike2k.de> On 04.11.2011 20:47, Timo Sirainen wrote: >> dovecot: imap(michael): Error: >> > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file >> > or directory > This is some weird filesystem? fstat() isn't supposed to ever fail with > ENOENT. Nope, plain ext3. I have these issues since upgrading to Ubuntu 11.10, which was a dovecot upgrade from 1.2.15 to 2.0.13. Currently I use a self-built 2.0.15 dovecot. I'm still getting errors once in a while for the dovecot.index.log file: Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking failed for file /home/dovecot/isa/dovecot.index.log: No such file or directory Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with file /home/dovecot/isa/dovecot.index.log: No such file or directory These errors occur regardless of the lock_method used. And occasionally I also get errors like: Nov 3 19:03:34 keira dovecot: imap(mikey): Error: Corrupted transaction log file /home/dovecot/mikey/dovecot.index.log seq 15: offset points outside file (32708 + 8 > 32708) (sync_offset=32800) I believe the latter was recently reported by someone else on this list as well. The files are there, though I guess that dovecot recreates them when the errors occurs so I don't know what the situation was by the time the error occurred... -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4480 bytes Desc: S/MIME Cryptographic Signature URL: From tss at iki.fi Fri Nov 4 22:30:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:30:13 +0200 Subject: [Dovecot] post login script In-Reply-To: References: Message-ID: <1320438617.21919.179.camel@hurina> On Fri, 2011-10-28 at 12:49 +0200, Osvaldo Alvarez Pozo wrote: > Hello all, > > I am testing postlogin script with dovecot 2. > > I works but i would like that the imap and pop session get executed > even if the post-login script fail. How/why would the script fail? > #/usr/local/bin/postlogin.sh > echo "UPDATE mailbox SET last_login = now() WHERE username = '$USER'" > | mysql -upostlogin -pXXXXXXXX postfixadmin > exec "$@" Looks like it would always exec $@ even if mysql update fails. From tss at iki.fi Fri Nov 4 22:33:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:33:09 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: <1320438789.21919.181.camel@hurina> On Tue, 2011-11-01 at 09:38 +0100, Lukas Wei? wrote: > >> and sql storage is difficult > to backup. Ive done that, by setting up dovecot with maildir format and > copy mails from old dbmail to new dovecot imap server with my mail > client (thunderbird). > > Thunderbird probably changes the INTERNALDATE of > the messages. Copy the mails some other way, like offlineimap or maybe > some other IMAP client. > > What is INTERNALDATE? The modification > timestamp is correct ("ls -l"), and the "Date: ... " line, too. Maildir file's mtime is the INTERNALDATE, yes. Maybe the the problem isn't then any timestamps, but that the messages were copied in reverse order (newest message first), so their IMAP UIDs are reversed. From tss at iki.fi Fri Nov 4 22:38:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:38:08 +0200 Subject: [Dovecot] LDAP w/SASL "Active Directory" authentication failing. In-Reply-To: References: Message-ID: <1320439089.21919.184.camel@hurina> On Tue, 2011-11-01 at 09:55 -0600, David Varela wrote: > I am running a Dovecot server (version 1.2.17) on FreeBSD 8.2, using > LDAP to authenticate Active Directory users. I can successfully bind and > authenticate using PLAIN and LDAP without SASL, but obviously passwords for > the bind user and the user being authenticated are being passed in plain > text. I've attempted to configure my server to us SASL however when I > attempt to authenticate a user I see authentication failures. I reviewed > the security log on my domain controller and see that the bind user is > binding properly, so the issue appears to be orginating from the user > authentication, however I cannot determine what the issue is. Here is all > the information regarding my configuration, along with the logs from the > server: SASL binding currently works only for the initial "ldap admin user" authentication. It doesn't work for individual user authentication requests (auth_bind=yes). > #auth_bind = yes Here you're not even attempting to use auth binds. > pass_attrs = mail=user And you're also not returning a password for user. > Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): > No password returned (and no nopassword) So Dovecot has no way of authenticating user. I'd suggest forgetting about SASL and enabling TLS instead. From tss at iki.fi Fri Nov 4 22:41:26 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:41:26 +0200 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <20111102085715.GA4089@dibs.tanso.net> References: <20111102085715.GA4089@dibs.tanso.net> Message-ID: <1320439286.21919.187.camel@hurina> On Wed, 2011-11-02 at 09:57 +0100, Jan-Frode Myklebust wrote: > IMHO it would be great if we could both provision and un-provision users > without having the provisioning system mess with files in the > filesystem. Provisioning new users works fine since dovecot will create > all files/directories on first login, but I haven?t found the tools to > un-provison them. I see "doveadm mailbox delete" can delete individual > mailboxes, but what about indexes, sieve-scripts and home-directories? > Are there any tools to clean out these ? Do you have all of those files in separate directories? Normally I'd think everything is under home dir, so you can just: rm -rf `doveadm user -f home $username` For deleting indexes and other dirs that are not under home.. well.. I guess you'll need some more scripting. Anyway, I don't think there's any good and safe way to just go delete user's directories. Especially home dir. From tss at iki.fi Fri Nov 4 22:44:30 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:44:30 +0200 Subject: [Dovecot] Different default realm per listener In-Reply-To: <4EB15375.30106@team.de.inter.net> References: <4EB15375.30106@team.de.inter.net> Message-ID: <1320439470.21919.190.camel@hurina> On Wed, 2011-11-02 at 15:28 +0100, Kai Arif wrote: > Hi all! > > I want create a large IMAP server setup where dovecot is listening on multiple interfaces and needs > a different default realm on each interface. I'm using dovecot 2.0.5 on FreeBSD9 and authenticate > against a LDAP database. > I found an old mail thread from 2007 that said I'm basically screwed. Has that change since than? Is > there any way to use different default realms per listener? v2.0 can almost do it: local 1.2.3.4 { auth_default_realm = foo.org } local 1.2.3.5 { auth_default_realm = bar.org } Unfortunately that will currently fail, because per-IP auth settings don't work.. It will hopefully be fixed some day. From tss at iki.fi Fri Nov 4 22:47:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:47:57 +0200 Subject: [Dovecot] IMP, Dovecot and multiple namespaces In-Reply-To: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> References: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> Message-ID: <1320439677.21919.192.camel@hurina> On Wed, 2011-11-02 at 09:37 -0500, Rick Romero wrote: > Hey guys, > > I'm wondering what the best way is to be backwards compatible with > Courier mailbox formats and not duplicate mailbox trees with Dovecot. > Is anyone doing this right now? > > My Dovecot 2.0.13 is configured as follows for Namespaces: > > namespace { > inbox = yes > location = > prefix = > separator = . Perhaps use '/' separator. > } > namespace { > hidden = yes > inbox = no > list = no > location = > prefix = INBOX. > separator = . Add "alias_for = " here. I also added it to wiki2. > } > > So I should have an INBOX, plus additional root folders - rather than > the Courier default of all folders being subfolders of the INBOX. > > So when I run LIST in IMAP I get - > 08 LIST "" % > * LIST (\HasChildren) "." "INBOX" INBOX.% now matches both INBOX. namespace's mailboxes, and also the INBOX mailbox's children. If you change the separator to '/', there's a difference between INBOX.% (compatibility namespace) and INBOX/% (INBOX's children). From tss at iki.fi Fri Nov 4 22:59:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:59:04 +0200 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103095424.67160@gmx.com> References: <20111103095424.67160@gmx.com> Message-ID: <1320440344.21919.196.camel@hurina> On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the filesystem operations take more than a second. From tss at iki.fi Fri Nov 4 23:03:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:03:25 +0200 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: References: Message-ID: <1320440605.21919.198.camel@hurina> On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes wrote: > I'm having a problem with dovecot ldap connection when ldap server is in > another firewall zone, firewall kills the ldap connection after a > determined period of inactivity, this is good from the firewall point of > view but is bad for dovecot because it never knows the connections has been > dropped, this creates longs timeouts in dovecot and finally it reconnects, > meanwhile many users fails to authenticate, I have seen this kind of post > in the list for a while but can't find a solution for it, so my question is > how to define a idle ldap time in dovecot so it can reconnect before the > firewall has dropped the connection or just close the connection under > inactivity so when a user authenticate doesn't fails for a while until > dovecot detects that the connection has hanged. Is this a feature request > or there is already a configuration for this??? Can't the firewall be changed to reject the LDAP packets instead of dropping them? Then Dovecot would immediately notice that the connection has died, and with a recent enough version it wouldn't even log an error about it. I guess some kind of an "ldap_idle_disconnect = 30s" setting could be added, but it's not a very high priority for me. From tss at iki.fi Fri Nov 4 23:09:49 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:09:49 +0200 Subject: [Dovecot] Restricting IMAP access In-Reply-To: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> References: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Message-ID: <1320440989.21919.202.camel@hurina> On Thu, 2011-11-03 at 19:13 +0100, Thierry de Montaudry wrote: > Hi list, > > I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. > I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). > Anybody has a trick for that? Running dovecot 2.0.13. > > I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. So assuming all users can log in from 127.0.0.1 and domain.com only elsewhere, you can do stuff like: a) Change password_query so that it succeeds only if '%l' = '127.0.0.1' or '%d' = 'domain.com' b) Add a deny=yes passdb with e.g. sqlite backend that basically does the same as a) but reversed. http://wiki2.dovecot.org/Authentication/RestrictAccess has passwd-file example, example config files should have also something. c) Add a new Dovecot proxy facing the public internet where it lets in only those users with '%d'='domain.com' (again maybe sqlite) Possibly other ways too.. From janfrode at tanso.net Fri Nov 4 23:19:31 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 4 Nov 2011 22:19:31 +0100 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <1320439286.21919.187.camel@hurina> References: <20111102085715.GA4089@dibs.tanso.net> <1320439286.21919.187.camel@hurina> Message-ID: <20111104211931.GA5430@dibs.tanso.net> On Fri, Nov 04, 2011 at 10:41:26PM +0200, Timo Sirainen wrote: > On Wed, 2011-11-02 at 09:57 +0100, Jan-Frode Myklebust wrote: > > IMHO it would be great if we could both provision and un-provision users > > without having the provisioning system mess with files in the > > filesystem. Provisioning new users works fine since dovecot will create > > all files/directories on first login, but I haven?t found the tools to > > un-provison them. I see "doveadm mailbox delete" can delete individual > > mailboxes, but what about indexes, sieve-scripts and home-directories? > > Are there any tools to clean out these ? > > Do you have all of those files in separate directories? Normally I'd > think everything is under home dir, so you can just: Unfortunately, for historical reasons, yes. The users doesn't have a home-directory, or the inbox is directly in their home.. so I don't think sieve fits there, and indexes are kept elsewhere for performance reasons. > > rm -rf `doveadm user -f home $username` Ah, great. Looking up the home dir via doveadm helps. > For deleting indexes and other dirs that are not under home.. well.. I > guess you'll need some more scripting. What happens to the indexes when we delete the mailboxes via "doveadm mailbox delete" ? Will that clean up the indexes ? -jf From tss at iki.fi Fri Nov 4 23:18:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:18:56 +0200 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EAED444.70608@strike.wu.ac.at> References: <4EAED444.70608@strike.wu.ac.at> Message-ID: <1320441536.21919.206.camel@hurina> On Mon, 2011-10-31 at 18:00 +0100, Alexander 'Leo' Bergolth wrote: > NAMESPACE_2='maildir:/var/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep' > NAMESPACE_2_PREFIX=Shared/spamrep/ > NAMESPACE_2_SEP=/ > NAMESPACE_2_TYPE=public > NAMESPACE_2_LIST=yes The settings are now named. http://wiki2.dovecot.org/MailLocation#Custom_namespace_location shows the basics. So for example if you have: namespace foo { .. } You can change it via: NAMESPACE/FOO/PREFIX=Shared/spamrep/ NAMESPACE/FOO/SEP=/ ..etc.. The '/' character is a bit problematic with shells scripts though, see the wiki link how to do it via "env" at least. Another problem is if you want to add more namespaces than you have configured. IIRC it's possible to simply add: NAMESPACE=foo And more than one is probably either "foo bar" or "foo,bar". If you get something to work, please add example to http://wiki2.dovecot.org/PostLoginScripting :) From tss at iki.fi Fri Nov 4 23:26:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:26:29 +0200 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <4EB43BB8.6090803@one.com> References: <4EB43BB8.6090803@one.com> Message-ID: <1320441989.21919.212.camel@hurina> On Fri, 2011-11-04 at 20:23 +0100, Peter Mogensen wrote: > I'm running Dovecot 2.0.15 and have a client which keeps causing > log-entries like: > > > Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted > transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: > indexid changed 1320419300 -> 1320419441 (sync_offset=0) Session A had TestMails open and created with index file whose ID was 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 17:08:20 EET 2011). Session B came and recreated the index files 141 seconds later with ID 1320419441. Either it didn't see A's original index files for some reason or it simply decided to recreate them for some reason. Either way this shouldn't have happened. Session A then notices that the indexes were recreated, and logs an error. > But it only seems to be reproducible, when there's another client > involved. Starting up Thunderbird with a login to the account seems to > be enough to cause this. (To my knowledge this causes a series of STATUS > command and then IDLE). > > The installation is using Maildir and lock_method = fcntl on local disks. What filesystem? Have changed any service imap {} settings? Can you reproduce this by running stress test with imaptest? http://imapwiki.org/ImapTest For example: imaptest user=testuser pass=testpass host=localhost Or if it's really related to creating the initial files: while imaptest user=testuser pass=testpass host=localhost secs=1; do rm -rf /home/testuser/Maildir done From tss at iki.fi Fri Nov 4 23:27:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:27:46 +0200 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <20111104211931.GA5430@dibs.tanso.net> References: <20111102085715.GA4089@dibs.tanso.net> <1320439286.21919.187.camel@hurina> <20111104211931.GA5430@dibs.tanso.net> Message-ID: <1320442066.21919.213.camel@hurina> On Fri, 2011-11-04 at 22:19 +0100, Jan-Frode Myklebust wrote: > > For deleting indexes and other dirs that are not under home.. well.. I > > guess you'll need some more scripting. > > What happens to the indexes when we delete the mailboxes via "doveadm > mailbox delete" ? Will that clean up the indexes ? Yes. So you could create a script that lists all mailboxes, deletes them and then rmdirs the index root dir. From tss at iki.fi Fri Nov 4 23:39:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:39:18 +0200 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <4EB44B1C.90505@mike2k.de> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> Message-ID: <1320442758.21919.222.camel@hurina> On Fri, 2011-11-04 at 21:29 +0100, Michael Stilkerich wrote: > On 04.11.2011 20:47, Timo Sirainen wrote: > >> dovecot: imap(michael): Error: > >> > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file > >> > or directory > > This is some weird filesystem? fstat() isn't supposed to ever fail with > > ENOENT. > > Nope, plain ext3. I have these issues since upgrading to Ubuntu 11.10, > which was a dovecot upgrade from 1.2.15 to 2.0.13. Currently I use a > self-built 2.0.15 dovecot. I'm still getting errors once in a while for > the dovecot.index.log file: > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking > failed for file /home/dovecot/isa/dovecot.index.log: No such file or > directory > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with > file /home/dovecot/isa/dovecot.index.log: No such file or directory These simply shouldn't happen. I'd say it's a kernel bug. You're running a default Ubuntu kernel? I wonder if other Ubuntu users have this problem. > And occasionally I also get errors like: > > Nov 3 19:03:34 keira dovecot: imap(mikey): Error: Corrupted transaction > log file /home/dovecot/mikey/dovecot.index.log seq 15: offset points > outside file (32708 + 8 > 32708) (sync_offset=32800) > > I believe the latter was recently reported by someone else on this list > as well. The files are there, though I guess that dovecot recreates them > when the errors occurs so I don't know what the situation was by the > time the error occurred... Try if you can reproduce these more easily with imaptest? http://imapwiki.org/ImapTest Be sure to run it against a test account since it trashes the whole mailbox. From tss at iki.fi Fri Nov 4 23:47:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:47:10 +0200 Subject: [Dovecot] Rewrite the ostream output method, to save messages in another directory. In-Reply-To: References: Message-ID: <1320443230.21919.227.camel@hurina> On Mon, 2011-10-31 at 13:25 -0200, Alex Baule wrote: > struct ostream * > o_stream_create_X_list(struct ostream *output, struct istream *input, const > char *type) > { > struct emexis_ostream *X_stream; > struct ostream *new_output; > int fd; > > fd = open("/tmp/email",O_WRONLY); > > new_output = o_stream_create_fd_file(fd, 0, TRUE); > o_stream_cork(new_output); > > emexis_stream = i_new(struct emexis_ostream, 1); > emexis_stream->ostream.sendv = o_stream_X_sendv; > emexis_stream->ostream.flush = o_stream_X_flush; > emexis_stream->ostream.iostream.close = o_stream_X_close; > emexis_stream->output = new_output; > emexis_stream->input = input; > > o_stream_ref(new_output); > > return o_stream_create(&X_stream->ostream); > } This part of the code looks ok. > With this new o_stream, i got this error. > > Oct 27 15:14:24 brc dovecot: imap(alex at exemplo.com.br): Error: > o_stream_send_istream(/storage/gss/emexis/messages/ > exemplo.com.br/alex/Maildir/.Emexis.Ext-Blacklist/tmp/1319735664.M568409P30815.brc) > failed: Bad file descriptor I guess this should call your o_stream_X_sendv(). What do you do in that? I guess you're calling o_stream_sendv() for the new_output stream? Check if it returns -1, if it does then you're somehow breaking the new_output. Maybe you're closing it too early? Try what happens if you create it with: new_output = o_stream_create_fd_file(fd, 0, FALSE); Then it doesn't automatically close the fd, and it's at least clearer if the problem is because the stream is closed too early or if some completely wrong fd is attempted to be used. > There is something wrong Timo ? Like missing something or i need to rewrite > another funcion ? Nope. I think this is completely internal to your code. From apm at one.com Fri Nov 4 23:49:22 2011 From: apm at one.com (Peter Mogensen) Date: Fri, 04 Nov 2011 22:49:22 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EB45DE2.3030909@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: > What filesystem? XFS > Have changed any service imap {} settings? nope. But I have a special auth plugin. I doubt that's the reason though, since it's an entirely different process. Thanx, I'll try gather more info. /Peter From tss at iki.fi Fri Nov 4 23:49:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:49:28 +0200 Subject: [Dovecot] doveadm segfaults on TCP connect - version 2.0.15 In-Reply-To: <3316c960405142bf1d81f1db2b350739.squirrel@webmail.islandemail.com> References: <3316c960405142bf1d81f1db2b350739.squirrel@webmail.islandemail.com> Message-ID: <1320443369.21919.229.camel@hurina> On Wed, 2011-10-19 at 12:40 -0700, ghandidrivesahumvee at rocketfish.com wrote: > I just recently installed Dovecot 2.0.15. Unfortunately, doveadm segfaults > when I attempt to connect to the local dovecot instance. I fixed a lot of bugs related to this for v2.1. I'm not sure if I'll bother backporting them, since it can't work perfectly anyway without breaking other stuff.. Hopefully v2.1.0 won't be too far away now :) From tss at iki.fi Fri Nov 4 23:50:51 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:50:51 +0200 Subject: [Dovecot] Proxy and Virtual IMAP Folders In-Reply-To: References: Message-ID: <1320443452.21919.230.camel@hurina> On Wed, 2011-10-19 at 21:17 -0400, Pedro Paixao wrote: > Is is posible to use the virtual plugin, in a IMAP proxy configuration, in a > way that the virtual plugin searches messages on the real server? > > Here's my scenario: > > [Client]----[Dovecot Proxy]-----[Real Imap Server] > > Client connects to Dovecot Proxy and gets all the folders that are on the > Real Imap Server, plus some virtual ones (generated by Dovecot Proxy) that > search for messages on the real server. In my config the Dovecot Proxy has > no email messages, real folders, or local delivery. Yes, you should be able to do this with v2.1's imapc backend. From tss at iki.fi Fri Nov 4 23:52:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:52:13 +0200 Subject: [Dovecot] imap proxy for Gmail In-Reply-To: References: Message-ID: <1320443533.21919.231.camel@hurina> On Wed, 2011-10-19 at 23:31 -0400, Jumping Mouse wrote: > Hello, b > > I have a situation were I want to be able to give teachers access to their Google apps for education email through Squirrelmail. The network that they are on blocks gmail. I am able to get Squirrelmail to communicate with Gmail via imap. I am looking at getting persistant imap connections to improve the user experience. I have looked at IMAPPROXY but this application is not compatible with the gmail imaps (ssl-encrypted imap over port 993). I recall reading somewhere that Dovecot has a built in imapproxy that could be used for this. Dovecot doesn't provide persistent IMAP proxying. You should be able to combine imapproxy with stunnel to do this. From tss at iki.fi Fri Nov 4 23:56:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:56:04 +0200 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <4E9FEC22.8060105@webfusion.com> References: <4E9FEC22.8060105@webfusion.com> Message-ID: <1320443764.21919.233.camel@hurina> On Thu, 2011-10-20 at 12:38 +0300, Mark Zealey wrote: > I'm currently changing some systems to use the redirector service, which > means that to get local deliveries going I need to get lmtp set up so it > can be redirected as well. This is working fine, however we have a > number of different ports running on our servers depending on which > brand a customer is using. When the time comes to do the user look up, > we use (port, user, domain) to do a unique database lookup which works > fine for everything (proxied pop, imap, lmtp & straight pop, imap) but > not straight lmtp - perhaps because it doesn't have a 2-stage login > process unlike the pop/imap protocols. Looking at the SQL query we are > issuing, the %a port variable is set to 0 even though the connection is > coming in from TCP. Is there a way to change this? We are running > dovecot 2.0.12 however looking through the changelogs I can't see this > would be fixed in the newest version. A quick check in code shows that this is already supposed to work. Set auth_debug=yes and see if LMTP's auth lookup sends lip, rip, lport and rport fields to auth process? From tss at iki.fi Fri Nov 4 23:59:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:59:03 +0200 Subject: [Dovecot] lmtp performance In-Reply-To: <201110171642.45954.joseba.torre@ehu.es> References: <4E9BE1FB.6040902@upm.es> <4E9C3C25.7070802@upm.es> <201110171642.45954.joseba.torre@ehu.es> Message-ID: <1320443944.21919.235.camel@hurina> On Mon, 2011-10-17 at 16:42 +0200, Joseba Torre wrote: > > Add to service lmtp: > > process_min_avail = 20 > I had a similar problem, and found the same solution. In my setup, only 1 lmtp > process was available and a 2nd one never was created, so the process was > really busy. With this parameter (I setup it to 10) I have diferent process > and all of them are used. Hmm. I wonder if perhaps it would be better to set: service lmtp { client_limit = 1 } I think I'll make that the default.. From tss at iki.fi Sat Nov 5 00:07:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 00:07:59 +0200 Subject: [Dovecot] [PATCH] Use SSL_MODE_RELEASE_BUFFERS if available to keep memory usage low In-Reply-To: References: Message-ID: <1320444480.21919.236.camel@hurina> On Thu, 2011-10-13 at 16:23 -0300, Cristian Rodr?guez wrote: > Use SSL_MODE_RELEASE_BUFFERS if available to keep memory usage low. Hmm. Committed. Although I wonder how big of a difference it makes. And if it increases CPU usage much? From maria_arrea at gmx.com Sat Nov 5 01:57:11 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Sat, 05 Nov 2011 00:57:11 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111104235711.67190@gmx.com> Timo, we are not using NFS, we use remote iSCSI volumes with ext4. Regards Maria ----- Original Message ----- From: Timo Sirainen Sent: 11/04/11 09:59 PM To: Maria Arrea Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the filesystem operations take more than a second. From noah-list at enabled.com Sat Nov 5 02:32:09 2011 From: noah-list at enabled.com (Noah) Date: Fri, 04 Nov 2011 17:32:09 -0700 Subject: [Dovecot] dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 Message-ID: <4EB48409.90803@enabled.com> Hi there I keep getting the following errors scrolling in my dovecot logs. I am wondering how I can nail down what is creating that problem. dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 also I have a trash folder with over 100k messages in mbox format. I am wondering if migrating to Maildir format would make it easier and quicker for dovecot to index my mail. I am getting these types of errors from time to time. dovecot: 2011-10-30 20:32:58 Error: IMAP(): Timeout while waiting for lock for transaction log file /home//mail/.imap/INBOX/dovecot.index.log Cheers, Noah From stan at hardwarefreak.com Sat Nov 5 10:26:45 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 05 Nov 2011 03:26:45 -0500 Subject: [Dovecot] dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 In-Reply-To: <4EB48409.90803@enabled.com> References: <4EB48409.90803@enabled.com> Message-ID: <4EB4F345.80000@hardwarefreak.com> On 11/4/2011 7:32 PM, Noah wrote: > Hi there > > I keep getting the following errors scrolling in my dovecot logs. I am > wondering how I can nail down what is creating that problem. > > dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash > UID 42022 got too little data: 90900 vs 1529791 What version of Dovecot are you using? > also I have a trash folder with over 100k messages in mbox format. I am > wondering if migrating to Maildir format would make it easier and > quicker for dovecot to index my mail. As I answered on the Postfix list, the solution to this is ditching procmail for dovecot-lda so indexing of new messages is incremental upon delivery, instead of megalithic when your MUA connects to the mailbox. Switching to maildir in this case will simply trade on problem for another. Your memory consumption may go down, but your disk access will skyrocket. > I am getting these types of errors from time to time. > > dovecot: 2011-10-30 20:32:58 Error: IMAP(): Timeout while waiting > for lock for transaction log file > /home//mail/.imap/INBOX/dovecot.index.log It would be really helpful if you would simply post a big dump of your machine specs, software configuration, user base, mailbox sizes, total mail storage space, concurrent user load, etc. You've now posted a total of 3 seemingly different problems on two mailing lists, related to your Dovecot problems. I get the feeling that there's a much larger overall problem here, say something like hardware totally under powered for the workload you're throwing at it, or something to that affect. Or maybe just a really bad system setup. The more information you can provide, the more quickly and thoroughly we can provide a solution or salient advice. -- Stan From laxlaw at gmx.de Sat Nov 5 11:57:19 2011 From: laxlaw at gmx.de (=?UTF-8?Q?Lukas_Wei=C3=9F?=) Date: Sat, 05 Nov 2011 10:57:19 +0100 Subject: [Dovecot] =?utf-8?q?Bad_creation_timestamp_of_migrated_mails_=28m?= =?utf-8?q?aildir_format=29=3F?= In-Reply-To: <1320438789.21919.181.camel@hurina> References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> <1320438789.21919.181.camel@hurina> Message-ID: On Fri, 04 Nov 2011 22:33:09 +0200, Timo Sirainen wrote: > On Tue, 2011-11-01 at 09:38 +0100, Lukas Wei? wrote: >>>> and sql storage is difficult to backup. Ive done that, by setting up dovecot with maildir format and copy mails from old dbmail to new dovecot imap server with my mail client (thunderbird). >> >>> Thunderbird probably changes the INTERNALDATE of >> the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. What is INTERNALDATE? The modification timestamp is correct ("ls -l"), and the "Date: ... " line, too. > Maildir file's mtime is the INTERNALDATE, yes. Maybe the the problem isn't then any timestamps, but that the messages were copied in reverse order (newest message first), so their IMAP UIDs are reversed. Possible. How do i avoid this? What is the preffered way to migrate mails from one to another imap server? Offlineimap? Can i do a save sync with that, even if my dovecot has newer mails now? From lists at kokelnet.de Sat Nov 5 16:13:30 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 15:13:30 +0100 Subject: [Dovecot] Dovecot SASL on another machine Message-ID: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> Hello list, is there a way to use Dovecot SASL Authentication for submission on a different machine dovecot runs? e.g.: Dovecot machine <-> postfix machine(for submission) How can I authenticate my senders on a different physical machine against Dovecot SASL? Thanks, Tobias From tss at iki.fi Sat Nov 5 17:43:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 17:43:10 +0200 Subject: [Dovecot] fts-lucene vs SEARCH header Re: 2.1: problems compiling fts-lucene - CLucene git version necessary? In-Reply-To: References: <7C9B9C67-A1FD-472B-8514-BC54F1B01D00@iki.fi> <1316446540.12936.114.camel@hurina> <20110920071245.GA3617217@gabi.sernet.de> Message-ID: <1320507790.21919.237.camel@hurina> On Thu, 2011-09-22 at 13:24 +0200, Lutz Pre?ler wrote: > Next problem :-) side effect: my test setup (some commits ago) with fts = lucene > in plugin section (ftps_lucene plugin without stemmer support): > SEARCH header is not working (empty result sets, no logging). Fixed finally. From tss at iki.fi Sat Nov 5 18:01:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:01:47 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> Message-ID: <1320508907.21919.239.camel@hurina> On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > is there a way to use Dovecot SASL Authentication for submission on a > different machine dovecot runs? With v2.0 you should be able to add inet_listener for auth service, which acts as an auth-client socket. > Dovecot machine <-> postfix machine(for submission) Looking at Postfix v2.8.0 code, looks like Postfix also supports it. From tss at iki.fi Sat Nov 5 18:06:11 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:06:11 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> <1320438789.21919.181.camel@hurina> Message-ID: <1320509172.21919.241.camel@hurina> On Sat, 2011-11-05 at 10:57 +0100, Lukas Wei? wrote: > > Maildir file's mtime is the INTERNALDATE, yes. Maybe the the > problem isn't then any timestamps, but that the messages were copied in > reverse order (newest message first), so their IMAP UIDs are > reversed. > > Possible. How do i avoid this? What is the preffered way to > migrate mails from one to another imap server? Offlineimap? Can i do a > save sync with that, even if my dovecot has newer mails now? If you've already moved the mails, there's not much you can do.. Or if you don't care about losing IMAP UIDs (= clients redownloading mails), you could simply delete dovecot-uidlist and dovecot.index* files and let Dovecot reassign UIDs in the "correct" order. From tss at iki.fi Sat Nov 5 18:15:05 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:15:05 +0200 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111104235711.67190@gmx.com> References: <20111104235711.67190@gmx.com> Message-ID: <1320509705.21919.248.camel@hurina> Well, doesn't matter if it's NFS or not. It still looks as if Dovecot process was stuck for 45 seconds, most likely waiting for disk I/O to finish.. What happens is something like: 1. Get the current time ("now") 2. See if lock file exists 3. Create lock file 4. fstat() the created lock file 5. Log a warning if fstat's ctime differs from "now" more than 30 seconds. (Actually I think the 30 seconds threshold is way too generous, it should be less than 1 second usually.) So steps 2 and 3 took 45 seconds to finish. Basically I guess the disk I/O load was very high at that time, or alternatively there was some unintentional delay caused by iSCSI (kernel/network bug/problem). On Sat, 2011-11-05 at 00:57 +0100, Maria Arrea wrote: > Timo, we are not using NFS, we use remote iSCSI volumes with ext4. > > Regards > > Maria > > ----- Original Message ----- > From: Timo Sirainen > Sent: 11/04/11 09:59 PM > To: Maria Arrea > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the fil > esystem operations take more than a second. From tss at iki.fi Sat Nov 5 18:23:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:23:14 +0200 Subject: [Dovecot] Attachements with SIS : shauld all the same files share the same inode ? In-Reply-To: <4E9BD704.9080801@univ-nantes.fr> References: <4E9BD704.9080801@univ-nantes.fr> Message-ID: <1320510194.21919.251.camel@hurina> On Mon, 2011-10-17 at 09:19 +0200, Yann Dupont wrote: > 104 instance of -rw------- 104 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-ffd0d1xxxx > > The files are hardlinked, so it seems ok, but I also see 9 others files > in the same directory : > > > -rw------- 1 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-0519df2a94fc9a4e1f4100008a6b99b6 > -rw------- 1 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-1790b92994fc9a4e1a4100008a6b99b6 > > as you can see, same prefix, If I compare them with diff, they are all > the same files. I guess you have "mail_attachment_fs = sis posix" (default)? Sounds like there could be a bug where SIS does comparison of files but for some reason thinks they are different. You could try if you can merge them with: mkdir /tmp/queue touch /tmp/queue/6e2df299fa96daec9b4735d07c494046429be4d6-foo doveadm sis deduplicate /var/attachments /tmp/queue From lists at kokelnet.de Sat Nov 5 18:31:56 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 17:31:56 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320508907.21919.239.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> Message-ID: <91de83aeeb686e189fb47e454ce86a06@hachmer.de> Am 05.11.2011 17:01, schrieb Timo Sirainen: > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: >> is there a way to use Dovecot SASL Authentication for submission on >> a >> different machine dovecot runs? > > With v2.0 you should be able to add inet_listener for auth service, > which acts as an auth-client socket. Thanks for your reply. I think I didn't couch correct. I want a postfix machine to authenticate against Dovecot SASL for submission service which hasn't installed dovecot locally. The postfix machine has to ask my central dovecot which is at the moment dovecot v1.2.15 but will be soon v2. Is this possible, if yes, how? Greetz, Tobias From tss at iki.fi Sat Nov 5 18:35:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:35:01 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <91de83aeeb686e189fb47e454ce86a06@hachmer.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <91de83aeeb686e189fb47e454ce86a06@hachmer.de> Message-ID: <1320510902.21919.253.camel@hurina> On Sat, 2011-11-05 at 17:31 +0100, Tobias Hachmer wrote: > Am 05.11.2011 17:01, schrieb Timo Sirainen: > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > >> is there a way to use Dovecot SASL Authentication for submission on > >> a > >> different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > Thanks for your reply. I think I didn't couch correct. > I want a postfix machine to authenticate against Dovecot SASL for > submission service which hasn't installed dovecot locally. > The postfix machine has to ask my central dovecot which is at the > moment dovecot v1.2.15 but will be soon v2. Yes, this is how I understood it. > Is this possible, if yes, how? With v2.0: service auth { inet_listener { port = 12345 } } Postfix: smtpd_sasl_path = inet:dovecot.foobar.com:12345 From lists at kokelnet.de Sat Nov 5 18:39:50 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 17:39:50 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320510902.21919.253.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <91de83aeeb686e189fb47e454ce86a06@hachmer.de> <1320510902.21919.253.camel@hurina> Message-ID: <80202ce555c1e3ca63e1c1ee3e8cd870@hachmer.de> Am 05.11.2011 17:35, schrieb Timo Sirainen: > On Sat, 2011-11-05 at 17:31 +0100, Tobias Hachmer wrote: >> Am 05.11.2011 17:01, schrieb Timo Sirainen: >> > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: >> >> is there a way to use Dovecot SASL Authentication for submission >> on >> >> a >> >> different machine dovecot runs? >> > >> > With v2.0 you should be able to add inet_listener for auth >> service, >> > which acts as an auth-client socket. >> >> Thanks for your reply. I think I didn't couch correct. >> I want a postfix machine to authenticate against Dovecot SASL for >> submission service which hasn't installed dovecot locally. >> The postfix machine has to ask my central dovecot which is at the >> moment dovecot v1.2.15 but will be soon v2. > > Yes, this is how I understood it. OK, sorry. >> Is this possible, if yes, how? > > With v2.0: > > service auth { > inet_listener { > port = 12345 > } > } > > Postfix: > > smtpd_sasl_path = inet:dovecot.foobar.com:12345 Thanks a lot for your quick help!!! Greetz, Tobias From tss at iki.fi Sat Nov 5 18:44:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:44:06 +0200 Subject: [Dovecot] Group ACL In-Reply-To: References: Message-ID: <1320511448.21919.257.camel@hurina> On Thu, 2011-10-06 at 08:38 -0700, Daniel L. Miller wrote: > Using Dovecot 2.0, LDAP userdb & passdb, with prefetch: > > hosts = myhost.mydomain > dn = cn=x,cn=y > dnpass = xyz123 > auth_bind = yes > auth_bind_userdn = uid=%n,ou=users,dc=x > ldap_version = 3 > base = ou=Users, dc=x > user_attrs = =home=/var/mail/%d/%n, \ > =mail=mdbox:/var/mail/%d/%n/mdbox, \ > =uid=vmail, \ > =gid=mail Would be good to put acl_groups here also so non-prefetch stuff has access to it also. > user_filter = (&(objectClass=inetOrgPerson)(mail=%u)) > > pass_attrs = mail=user, \ > userPassword=password, \ > =userdb_home=/var/mail/%d/%n, \ > =userdb_mail=mdbox:/var/mail/%d/%n/mdbox, \ > =userdb_uid=vmail, \ > =userdb_gid=mail, \ > =userdb_acl_groups='allshared' I think the problem may simply be those '' characters in there. You're now in "'allshared'" group rather than in "allshared" group.. If that doesn't help: > I have added permissions for "$allshared" to a mailbox's Inbox. It does > not appear in the shared folders list. Other mailboxes, with explicit > permission for a given username, work fine. What am I doing wrong? 1. Make sure that acl_groups setting is enabled for that user by setting mail_debug=yes and looking at the log: imap: Debug: Added userdb setting: plugin/acl_groups=allshared 2. Check with: doveadm acl debug -u user From tss at iki.fi Sat Nov 5 19:20:19 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:20:19 +0200 Subject: [Dovecot] Restarting dovecot periodically fails with "Socket already exists: /var/run/dovecot/anvil" In-Reply-To: <4E97000D.7030908@ubuntu.com> References: <4E97000D.7030908@ubuntu.com> Message-ID: <1320513622.21919.260.camel@hurina> On Thu, 2011-10-13 at 16:13 +0100, James Page wrote: > restarting dovecot regularly fails with version 2.0.13 on Ubuntu 11.10 > due to a running child process exiting after the main dovecot process. > > Oct 13 15:04:58 ubuntu dovecot: master: Error: service(anvil): Socket > already exists: /var/run/dovecot/anvil > This is an Ubuntu specific issue as we use upstart to manage dovecot > which monitors the status of the main dovecot process; the Debian init > script appears to make provision for this happening. http://hg.dovecot.org/dovecot-2.1/rev/6894298ae5fd should apply to v2.0 as well, but since it's a bit large change I don't know if I should backport it, at least without some more testing. From mpapet at yahoo.com Sat Nov 5 19:33:20 2011 From: mpapet at yahoo.com (Michael Papet) Date: Sat, 5 Nov 2011 10:33:20 -0700 (PDT) Subject: [Dovecot] Newbie Question/Feature Request Message-ID: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> Hi, I'm working on a plugin for qpsmtpd (a perl smtp frontend) that delivers mail via dovecot's LDA. I can only get mail delivered by writing the email to file, then calling 'deliver -d someone -f someone-else < /path/to/temp/email' All good, expected behavior. A perl snippet: local $/=undef; open FH, "< ./demo.mail"; my $stringified = ; # this is roughly equivalent to how other LDA queues work. close FH; open (CMD_OUT, " /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom $stringified |"); my $test = ; print $test; The above code errors out because the expected behavior is $stringified is supposed to be a file name. It would be great if deliver could be fed the email as a string so I don't have to add a step of writing to file. Or, maybe there's an undocumented method? Thanks. mpapet From tss at iki.fi Sat Nov 5 19:39:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:39:50 +0200 Subject: [Dovecot] dsync with quotas In-Reply-To: <8762kozbzo.fsf@algae.riseup.net> References: <8762kozbzo.fsf@algae.riseup.net> Message-ID: <1320514790.21919.262.camel@hurina> On Mon, 2011-09-19 at 13:19 -0400, Micah Anderson wrote: > I have been working on converting people from courier maildir -> dovecot > mdbox and during some of the dsync runs I'm seeing the > quota_exceeded_message be printed as an Error: > > dsync(): Error: Can't save message to mailbox INBOX: You are over quota. To avoid losing mail, immediately empty your Trash and Sent folders and \ > delete emails with large attachments. > dsync(): Info: INBOX: Couldn't keep all uids^M > > Its possible the user was over quota on the originating courier side, > but I would still like to migrate their mail proprely to mdbox, but it > seems like being over quota is inhibiting that. Are the quota > calculations including both the maildir files as well as the converted > mdbox files, resulting in a double counting? Depends on how you did the conversion and what quota backend you were using. If they were using different quota files, there shouldn't be double counting. But if for example you did a migration within the same server and quota was set to dict quota, it was possibly being counted double (both thought they owned the quota file). > I'm not entirely sure if the messages above indicate that the migration > failed for that user or not, so I've been manually increasing their > quota, then redoing the dsync mirror until it works properly. > > I wonder if it would be better if I turned off quota entirely during > migration so I don't run into this problem? Added: http://hg.dovecot.org/dovecot-2.0/rev/00157fd523ea http://hg.dovecot.org/dovecot-2.0/rev/0cf3215cdf2d From tss at iki.fi Sat Nov 5 19:43:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:43:09 +0200 Subject: [Dovecot] Newbie Question/Feature Request In-Reply-To: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> References: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> Message-ID: <1320514990.21919.264.camel@hurina> On Sat, 2011-11-05 at 10:33 -0700, Michael Papet wrote: > open (CMD_OUT, " /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom $stringified |"); > my $test = ; > print $test; > > The above code errors out because the expected behavior is $stringified is supposed to be a file name. It would be great if deliver could be fed the email as a string so I don't have to add a step of writing to file. Or, maybe there's an undocumented method? I don't remember exactly how Perl works, and this is untested, but I think it should be something like: open (CMD_OUT, "| /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom"); print CMD_OUT $stringified; my $test = ; print $test; The idea is that you write the string to the pipe to deliver that you open, and deliver reads it from stdin. From tss at iki.fi Sat Nov 5 20:00:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 20:00:50 +0200 Subject: [Dovecot] Safely restoring backups In-Reply-To: <87sjn4rg7y.fsf@algae.riseup.net> References: <87sjn4rg7y.fsf@algae.riseup.net> Message-ID: <1320516051.21919.274.camel@hurina> On Fri, 2011-10-07 at 11:09 -0400, Micah Anderson wrote: > I'm trying to determine what the best way to restore mail with mdbox > is. Restoring using maildir was trivial, I just used rsync > --ignore-existing which wrote any mails that were removed and didn't > touch things that already existed[1]. If a mail had changed flag, the maildir file got duplicated, which Dovecot complained about if it noticed it. > With mdbox things have become more > complicated, and I haven't found a way to restore mail that doesn't > result in many message duplicates. Do you need to restore mails so often that this is really a problem? :) > 1. I tried rsyncing the different backup directories back to the mail > storage host, and then doing: > > 'dsync -R backup -u $user mdbox:/path/to/to/daily.1/mdbox'[2] > 'dsync -R backup -u $user mdbox:/path/to/to/daily.2/mdbox' > > This works ok, but it gives duplicates of every mail that already exists > for every daily/weekly I dsync. This is what the rsync --ignore-existing > avoided. This is particularly annoying if I restore the weekly, and > multiple daily directories, because then you get a copy for every one > you restore. I had thought that the individual message's GIDs would keep > them from being duplicated? GUIDs can be used to identify messages, but there's no automatic deduplication. It's fine to e.g. copy a message from INBOX to INBOX, which duplicates it. Dovecot shouldn't prevent that. > Is there a way I can restore things from backup and avoid duplicates? > Is there another method I should try? Here's one way, although somewhat slow (and not fully tested): doveadm -f tab fetch -u user at domain 'mailbox guid' all | sort > guids1 doveadm -f tab fetch -o mdbox:/backups/user -u user at domain 'mailbox-guid guid' all | sort > guids2 diff -u guids1 guids2 | grep '^+[+]' | sed 's/^+//' | awk { system("doveadm import -u user at domain mdbox:/backups/user restored mailbox-guid "$1" guid "$2); } > 3. why doesn't doveadm import have a -s option to subscribe? I suppose it could.. Added to v2.1: http://hg.dovecot.org/dovecot-2.1/rev/afec4ceda8e1 From apm at one.com Sat Nov 5 21:17:04 2011 From: apm at one.com (Peter Mogensen) Date: Sat, 05 Nov 2011 20:17:04 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EB58BB0.2090900@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: >> Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted >> transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: >> indexid changed 1320419300 -> 1320419441 (sync_offset=0) > > Session A had TestMails open and created with index file whose ID was > 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 > 17:08:20 EET 2011). > > Session B came and recreated the index files 141 seconds later with ID > 1320419441. Either it didn't see A's original index files for some > reason or it simply decided to recreate them for some reason. Either way > this shouldn't have happened. > > Session A then notices that the indexes were recreated, and logs an > error. Oh... wait a minute... The timestamp is UTC, so 17:08:20 is about 2:22 before the log line. 2:22 is 142 seconds. So... given that the errors doesn't appear every time the client runs the series of APPEND requests, but (now I come to think of it) probably never the first time he runs it, but the second time - and that he did run the script a few minutes before this log line with out errors, - then... the problem might be that the first run of the script doesn't finish correctly. If session A is the first run of the script, then it should have finished and logged out long before session B. But maybe the problem is the first run not finishing properly. /Peter From p at state-of-mind.de Sat Nov 5 21:43:50 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sat, 5 Nov 2011 20:43:50 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320508907.21919.239.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> Message-ID: <20111105194350.GA3782@state-of-mind.de> * Timo Sirainen : > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > is there a way to use Dovecot SASL Authentication for submission on a > > different machine dovecot runs? > > With v2.0 you should be able to add inet_listener for auth service, > which acts as an auth-client socket. > > > Dovecot machine <-> postfix machine(for submission) > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. Document it and Wietse will support it on the mailing list. If its not documented support will be declined. Should I? p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From lists at wildgooses.com Sat Nov 5 22:09:38 2011 From: lists at wildgooses.com (Ed W) Date: Sat, 05 Nov 2011 20:09:38 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2C704.1010107@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> Message-ID: <4EB59802.9040205@wildgooses.com> On 03/11/2011 16:53, Patrick Westenberg wrote: > Ed W schrieb: > >>> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >>> thinking about a SSD based LUN for the indexes. As I'm using multiple >>> servers this LUN will use OCFS2. >> >> Given that the SAN always has the network latency behind it, might you >> be better to look at putting the SSDs in the frontend machines? >> Obviously this then needs some way to make users "sticky" to one machine >> (or some few machines) where the indexes are stored? > > Storing the indexes on several machines? > In this case I have to synchronize them. > See the "sticky" in my reply. You use one of several techniques to ensure that users always end up on the server with the indexes on. That way much of the IO is served from that local machine and you only access the SAN for the (in theory much less frequent) access to the mail files themselves. Clearly if the machine with the indexes on dies then the load balancer needs to pick a new machine and there will be delay/io/etc while the indexes are regenerated. Various techniques could mitigate this... I don't have such a larger system - please ignore all my advice... The basis for the suggestion is that I understand file access (locking in particular) is "expensive" on OCFS2/GFS. Therefore I read here on this list that others have found performance issues accessing maildir over OCFS2? It's also not hard to find benchmarks that show OCFS2/GFS are "fast", but slower than accessing the same storage without using a cluster filesystem - this makes sense. Hence it seems like a trade between convenience of storing everything on a central store and "some" performance improvement from a more complex system... I think if you search on benchmarks of DRBD vs OCFS2 and read here on the list about the "director" and "proxy" services you can see the point? I'm just trying to help you see the effects you might want to measure! (I don't have a system large enough to know much about this stuff from experience...) Good luck! Ed W From dovecot at knutejohnson.com Sun Nov 6 00:15:02 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sat, 05 Nov 2011 15:15:02 -0700 Subject: [Dovecot] Converting from 1.12 to 2.0 questions about mail location? Message-ID: <4EB5B566.7070808@knutejohnson.com> I'm building a new server and using a newer version of ubuntu server that uses Dovecot 2.0. I thought I would like to use Maildir instead of the mbox that was used in the previous server. What do I have to do to convert my mbox mail to Maildir? Or should I just use mbox and copy it over :-)? Thanks, -- Knute Johnson From stan at hardwarefreak.com Sun Nov 6 00:46:57 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 05 Nov 2011 17:46:57 -0500 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2B15D.1010005@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> Message-ID: <4EB5BCE1.3020608@hardwarefreak.com> On 11/3/2011 10:21 AM, Ed W wrote: > >> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >> thinking about a SSD based LUN for the indexes. As I'm using multiple >> servers this LUN will use OCFS2. > > Given that the SAN always has the network latency behind it, might you > be better to look at putting the SSDs in the frontend machines? The latency of the GbE IP network, iSCSI HBAs, GbE switches, etc, is but a fraction of the overhead of the out of band OCFS metadata exchange between cluster members, and the general overhead of OCFS, or of cluster filesystems in general. > Obviously this then needs some way to make users "sticky" to one machine > (or some few machines) where the indexes are stored? > > This seems theoretically likely to give you higher IOPs to the index > than having them on the OCFS2 storage? (At a trade off with more > complexity for the load balancer front end...) Following this logic, simply using local mechanical disk would yield improvement without the cost of SSDs. Depending on the number of nodes, putting a couple of SSDs in the SAN controller may likely be cheaper overall than adding mech disks to each node, let alone SSDs to each node. The random IO latency of SSD is so considerably lower than mechanical disk, even with the OCFS and iSCSI SAN overhead, overall read/write latency will likely be lower than using local mech disk in the nodes. And you get to retain centralized storage of the indexes, eliminating stickiness complexity issues. Something else to consider is the read/write caching performance of NexentaStor (I've never used it, know nothing about it). If it's very good, and the NexentaStor host has gobs of RAM (think 64-128GB), then adding SSDs for indexes may not improve performance much, if any, depending on the concurrent user load. I've read cases where adding SLC-SSD to high end FC SAN controllers with gobs of writeback cache RAM yielded little benefit with similar random IO workloads, simply because the cache was never taxed enough to force regular flushing. If your cache is large and fast enough to buffer most of your IOPS, then your current spindle speed is already irrelevant. In such a case adding SSD will yield little, or no, advantage. -- Stan From knute2011 at knutejohnson.com Sun Nov 6 01:35:46 2011 From: knute2011 at knutejohnson.com (Knute Johnson) Date: Sat, 05 Nov 2011 16:35:46 -0700 Subject: [Dovecot] Converting from 1.12 to 2.0 questions about mail location? In-Reply-To: <4EB5B566.7070808@knutejohnson.com> References: <4EB5B566.7070808@knutejohnson.com> Message-ID: <4EB5C852.5060301@knutejohnson.com> On 11/5/2011 3:15 PM, Knute Johnson wrote: > I'm building a new server and using a newer version of ubuntu server > that uses Dovecot 2.0. I thought I would like to use Maildir instead of > the mbox that was used in the previous server. What do I have to do to > convert my mbox mail to Maildir? Or should I just use mbox and copy it > over :-)? > > Thanks, > After looking at it some more, I think I'm just going to keep the mbox format mail. Thanks, -- Knute Johnson knute2011 at knutejohnson.com From e-frog at gmx.de Sun Nov 6 18:03:47 2011 From: e-frog at gmx.de (e-frog) Date: Sun, 06 Nov 2011 17:03:47 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub Message-ID: <4EB6AFE3.3020001@gmx.de> Hi Timo, I'm using the virtual plugin and have 2 mailboxes configured in the "virtual" namespace (dovecot -n attached). Using lsub with dovecot 2.0.15 leads to the following result: 1 lsub "" "virtual/*" * LSUB () "/" "virtual/Lists" * LSUB () "/" "virtual/unread" 1 OK Lsub completed. However with dovecot 2.1 (latest hg as of today) and the same config there is a 3rd mailbox "virtual" listed which doesn't exist: 1 lsub "" "virtual/*" * LSUB () "/" "virtual" * LSUB () "/" "virtual/Lists" * LSUB () "/" "virtual/unread" 1 OK Lsub completed. Thunderbird shows the "virtual" mailbox as selectable and not grayed out as with 2.0.15. Clicking on that mailbox gives an error message "Mailbox doesn't exist: virtual". Thanks, e-frog -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf-n.2.0.15.txt URL: From mds at schinhammer.de Mon Nov 7 01:54:28 2011 From: mds at schinhammer.de (Michael Schinhammer) Date: Mon, 07 Nov 2011 00:54:28 +0100 Subject: [Dovecot] Problems with password verification on a BSD box against PAM Message-ID: <4EB71E34.5050706@schinhammer.de> Spam detection software, running on the system "ns.089.at", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hi ML, i want to use my local users database on my Freebsd 8.2 box and got the following error in the maillog. The password is ok and it working on the console. i can establish the ssl connection after login "a login ux password" the error occures i cant login. [...] Content analysis details: (6.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.2 MR_NOT_ATTRIBUTED_IP Beta rule: an non-attributed IPv4 found in headers 0.6 J_CHICKENPOX_65 BODY: 6alpha-pock-5alpha 0.6 J_CHICKENPOX_73 BODY: 7alpha-pock-3alpha 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS 0.0 NO_RDNS2 Sending MTA has no reverse DNS 3.5 TO_NO_BRKTS_DIRECT To: misformatted and direct-to-MX -------------- next part -------------- An embedded message was scrubbed... From: Michael Schinhammer Subject: Problems with password verification on a BSD box against PAM Date: Mon, 07 Nov 2011 00:54:28 +0100 Size: 2066 URL: From pw at wk-serv.de Mon Nov 7 02:08:23 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 07 Nov 2011 01:08:23 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB59802.9040205@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> Message-ID: <4EB72177.8090801@wk-serv.de> Ed W schrieb: > See the "sticky" in my reply. You use one of several techniques to > ensure that users always end up on the server with the indexes on. That > way much of the IO is served from that local machine and you only access > the SAN for the (in theory much less frequent) access to the mail files > themselves. I know you can afford that (IMAP) users always end up on one particular server but afaik this only works for incoming IMAP connections. My mail exchangers use dovecot-lda and I think indexes will be written from these servers too or am I wrong with this? Patrick From komodo at uvt.cz Mon Nov 7 12:10:20 2011 From: komodo at uvt.cz (komodo) Date: Mon, 7 Nov 2011 11:10:20 +0100 Subject: [Dovecot] Missing public folder Message-ID: <201111071110.20143.komodo@uvt.cz> Hi i am using dovecot 2.0.13 and i can't see public folder in my mail client. But what is strange, it was working after instalation, and i don't know when it dissapeared. Maybe after some upgrade ? When i enable debug, then i can see in my log that folder exists and no errors. Please where should i find the problem ? Thanks Martin Here is log Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: Namespace : type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: maildir++: root=/home/komodo/Maildir, index=, control=, inbox=/home/komodo/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: Namespace : type=public, prefix=spam_learner., sep=., inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/shared/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: maildir++: root=/home/shared/Maildir, index=, control=, inbox= Here is my conf # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab037.1 x86_64 CentOS release 5.5 (Final) default_client_limit = 1300 disable_plaintext_auth = no hostname = somehost.com mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { location = maildir:/home/shared/Maildir prefix = spam_learner. separator = . subscriptions = no type = public } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/before.d/ sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /usr/sbin/sendmail.exim service imap-login { process_limit = 1000 process_min_avail = 30 } ssl_cert = References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> Message-ID: <4EB7BF9A.9020900@webfusion.com> 04-11-2011 23:56, Timo Sirainen yazm??: > On Thu, 2011-10-20 at 12:38 +0300, Mark Zealey wrote: > >> I'm currently changing some systems to use the redirector service, which >> means that to get local deliveries going I need to get lmtp set up so it >> can be redirected as well. This is working fine, however we have a >> number of different ports running on our servers depending on which >> brand a customer is using. When the time comes to do the user look up, >> we use (port, user, domain) to do a unique database lookup which works >> fine for everything (proxied pop, imap, lmtp& straight pop, imap) but >> not straight lmtp - perhaps because it doesn't have a 2-stage login >> process unlike the pop/imap protocols. Looking at the SQL query we are >> issuing, the %a port variable is set to 0 even though the connection is >> coming in from TCP. Is there a way to change this? We are running >> dovecot 2.0.12 however looking through the changelogs I can't see this >> would be fixed in the newest version. > A quick check in code shows that this is already supposed to work. Set > auth_debug=yes and see if LMTP's auth lookup sends lip, rip, lport and > rport fields to auth process? From my reading of the code in 2.0.15, you pass the local & remote IP in lmtp/commands.c:427 but the struct mail_storage_service_input only has remote_ip and local_ip fields, not port fields (unlike the auth_user_info struct which has both ip's and port entries as well). I think that the mail_storage_service_input struct needs the uint local_port, remote_port fields adding in & appropriate code changes to pass these through? Mark From gordon.grubert at uni-greifswald.de Sun Nov 6 20:56:05 2011 From: gordon.grubert at uni-greifswald.de (Gordon Grubert) Date: Sun, 06 Nov 2011 19:56:05 +0100 Subject: [Dovecot] Dovecot crashes totally In-Reply-To: <1320435812.21919.150.camel@hurina> References: <4EA317B5.3090209@uni-greifswald.de> <1320435812.21919.150.camel@hurina> Message-ID: <4EB6D845.7040208@uni-greifswald.de> On 11/04/2011 08:43 PM, Timo Sirainen wrote: > On Sat, 2011-10-22 at 21:21 +0200, Gordon Grubert wrote: >> Hello, >> >> our dovecot server crashes totally without any really useful >> log messages. The error log can be found in the attachment. >> The only way to get dovecot running again is a complete >> system restart. > > How often does it break? If really a "complete system restart" is needed > to fix it, it doesn't sound like a Dovecot problem. Check if it's enough > to stop dovecot and then make sure there aren't any dovecot processes > lying around afterwards. Currently, the problem occurred three times. The last time some days ago. The last "crash" was in the night and, therefore, we used the chance for a detailed debugging of the system. You could be right, that it's not a dovecot problem. Next to dovecot, we found other processes hanging and could not be killed by "kill -9". Additionally, we found a commonness of all of these processes: They hanged while trying to access the mailbox volume. Therefore, we repaired the filesystem. Now, we're watching the system ... >> Oct 11 09:55:23 mailserver2 dovecot: master: Error: service(imap): >> Initial status notification not received in 30 seconds, killing the >> process >> Oct 11 09:56:23 mailserver2 dovecot: imap-login: Error: master(imap): >> Auth request timed out (received 0/12 bytes) > > Kind of looks like auth process is hanging. You could see if stracing it > shows anything useful. Also are any errors logged about LDAP? Is LDAP > running on the same server? Dovecot authenticates against postfix and postfix has an LDAP connection. The LDAP is running on an external cluster. Here, no errors are reported. We hope, that the filesystem error was the reason for the problem and, that the problem is fixed by repairing it. Best regards, Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5396 bytes Desc: S/MIME Cryptographic Signature URL: From yuichi at sklc.co.jp Mon Nov 7 17:12:33 2011 From: yuichi at sklc.co.jp (Yuuichi Ikeda (SKLC)) Date: Tue, 08 Nov 2011 00:12:33 +0900 Subject: [Dovecot] POP3/IMAPv4 CRAM-MD5 Authentication failed.(Re-post) Message-ID: <20111108001232.2ACB.1986BE3C@sklc.co.jp> Hi, I'm Yuuichi Ikeda from Japan User. OS:Solaris 10 9/10 s10x_u9wos_14a X86 Mem:8GB HDD:3TB gcc:gcc (GCC) 4.1.2 gcc-prefix:/unsupported/gcc Dovecot Version:2.0.15 configure:./configure --prefix=/opt/dovecot_2 --sysconfdir=/opt/dovecot_2/conf --mandir=/opt/man --enable-shared --with-mysql --with-zlib --with-sqlite --with-sql=plugin --with-ssldir=/opt/openssl --with-rundir=/var/run --with-libiconv-prefix=/opt/libiconv Dovecot Configuration > # 2.0.15: /opt/dovecot_2/conf/dovecot/dovecot.conf > # OS: SunOS 5.10 i86pc > auth_debug = yes > auth_mechanisms = cram-md5 > auth_ssl_require_client_cert = yes > auth_ssl_username_from_cert = yes > auth_verbose = yes > base_dir = /var/run/dovecot/ > doveadm_worker_count = 10 > log_path = /var/log/dovecot/dovecot.log > login_greeting = ready. > login_trusted_networks = 192.168.1.0/24 > mail_location = maildir:~/Maildir > passdb { > driver = pam > } > passdb { > args = /opt/dovecot_2/conf/dovecot/passwd > driver = passwd-file > } > plugin { > acl = vfile:/opt/dovecot_2/conf/dovecot/global-acls:cache_secs=300 > acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes > } > protocols = imap pop3 > service auth { > executable = /opt/dovecot_2/libexec/dovecot/auth > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > } > service imap-login { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/imap-login > inet_listener imap { > port = 143 > ssl = no > } > inet_listener imaps { > port = 993 > ssl = yes > } > } > service imap { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/imap > } > service lmtp { > unix_listener lmtp { > mode = 0666 > } > } > service pop3-login { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/pop3-login > inet_listener pop3 { > port = 110 > ssl = no > } > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service pop3 { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/pop3 > } > ssl_ca = ssl_cert = ssl_key = ssl_verify_client_cert = yes > userdb { > args = blocking=yes > driver = passwd > } > protocol imap { > imap_logout_format = bytes=%i/%o > imap_max_line_length = 64 k > mail_max_userip_connections = 10 > mail_plugins = > } > protocol lda { > hostname = mailsv.sklc.co.jp > info_log_path = /var/log/dovecot/deliver.log > log_path = /var/log/dovecot/deliver.log > mail_plugins = > postmaster_address = postmaster at sklc.co.jp > sendmail_path = /usr/lib/sendmail > } > protocol lmtp { > mail_plugins = > } > protocol pop3 { > mail_plugins = > pop3_save_uidl = yes > pop3_uidl_format = %v-%u > } If it attests by connecting by POP3 or IMAPv4, the following messages will be displayed and attestation will go wrong. > Nov 07 23:12:40 auth: Debug: auth client connected (pid=20018) > Nov 07 23:12:40 auth: Debug: client in: AUTH 1 CRAM-MD5 service=pop3 secured no-penalty lip=192.168.1.1 rip=192.168.1.110 lport=110 rport=57054 > Nov 07 23:12:40 auth: Info: CRAM-MD5(?,192.168.1.110): Client didn't present valid SSL certificate > Nov 07 23:12:40 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate > Nov 07 23:12:40 pop3-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.110, lip=192.168.1.1, secured > Nov 07 23:16:32 auth: Debug: auth client connected (pid=20126) > Nov 07 23:16:32 auth: Debug: client in: AUTH 1 CRAM-MD5 service=imap secured no-penalty lip=192.168.1.1 rip=192.168.1.1 lport=143 rport=58734 > Nov 07 23:16:32 auth: Info: CRAM-MD5(?,192.168.1.1): Client didn't present valid SSL certificate > Nov 07 23:16:32 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate > Nov 07 23:16:32 imap-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.1, lip=192.168.1.1, secured What will you do and will become like this? If some people know ways of coping, please let me know. ============================================================= Information-system part. Sankei-Koumuten Co.,Ltd. Yuuichi Ikeda Mail:yuichi at sklc.co.jp Tel.+81-3-3623-6474 Fax.+81-3-3623-6475 Our company promotes "Team minus 6 percent" jus, Hatena Joined member. LPIC-2 Certified. ============================================================= From micah at riseup.net Mon Nov 7 19:13:07 2011 From: micah at riseup.net (micah anderson) Date: Mon, 07 Nov 2011 12:13:07 -0500 Subject: [Dovecot] Accessing a strange mailbox In-Reply-To: <1320435243.21919.144.camel@hurina> References: <8762jie6ut.fsf@algae.riseup.net> <1320435243.21919.144.camel@hurina> Message-ID: <87aa87onz0.fsf@algae.riseup.net> Hi, thanks for the reply! On Fri, 04 Nov 2011 21:34:03 +0200, Timo Sirainen wrote: > On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote: > > I have a user who has a mailbox called: > > > > A----->B > > > > It seemed to work in courier fine, they managed to create it, and there > > are mails in it. However, dovecot is not letting the user access it, the > > IMAP server gives an error. I tried to rename it but I would also get an > > error: > > > > # doveadm mailbox rename 'A----->B' AtoB > > doveadm(root): Error: Can't rename mailbox A----->B to AtoB: Mailbox doesn't exist: A----->B > > Maybe it's not named that? What does it look like in filesystem? Worked > fine with me: I moved the user from courier maildir to mdbox, and I've still got the maildir folders around, the filesystem shows it as: drwx------ 6 mail mail 4096 Oct 20 07:56 .A----->B I just realized my mistake, my 'doveadm mailbox rename' command wasn't limited to the particular user (I was not passing -u ) so it was looking in the wrong namespace for that user. Once I specified the user, it worked properly. Micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From user+dovecot at localhost.localdomain.org Mon Nov 7 22:13:25 2011 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 07 Nov 2011 21:13:25 +0100 Subject: [Dovecot] POP3/IMAPv4 CRAM-MD5 Authentication failed.(Re-post) In-Reply-To: <20111108001232.2ACB.1986BE3C@sklc.co.jp> References: <20111108001232.2ACB.1986BE3C@sklc.co.jp> Message-ID: <4EB83BE5.5030006@localhost.localdomain.org> On 11/07/2011 04:12 PM Yuuichi Ikeda (SKLC) wrote: > Hi, I'm Yuuichi Ikeda from Japan User. > > > Dovecot Configuration >> ? >> ssl_ca = > ssl_cert = > ssl_key = > ssl_verify_client_cert = yes > ? > If it attests by connecting by POP3 or IMAPv4, the following messages will > be displayed and attestation will go wrong. > >> Nov 07 23:12:40 auth: Info: CRAM-MD5(?,192.168.1.110): Client didn't present valid SSL certificate >> Nov 07 23:12:40 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate >> Nov 07 23:12:40 pop3-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.110, lip=192.168.1.1, secured > >> Nov 07 23:16:32 auth: Info: CRAM-MD5(?,192.168.1.1): Client didn't present valid SSL certificate >> Nov 07 23:16:32 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate >> Nov 07 23:16:32 imap-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.1, lip=192.168.1.1, secured > > What will you do and will become like this? If some people know ways of > coping, please let me know. Are you sure you want to verify the client's certificate (ssl_verify_client_cert = yes)? If not, just remove this line and try again. Regards, Pascal -- The trapper recommends today: cafebabe.1131121 at localdomain.org From alietsantiesteban at gmail.com Tue Nov 8 01:02:23 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Mon, 7 Nov 2011 18:02:23 -0500 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: <1320440605.21919.198.camel@hurina> References: <1320440605.21919.198.camel@hurina> Message-ID: We checked with the firewall admins and they can not change the drop action, this model doesn't support reject, only drops, but for testing they disabled the ldap protocol idle timeout wich was set to 30 mins to never so the firewall never drops ldap idle connections, we also verified the clientidletimeout option in Openldap but is set to 0 wich means never close a idle connection. After testing again we see the connection hanging again after user inactivity, we will keep looking for other issues and maybe do some packet captures to see what is really happening. best regards, btw it would be great this ldap_idle_disconnect = 30s 2011/11/4 Timo Sirainen > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes wrote: > > I'm having a problem with dovecot ldap connection when ldap server is in > > another firewall zone, firewall kills the ldap connection after a > > determined period of inactivity, this is good from the firewall point of > > view but is bad for dovecot because it never knows the connections has > been > > dropped, this creates longs timeouts in dovecot and finally it > reconnects, > > meanwhile many users fails to authenticate, I have seen this kind of post > > in the list for a while but can't find a solution for it, so my question > is > > how to define a idle ldap time in dovecot so it can reconnect before the > > firewall has dropped the connection or just close the connection under > > inactivity so when a user authenticate doesn't fails for a while until > > dovecot detects that the connection has hanged. Is this a feature request > > or there is already a configuration for this??? > > Can't the firewall be changed to reject the LDAP packets instead of > dropping them? Then Dovecot would immediately notice that the connection > has died, and with a recent enough version it wouldn't even log an error > about it. > > I guess some kind of an "ldap_idle_disconnect = 30s" setting could be > added, but it's not a very high priority for me. > > > From tss at iki.fi Tue Nov 8 01:11:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 08 Nov 2011 01:11:02 +0200 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: References: <1320440605.21919.198.camel@hurina> Message-ID: <1320707463.21919.337.camel@hurina> If you set openldap server to close idle clients sooner than the connection itself is dropped by firewall (or whatever), then Dovecot sees the disconnection and won't hang. So you could try something like clientidletimeout=5 mins On Mon, 2011-11-07 at 18:02 -0500, Aliet Santiesteban Sifontes wrote: > We checked with the firewall admins and they can not change the drop > action, this model doesn't support reject, only drops, but for testing > they disabled the ldap protocol idle timeout wich was set to 30 mins > to never so the firewall never drops ldap idle connections, we also > verified the clientidletimeout option in Openldap but is set to 0 wich > means never close a idle connection. After testing again we see the > connection hanging again after user inactivity, we will keep looking > for other issues and maybe do some packet captures to see what is > really happening. > best regards, btw it would be great this ldap_idle_disconnect = 30s > > 2011/11/4 Timo Sirainen > > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes > wrote: > > I'm having a problem with dovecot ldap connection when ldap > server is in > > another firewall zone, firewall kills the ldap connection > after a > > determined period of inactivity, this is good from the > firewall point of > > view but is bad for dovecot because it never knows the > connections has been > > dropped, this creates longs timeouts in dovecot and finally > it reconnects, > > meanwhile many users fails to authenticate, I have seen this > kind of post > > in the list for a while but can't find a solution for it, so > my question is > > how to define a idle ldap time in dovecot so it can > reconnect before the > > firewall has dropped the connection or just close the > connection under > > inactivity so when a user authenticate doesn't fails for a > while until > > dovecot detects that the connection has hanged. Is this a > feature request > > or there is already a configuration for this??? > > > Can't the firewall be changed to reject the LDAP packets > instead of > dropping them? Then Dovecot would immediately notice that the > connection > has died, and with a recent enough version it wouldn't even > log an error > about it. > > I guess some kind of an "ldap_idle_disconnect = 30s" setting > could be > added, but it's not a very high priority for me. > > > From alietsantiesteban at gmail.com Tue Nov 8 04:41:12 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Mon, 7 Nov 2011 21:41:12 -0500 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: <1320707463.21919.337.camel@hurina> References: <1320440605.21919.198.camel@hurina> <1320707463.21919.337.camel@hurina> Message-ID: We will try this as next step to find a workaround, the problem with client idletimeout=5 mins in openldap server is that is a global server definition and have the net effect of changing replication refreshAndPersit into type refreshOnly which is not a welcome side effect, we will look other options, still the better candidate is ldap_idle_disconnect in dovecot side or any other kind of logic able to detect this kind of problems. best regards 2011/11/7 Timo Sirainen > If you set openldap server to close idle clients sooner than the > connection itself is dropped by firewall (or whatever), then Dovecot > sees the disconnection and won't hang. So you could try something like > clientidletimeout=5 mins > > On Mon, 2011-11-07 at 18:02 -0500, Aliet Santiesteban Sifontes wrote: > > We checked with the firewall admins and they can not change the drop > > action, this model doesn't support reject, only drops, but for testing > > they disabled the ldap protocol idle timeout wich was set to 30 mins > > to never so the firewall never drops ldap idle connections, we also > > verified the clientidletimeout option in Openldap but is set to 0 wich > > means never close a idle connection. After testing again we see the > > connection hanging again after user inactivity, we will keep looking > > for other issues and maybe do some packet captures to see what is > > really happening. > > best regards, btw it would be great this ldap_idle_disconnect = 30s > > > > 2011/11/4 Timo Sirainen > > > > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes > > wrote: > > > I'm having a problem with dovecot ldap connection when ldap > > server is in > > > another firewall zone, firewall kills the ldap connection > > after a > > > determined period of inactivity, this is good from the > > firewall point of > > > view but is bad for dovecot because it never knows the > > connections has been > > > dropped, this creates longs timeouts in dovecot and finally > > it reconnects, > > > meanwhile many users fails to authenticate, I have seen this > > kind of post > > > in the list for a while but can't find a solution for it, so > > my question is > > > how to define a idle ldap time in dovecot so it can > > reconnect before the > > > firewall has dropped the connection or just close the > > connection under > > > inactivity so when a user authenticate doesn't fails for a > > while until > > > dovecot detects that the connection has hanged. Is this a > > feature request > > > or there is already a configuration for this??? > > > > > > Can't the firewall be changed to reject the LDAP packets > > instead of > > dropping them? Then Dovecot would immediately notice that the > > connection > > has died, and with a recent enough version it wouldn't even > > log an error > > about it. > > > > I guess some kind of an "ldap_idle_disconnect = 30s" setting > > could be > > added, but it's not a very high priority for me. > > > > > > > > > From Yann.Dupont at univ-nantes.fr Tue Nov 8 12:05:42 2011 From: Yann.Dupont at univ-nantes.fr (Yann Dupont) Date: Tue, 08 Nov 2011 11:05:42 +0100 Subject: [Dovecot] Attachements with SIS : shauld all the same files share the same inode ? In-Reply-To: <1320510194.21919.251.camel@hurina> References: <4E9BD704.9080801@univ-nantes.fr> <1320510194.21919.251.camel@hurina> Message-ID: <4EB8FEF6.7000208@univ-nantes.fr> Hi timo, thanks for your answer Le 05/11/2011 17:23, Timo Sirainen a ?crit : > On Mon, 2011-10-17 at 09:19 +0200, Yann Dupont wrote: >> 104 instance of -rw------- 104 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-ffd0d1xxxx >> >> The files are hardlinked, so it seems ok, but I also see 9 others files >> in the same directory : >> >> >> -rw------- 1 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-0519df2a94fc9a4e1f4100008a6b99b6 >> -rw------- 1 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-1790b92994fc9a4e1a4100008a6b99b6 >> >> as you can see, same prefix, If I compare them with diff, they are all >> the same files. > > I guess you have "mail_attachment_fs = sis posix" (default)? Sounds like yes that's the case. > there could be a bug where SIS does comparison of files but for some > reason thinks they are different. > Well I can't try anymore on this, because this particular attachment is not there. In fact this server keeps only 2 weeks of mails for 5000+ accounts (a form of backup for the main servers). I'll take a new one : -> rw------- 103 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c0c2230ecdd1b74ec91600008a6b99b6 -rw------- 103 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c1c1290ecdd1b74ec51600008a6b99b6 -rw------- 1 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 So f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 has only one link, but diff f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c1c1290ecdd1b74ec51600008a6b99b6 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 tell nothing : they are same files. > You could try if you can merge them with: > > mkdir /tmp/queue > touch /tmp/queue/6e2df299fa96daec9b4735d07c494046429be4d6-foo > doveadm sis deduplicate /var/attachments /tmp/queue > > hum I didn't get what you want to do ? Does putting a 0 byte file there force a complete rescan ? Tried it : the 0 byte length /tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo disappear from the /tmp/queue, but f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 still has one link. A strace reveal this stat("/vmail/attachments-deduplic", {st_mode=S_IFDIR|0700, st_size=8192, ...}) = 0 open("/tmp/queue", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 8 getdents(8, /* 3 entries */, 32768) = 112 stat("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 link("/vmail/attachments-deduplic/f7/c9/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo", "/vmail/attachments-deduplic/f7/c9/hashes/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667") = -1 ENOENT (No such file or directory) mkdir("/vmail/attachments-deduplic/f7/c9/hashes", 0700) = -1 EEXIST (File exists) unlink("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo") = 0 getdents(8, /* 0 entries */, 32768) = 0 close(8) = 0 Tried with a copy of the file but this doesn't work either : stat("/vmail/attachments-deduplic", {st_mode=S_IFDIR|0700, st_size=8192, ...}) = 0 open("/tmp/queue", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 8 getdents(8, /* 3 entries */, 32768) = 144 stat("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6", {st_mode=S_IFREG|0600, st_size=571037, ...}) = 0 write(2, "doveadm(root): Fatal: /tmp/queue"..., 170doveadm(root): Fatal: /tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 is not a valid sis-queue file, is the queue directory correct? How can I help further ? Thanks, -- Yann Dupont - Service IRTS, DSI Universit? de Nantes Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont at univ-nantes.fr From listas.correo at yahoo.es Tue Nov 8 12:20:22 2011 From: listas.correo at yahoo.es (mailing lists) Date: Tue, 8 Nov 2011 10:20:22 +0000 (GMT) Subject: [Dovecot] Failing to share folders when listescape is enabled (2.0.14) Message-ID: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> Hello, How I can share a folder with dots when the listescape plugin is enabled? In this example user001 is sharing two foders named "docs-abc" and "docs-a.b.c" to user002. The first folder (without dots) is seen by user002 but the second is not found by dovecot because it search a system folder named ".docs-a.b.c" but the system folder is ".docs-a\2eb\2ec" Nov? 8 11:03:52 imap2 dovecot: imap(user002): Debug: acl vfile: file /var/virtual-maildir/user001/.docs-a.b.c/dovecot-acl not found drwx------ 2 vmail vmail 4096 2011-11-08 10:54 cur drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .docs-a\2eb\2ec drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .docs-abc -rw------- 1 vmail vmail?? 20 2011-11-08 10:57 dovecot-acl-list -rw------- 1 vmail vmail? 248 2011-11-08 10:55 dovecot.index.log -rw------- 1 vmail vmail?? 96 2011-11-08 10:54 dovecot.mailbox.log -rw------- 1 vmail vmail?? 51 2011-11-08 10:55 dovecot-uidlist -rw------- 1 vmail vmail??? 8 2011-11-08 10:57 dovecot-uidvalidity -r--r--r-- 1 vmail vmail??? 0 2011-11-08 10:54 dovecot-uidvalidity.4eb8fc5b drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .Drafts -rw------- 1 vmail vmail?? 15 2011-11-08 10:54 maildirsize drwx------ 2 vmail vmail 4096 2011-11-08 10:54 new drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Sent drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Spam -rw------- 1 vmail vmail?? 23 2011-11-08 10:54 subscriptions drwx------ 2 vmail vmail 4096 2011-11-08 10:54 tmp drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Trash # cat /var/virtual-maildir/user001/subscriptions Trash Sent Drafts Spam docs-abc docs-a\2eb\2ec # telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. . login user001 XXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in . create docs-abc??? . OK Create completed. . create docs-a.b.c . OK Create completed. . setacl docs-abc user002 lrsw . OK Setacl complete. . setacl docs-a.b.c user002 lrsw . OK Setacl complete. . subscribe docs-abc . OK Subscribe completed. . subscribe docs-a.b.c . OK Subscribe completed. . logout * BYE Logging out . OK Logout completed. Connection closed by foreign host. # telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. . login user002 XXXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in . subscribe shared/user001/docs-abc . OK Subscribe completed. . subscribe shared/user001/docs-a.b.c . NO Mailbox doesn't exist: shared/user001/docs-a.b.c . logout * BYE Logging out . OK Logout completed. Connection closed by foreign host. ??? /-----------------/ # dovecot -n # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34.7-0.7-xen x86_64 openSUSE 11.3 (x86_64) auth_debug = yes auth_verbose = yes base_dir = /var/run/dovecot/ dict { ? expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_proxy_port = 24244 doveadm_socket_path = localhost:24244 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_debug = yes mail_fsync = always mail_gid = 5000 mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = acl quota notify expire listescape mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { ? inbox = yes ? list = yes ? location = ? prefix = ? separator = / ? subscriptions = yes ? type = private } namespace { ? list = children ? location = maildir:/var/virtual-maildir/%%n:INDEX=~/shared.%%n ? prefix = shared/%%n/ ? separator = / ? subscriptions = no ? type = shared } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } plugin { ? acl = vfile ? acl_shared_dict = file:/var/maildir/vol00/shared-mailboxes ? autocreate = Trash ? autocreate2 = Sent ? autocreate3 = Drafts ? autocreate4 = Spam ? autosubscribe = Trash ? autosubscribe2 = Sent ? autosubscribe3 = Drafts ? autosubscribe4 = Spam ? expire = Trash ? expire2 = Trash/* ? expire5 = Spam ? expire6 = Spam/* ? expire_dict = proxy::expire ? mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append ? mail_log_fields = uid box msgid from subject size vsize flags ? mail_plugins = " mail_log notify" ? quota = maildir:Quota ? quota_exceeded_message = Quota limit reached ? quota_rule2 = Trash:storage=+50M ? quota_warning = storage=95%% quota-warning 95 %u ? quota_warning2 = storage=90%% quota-warning 90 %u ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } service auth { ? unix_listener auth-userdb { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service dict { ? unix_listener dict { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service doveadm { ? inet_listener { ??? port = 24244 ? } } service lmtp { ? inet_listener lmtp { ??? port = 24 ? } ? unix_listener lmtp { ??? group = vmail ??? mode = 0666 ??? user = vmail ? } } service pop3-login { ? inet_listener pop3 { ??? port = 110 ? } } service quota-warning { ? executable = script /etc/dovecot/quota-warning.sh ? unix_listener quota-warning { ??? user = vmail ? } } ssl = no userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } verbose_proctitle = yes protocol lmtp { ? mail_plugins = acl quota notify expire listescape sieve } protocol lda { ? mail_plugins = acl quota notify expire listescape sieve } protocol imap { ? mail_max_userip_connections = 100 ? mail_plugins = acl quota notify expire listescape imap_acl autocreate imap_quota } protocol pop3 { ? pop3_uidl_format = %v-%u } local 10.1.245.101/28/28 { ? doveadm_password = secret } From tss at iki.fi Tue Nov 8 14:26:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 14:26:10 +0200 Subject: [Dovecot] Failing to share folders when listescape is enabled (2.0.14) In-Reply-To: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> References: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> Message-ID: <3F867C40-86D5-4572-A6A7-8B6C685245B9@iki.fi> On 8.11.2011, at 12.20, mailing lists wrote: > How I can share a folder with dots when the listescape plugin is enabled? Sorry, doesn't work in v2.0 and I don't think it's possible to fix it without major changes. Those major changes are done in v2.1 code tree though, and it works there. From p.heinlein at heinlein-support.de Tue Nov 8 15:50:25 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 14:50:25 +0100 Subject: [Dovecot] Performance-Tuning Message-ID: <201111081450.25798.p.heinlein@heinlein-support.de> Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff). I have to change my setup, maybe somebody else here have hard facts: *) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout? *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Thanks for any hints an tricks, Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From ricardo at wenn.com Tue Nov 8 15:59:01 2011 From: ricardo at wenn.com (Ricardo Branco) Date: Tue, 8 Nov 2011 13:59:01 +0000 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <28618879-1320760744-cardhu_decombobulator_blackberry.rim.net-908347421-@b13.c3.bise7.blackberry> What is the setup on the EVA, FC or iSCSI? Sent from my BlackBerry? wireless device -----Original Message----- From: Peer Heinlein Sender: dovecot-bounces at dovecot.org Date: Tue, 8 Nov 2011 14:50:25 To: Subject: [Dovecot] Performance-Tuning Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff). I have to change my setup, maybe somebody else here have hard facts: *) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout? *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Thanks for any hints an tricks, Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From mstevens at imt-systems.com Tue Nov 8 16:03:07 2011 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 08 Nov 2011 15:03:07 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> On 08.11.2011 14:50, Peer Heinlein wrote: > *) Is mdbox really faster? I'd like to have mdbox to have better > performance > in running my backup-processes. But does it bring some performance > boosts > to? Hi Peer, We have switched our mailbox storage format from maildir to mdbox! Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly. Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4. Best regards, Morten From javierdemiguel at us.es Tue Nov 8 16:15:39 2011 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_de_Miguel_Rodr=EDguez?=) Date: Tue, 08 Nov 2011 15:15:39 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EB9398B.1000904@us.es> We are very happy with mdbox+zlib+ext4 + iSCSI SAN (HP Lefthand in our setup) If you have CPU to spare, consider using zlib with mdbox. You are trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Mdbox has halved our backup windows (2,8 TB uncompressed mailboxes, 2 TB compressed) and backup software is happier because there are few (100.000+ files with mdbox) to backup instead of several millions (Maildir) Regards Javier > Hi, > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. > > I always wanted to make some mesurements about several influences to the > performance (switch to ext4, switch to mdbox), but I never had enough time > to do that. > > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). > > I have to change my setup, maybe somebody else here have hard facts: > > *) Is ext4 faster? How much faster? > *) Is it faster because of the ext4 kernel-module (which can be used on ext3 > to) or because of the ext4 filesystem layout? > > > *) Is mdbox really faster? I'd like to have mdbox to have better performance > in running my backup-processes. But does it bring some performance boosts > to? > > > Thanks for any hints an tricks, > > Peer > > From Ralf.Hildebrandt at charite.de Tue Nov 8 16:16:12 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 8 Nov 2011 15:16:12 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> Message-ID: <20111108141612.GE27296@charite.de> * Morten Stevens : > We have switched our mailbox storage format from maildir to mdbox! I wonder how I can incrementally change over from Maildir to mdbox? I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa). That would allow for a smooth transition... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From p.heinlein at heinlein-support.de Tue Nov 8 16:19:05 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:19:05 +0100 Subject: [Dovecot] Multiple Patitions with with mdbox Message-ID: <201111081519.05867.p.heinlein@heinlein-support.de> Having > 10 TByte mailstore filesystem-checks takes too much time. At the moment we have four different partitions, but I don't like to set symlinks or LDAP-flags to sort customers and their domains to there individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate the path automatically. How do you handle >> 10 TB mailstore? I'm very interested in the feature "alternative mailstore" with mdbox, because that makes it very easy to use at least TWO filesystems without any tricky configuration. I think I'd love to have alternative mailstores. Why does dbox doesn't look for its m.*-files in more then two directorys? Sure, looking in 4 directorys would lead to four disc operations, but maybe it could be very helpful. Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Tue Nov 8 16:22:24 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:22:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EB9398B.1000904@us.es> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EB9398B.1000904@us.es> Message-ID: <201111081522.24386.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodr?guez: Hi, > If you have CPU to spare, consider using zlib with mdbox. You are > trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Hey. This point is great. I hadn't realized that. Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks. > compressed) and backup software is happier because there are few > (100.000+ files with mdbox) to backup instead of several millions > (Maildir) Yes, that#s the main reason why I want to switch to mbox. At the moment our roundtrip-time for the backup is > 24h... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From javierdemiguel at us.es Tue Nov 8 16:29:02 2011 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_de_Miguel_Rodr=EDguez?=) Date: Tue, 08 Nov 2011 15:29:02 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081522.24386.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EB9398B.1000904@us.es> <201111081522.24386.p.heinlein@heinlein-support.de> Message-ID: <4EB93CAE.3080308@us.es> Other important thing to consider is message expunging. With mdbox you are "delaying" the I/O associated with deleting e-mails. We have a nightly cronjob that expunge messages from mdboxes. If you have en EVA (wich one? 4.400? 6.400? ) you also can consider RAID 1+0 or SSD for indexes. Indexes are hammered in mdbox. Regards Javier > Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodr?guez: > > > Hi, > >> If you have CPU to spare, consider using zlib with mdbox. You are >> trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). > Hey. This point is great. I hadn't realized that. > > Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks. > >> compressed) and backup software is happier because there are few >> (100.000+ files with mdbox) to backup instead of several millions >> (Maildir) > Yes, that#s the main reason why I want to switch to mbox. At the moment our > roundtrip-time for the backup is> 24h... > > > Peer > > From p.heinlein at heinlein-support.de Tue Nov 8 16:34:51 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:34:51 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <201111081534.52068.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: Hi, > I wonder how I can incrementally change over from Maildir to mdbox? If you have double diskspace: Just use "dsync mirror" in the background to prepare the change. After that it's just a short downtime to migrate incremental the last changes, or it's just a question of a short login-script. if [ -d ~/Maildir ] ; then dsync mirror voodoo-magic rm -R ~/Maildr fi > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or > vice versa). I wonder about that problem too. Even the last-last-last-quick sync would be so much IO, that I can't handle it in realtime in the morning at 9 a.m. Looks like a nightly downtime for the last incremental run. It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes. In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So having users with maildir and mdbox mixed up will break their shared folders... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Tue Nov 8 16:36:38 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:36:38 +0100 Subject: [Dovecot] dsync and zlib Message-ID: <201111081536.38710.p.heinlein@heinlein-support.de> Will dsync also use zlib-compression when copying mails from one side to another? Wouldn't this be a good way to compress existing mails? http://wiki2.dovecot.org/Plugins/Zlib says, there's no way to do that. But dsync should respect the zlib-Plugin... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From jester at arcor.de Tue Nov 8 16:42:45 2011 From: jester at arcor.de (Oliver Eales) Date: Tue, 08 Nov 2011 15:42:45 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <4EB93FE5.8020903@arcor.de> On 08.11.2011 15:16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! I never tried it, but it should be possible to provide the mail_location from the user repsoitory (LDAP, SQL, whatever) So you can keep your global config, and use a script to convert one mailbox after another, and add a mail_location extra userdb field in the user repository to overwrite the global setting on a per-user-basis. Regards, Oliver From Ralf.Hildebrandt at charite.de Tue Nov 8 16:44:55 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 8 Nov 2011 15:44:55 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <20111108144455.GG27296@charite.de> * Peer Heinlein : > Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: > > Hi, > > > I wonder how I can incrementally change over from Maildir to mdbox? > > If you have double diskspace: haha :) no. I thought of a per-user migration, that way I don't need extra space. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Nov 8 16:57:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 16:57:42 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <1F29E1C1-7D8F-49F6-AF01-7B50D69F78DB@iki.fi> On 8.11.2011, at 16.34, Peer Heinlein wrote: >> I can of course use dsync to mirror Maildir: to mdbox:, but how can I >> make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or >> vice versa). > > I wonder about that problem too. Even the last-last-last-quick sync would be > so much IO, that I can't handle it in realtime in the morning at 9 a.m. > > Looks like a nightly downtime for the last incremental run. > > It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP > attributes. Easy! > In this case the whole migration process could be split up into > groups. Unfortunately we have shared folders and I don't know a way to read > the *remote* mailbox-format from LDAP... So having users with maildir and > mdbox mixed up will break their shared folders... Not so easy.. Only the home directory can be currently looked up from userdb for shared folders. There is also automatic detection of Maildir and mbox when mail_location isn't set, but no such code for mdbox. It could be added without much trouble though. But for shared folders, assuming you'd want per-user \seen flags, it would also need something like: mail_location = auto::INDEX=~/shared-indexes This "auto" doesn't exist yet either. And then there's the biggest problem: You can't have per-user \seen flags with mdbox, because you can't change the index file path without breaking mdbox. From tss at iki.fi Tue Nov 8 16:59:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 16:59:03 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <877AD4C4-F8A0-48F5-812D-5AAC6FCF13C2@iki.fi> On 8.11.2011, at 16.16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! > > I wonder how I can incrementally change over from Maildir to mdbox? > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or > vice versa). > > That would allow for a smooth transition... If you don't have shared folders (as explained in previous mail) and you can have per-user mail_location in the userdb, this should be pretty easy. The man page for dsync lists the steps that can be used for online migration. From tss at iki.fi Tue Nov 8 17:00:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:00:29 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> Message-ID: On 8.11.2011, at 16.03, Morten Stevens wrote: > We have switched our mailbox storage format from maildir to mdbox! > > Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly. > > Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4. You don't happen to have any specific numbers/graphs that can be used to compare maildir vs. mdbox in the same hardware? I'd be interested in seeing those, such as a graph of disk iops spanning a month before/after mdbox switch. From dirk.jahnke-zumbusch at desy.de Tue Nov 8 17:03:24 2011 From: dirk.jahnke-zumbusch at desy.de (Jahnke-Zumbusch, Dirk) Date: Tue, 8 Nov 2011 16:03:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EB93FE5.8020903@arcor.de> References: <201111081450.25798.p.heinlein@heinlein-support.de><7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com><20111108141612.GE27296@charite.de> <4EB93FE5.8020903@arcor.de> Message-ID: <7C0189D0CDB51646B7814F4377D356F501536694@ADXV4.win.desy.de> Hi there, >I never tried it, but it should be possible to provide the mail_location >from the user repsoitory (LDAP, SQL, whatever) Actually this works :-) Our userdb looks similar to: account1:xyz:000:000::/account1s/home/dir::userdb_mail=maildir:/account1s/home/dir/Maildir account2:xyz:000:000::/account2s/home/dir::userdb_mail=mdbox:/ account2s/home/dir http://wiki2.dovecot.org/UserDatabase/ExtraFields Concerning Maildir backups: what about a backup-to-disc-to-tape scheme using snapshots for the "to-disc" part and something like perpetual incrementals afterwards for the "top-tape" (secondary store) ? Regards Dirk -- Dirk Jahnke-Zumbusch Deutsches Elektronen-Synchrotron DESY IT Information Fabrics Member of the Helmholtz Association D-22603 Hamburg Notkestrasse 85 / 22607 Hamburg T: +49-40-899.81760 F: +49-40-899.41760 dirk.jahnke-zumbusch at desy.de >So you can keep your global config, and use a script to convert one >mailbox after another, and add a mail_location extra userdb field in the >user repository to overwrite the global setting on a per-user-basis. > >Regards, >Oliver From tss at iki.fi Tue Nov 8 17:04:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:04:07 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: On 8.11.2011, at 15.50, Peer Heinlein wrote: > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). A few more ideas for Maildir if you haven't done yet: - maildir_very_dirty_syncs = yes - pop3_no_flag_updates = yes - Switching to dict file quota instead of Maildir++ quota. From tss at iki.fi Tue Nov 8 17:10:11 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:10:11 +0200 Subject: [Dovecot] dsync and zlib In-Reply-To: <201111081536.38710.p.heinlein@heinlein-support.de> References: <201111081536.38710.p.heinlein@heinlein-support.de> Message-ID: On 8.11.2011, at 16.36, Peer Heinlein wrote: > Will dsync also use zlib-compression when copying mails from one side to > another? As long as the zlib plugin is globally enabled and zlib_save setting is set. > Wouldn't this be a good way to compress existing mails? > > http://wiki2.dovecot.org/Plugins/Zlib > > says, there's no way to do that. But dsync should respect the zlib-Plugin... Well .. You could think of compressing existing mails with dsync the same as "migrating from maildir" to "migrating to maildir with zlib enabled" :) I'll add a note to the wiki page. From tss at iki.fi Tue Nov 8 17:37:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:37:01 +0200 Subject: [Dovecot] Multiple Patitions with with mdbox In-Reply-To: <201111081519.05867.p.heinlein@heinlein-support.de> References: <201111081519.05867.p.heinlein@heinlein-support.de> Message-ID: <608C2C35-40AD-41C7-9924-9D4F0BE232A2@iki.fi> On 8.11.2011, at 16.19, Peer Heinlein wrote: > Having > 10 TByte mailstore filesystem-checks takes too much time. > > At the moment we have four different partitions, but I don't like to set > symlinks or LDAP-flags to sort customers and their domains to there > individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate > the path automatically. Why not symlinks? You could use e.g. /mail/%16Hu/%d/%n as part of the path which would give you one hex letter when hashing the username (0..9, a..f). So then you could create symlinks like: /mail/0 -> /storage1 /mail/1 -> /storage2 ..etc.. > I'm very interested in the feature "alternative mailstore" with mdbox, > because that makes it very easy to use at least TWO filesystems without any > tricky configuration. > > I think I'd love to have alternative mailstores. Why does dbox doesn't > look for its m.*-files in more then two directorys? Sure, looking in 4 > directorys would lead to four disc operations, but maybe it could be very > helpful. The current implementation uses a simple message flag to tell when/where to move the mail, so the code would have to get much more complicated with more alt paths. From michael.abbott at apple.com Tue Nov 8 18:43:17 2011 From: michael.abbott at apple.com (Mike Abbott) Date: Tue, 08 Nov 2011 10:43:17 -0600 Subject: [Dovecot] another sizeof tweak Message-ID: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { Should that be sizeof(*old_rec)? From tss at iki.fi Tue Nov 8 18:46:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 18:46:07 +0200 Subject: [Dovecot] another sizeof tweak In-Reply-To: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> References: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Message-ID: On 8.11.2011, at 18.43, Mike Abbott wrote: > Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: > memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { > Should that be sizeof(*old_rec)? Yep, looks like was fixed in v2.1 already. I'll fix it for v2.0 too. From rostetter at mail.utexas.edu Tue Nov 8 18:53:32 2011 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Tue, 08 Nov 2011 10:53:32 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> Quoting Peer Heinlein : > It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP > attributes. In this case the whole migration process could be split up into > groups. Unfortunately we have shared folders and I don't know a way to read > the *remote* mailbox-format from LDAP... So having users with maildir and > mdbox mixed up will break their shared folders... May not work for you, but... The way I did this when I migrated was to run two dovecot instances, and have perdition software on a front-end (could be on the same machine instead of a front-end, I just happen to have a front-end machine to do it). Perdition will query ldap for the info per user/connection, and send the connection to the correct dovecot instance based on the ldap lookup. Worked for me, your milage may vary... -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From jeroen at bizworx-it.nl Tue Nov 8 20:34:30 2011 From: jeroen at bizworx-it.nl (Jeroen Grusewski) Date: Tue, 8 Nov 2011 19:34:30 +0100 Subject: [Dovecot] Dovecot alias and mailbox problem Message-ID: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> Hi All, My Postfix / Dovecot / MySQL is running fine but I have the following issue: When I create the following mailboxes test at example.com and test1 at example.com it is working fine. I can create an alias test2 at examples.com => test1 at example.com that is also working without a problem, only when I create test at example.com => test1 at example.com email is only delivered to test1 at .... instead to both test at .... and test1 at .... I believe it is properly just a setting but please help me out here ! Let me know if you need any more information before I post configs etc. Best regards, Jeroen From tom at whyscream.net Tue Nov 8 20:54:46 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Tue, 08 Nov 2011 19:54:46 +0100 Subject: [Dovecot] Dovecot alias and mailbox problem In-Reply-To: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> References: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> Message-ID: <4EB97AF6.1020606@whyscream.net> On 08-11-11 19:34, Jeroen Grusewski wrote: > > Hi All, > > My Postfix / Dovecot / MySQL is running fine but I have the following issue: > > When I create the following mailboxes test at example.com and test1 at example.com it is working fine. > I can create an alias test2 at examples.com => test1 at example.com that is also working without a problem, > only when I create test at example.com => test1 at example.com email is only delivered to test1 at .... instead to > both test at .... and test1 at .... > > I believe it is properly just a setting but please help me out here ! > > Let me know if you need any more information before I post configs etc. > You probably want to create the alias as test at example.com => test1 at example.com,test at example.com. Depending on how you setup postfix, this should deliver to both mailboxes (and not create an infinite loop). Note that aliasing is handled within Postfix, not in Dovecot, so if you have further questions, you're probably on the wrong list. -- Regards, Tom From p.heinlein at heinlein-support.de Tue Nov 8 21:40:08 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 20:40:08 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> References: <201111081450.25798.p.heinlein@heinlein-support.de> <201111081534.52068.p.heinlein@heinlein-support.de> <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> Message-ID: <201111082040.08183.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 17:53:32 schrieb Eric Rostetter: > May not work for you, but... > > The way I did this when I migrated was to run two dovecot instances, and > have perdition software on a front-end (could be on the same machine > instead of a front-end, I just happen to have a front-end machine to do > it). You could do that with Dovecot, too. > Perdition will query ldap for the info per user/connection, and send the > connection to the correct dovecot instance based on the ldap lookup. > Worked for me, your milage may vary... The problem is: You're running in problems with shared folders. You can't read your neighbors storage-engine from ldap. It's easy to read the user's storage engine from ldap. So there's no need to use perdition for that :-) But you can't read or proxy the storage engine from somebody who shared you his folders. That's my problem :-( Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From rostetter at mail.utexas.edu Tue Nov 8 21:50:56 2011 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Tue, 08 Nov 2011 13:50:56 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111082040.08183.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <201111081534.52068.p.heinlein@heinlein-support.de> <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> <201111082040.08183.p.heinlein@heinlein-support.de> Message-ID: <20111108135056.312858jv1jl7bhgk@mail.ph.utexas.edu> Quoting Peer Heinlein : > The problem is: You're running in problems with shared folders. You can't > read your neighbors storage-engine from ldap. Yes, but I didn't have any shared folders, so it worked. Your milage may vary, as I said... :) -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From tss at iki.fi Tue Nov 8 23:09:52 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 08 Nov 2011 23:09:52 +0200 Subject: [Dovecot] another sizeof tweak In-Reply-To: References: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Message-ID: <1320786593.21919.345.camel@hurina> On Tue, 2011-11-08 at 18:46 +0200, Timo Sirainen wrote: > On 8.11.2011, at 18.43, Mike Abbott wrote: > > > Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: > > memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { > > Should that be sizeof(*old_rec)? > > Yep, looks like was fixed in v2.1 already. I'll fix it for v2.0 too. Actually it doesn't matter in v2.0 since this is mailbox list indexing code, which is forcibly disabled (and broken) in v2.0. From tss at iki.fi Wed Nov 9 00:35:37 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 00:35:37 +0200 Subject: [Dovecot] v2.1.beta1 released Message-ID: <1320791738.21919.350.camel@hurina> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig Here's the first beta release of Dovecot v2.1. This version has already been tested quite a lot, so I'm not expecting any major bugs. So please upgrade and see if you can find any problems. I'm optimistic about getting rc1 released this year and perhaps even v2.1.0. Since alpha2 there have been a lot of fixes, especially to imapc backend. There are probably also some small new features, but nothing huge. As a reminder, here's the largest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now From stephan at rename-it.nl Wed Nov 9 01:37:05 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 09 Nov 2011 00:37:05 +0100 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320791738.21919.350.camel@hurina> References: <1320791738.21919.350.camel@hurina> Message-ID: <4EB9BD21.6060501@rename-it.nl> On 11/8/2011 11:35 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig > > Here's the first beta release of Dovecot v2.1. This version has already > been tested quite a lot, so I'm not expecting any major bugs. So please > upgrade and see if you can find any problems. I'm optimistic about > getting rc1 released this year and perhaps even v2.1.0. > Ok, I guess it is time for a Pigeonhole release for Dovecot v2.1. I'm a bit swamped this week, but next weekend I should be able to compose one. For now you can use the dovecot-2.0-pigeonhole hg and patch it with http://hg.rename-it.nl/pigeonhole-0.2-dovecot-2.1-patches/file/tip/pigeonhole-0.2-dovecot-2.1.patch to make it compile against v2.1. Regards, Stephan. From stan at hardwarefreak.com Wed Nov 9 09:34:04 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 01:34:04 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EBA2CEC.1040701@hardwarefreak.com> On 11/8/2011 7:50 AM, Peer Heinlein wrote: > I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. That's a lot of mail (likely a large user base--not given), on a filesystem not designed for such, on a decent SAN controller--LUN RAID configuration not given. > I always wanted to make some mesurements about several influences to the > performance (switch to ext4, switch to mdbox), but I never had enough time > to do that. If you're going to switch filesystems, for this size dataset and concurrent workload, you're moving in the wrong direction. > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). EXT3/4 are not designed, nor optimized, for high concurrency workloads. > I have to change my setup, maybe somebody else here have hard facts: > > *) Is ext4 faster? How much faster? Simulated maildir workload test on 2.6.35-rc5, 128 threads (No data published for newer kernels): http://btrfs.boxacle.net/repository/raid/2.6.35-rc5/2.6.35-rc5/2.6.35-rc5_Mail_server_simulation._num_threads=128.html As you can see EXT4 shows a small gain over EXT3, ~20%. If you really want high performance it's time to move to XFS, properly configured to match the underlying RAID characteristics of the LUN(s) you're mounting. You'll prefer kernel 2.6.39+, 2.6.36 at minimum, so you get the delayed logging feature (2.6.35 had delayed logging but had problems in other areas). I'll assume with a >10TB mail store that you're seeing greater than 128 concurrent user operations regularly. As you can see from the graph, XFS will give you ~50% greater ops/s than EXT4 and ~90% greater than EXT3--yes, almost double that of EXT3. As the concurrency increases, so will this performance gap, as XFS was designed from day 1 for high concurrency workloads. This is a simulated mail server benchmark. However you should see similar gains with Dovecot. The XFS delayed logging feature will dramatically reduce the number of physical IOs required for journal writes (i.e. metadata IO), as will delayed allocation, a feature of XFS since its inception in 1994. EXT4 was the first of its lineage to gain delayed allocation, some 10+ years later, after Ted T'so studied the XFS code. In short, if you want an 'enterprise caliber' production Linux filesystem tailor made for high IO concurrency, XFS is it. JFS yields similar performance, but hasn't been actively developed for 8 years or so. XFS has substantial ongoing feature and fix development. > *) Is it faster because of the ext4 kernel-module (which can be used on ext3 > to) or because of the ext4 filesystem layout? AIUI, the bulk of the EXT4 performance advantage over EXT3 is the delayed allocation logic. The new EXT4 extent based on disk layout yields little in the way of additional performance, but much in free space management, fragmentation mitigation, etc. > *) Is mdbox really faster? I'd like to have mdbox to have better performance > in running my backup-processes. But does it bring some performance boosts > to? mdbox will substantially decrease physical IOs to your storage back end due to dramatically less metadata operations compared to maildir. You've stated you currently have a storage IOPS bottleneck, so I'd have to assume mdbox will seriously increase your overall performance. Good old mbox will do so as well, but everyone shuns it for various reasons, some valid, some not so valid. If you have an appropriate LUN available (sufficient size and spindle speed/count of member disks), properly create an XFS filesystem on it (read much before creating it), and moved to mdbox atop that, I think you'll be really surprised by how much you gain from simply changing filesystems and mailbox storage formats. If you double the size of the LUN you could potentially carry twice as many users with, fewer IOPS than you're seeing now, on essentially the same hardware platform. -- Stan From ghandidrivesahumvee at rocketfish.com Wed Nov 9 09:40:14 2011 From: ghandidrivesahumvee at rocketfish.com (Dovecot-GDH) Date: Tue, 8 Nov 2011 23:40:14 -0800 Subject: [Dovecot] Multiple Patitions with with mdbox In-Reply-To: <201111081519.05867.p.heinlein@heinlein-support.de> References: <201111081519.05867.p.heinlein@heinlein-support.de> Message-ID: > How do you handle >> 10 TB mailstore? ZFS: no need to fsck. GlusterFS: "always-online". On Nov 8, 2011, at 6:19 AM, Peer Heinlein wrote: > > Having > 10 TByte mailstore filesystem-checks takes too much time. > > At the moment we have four different partitions, but I don't like to set > symlinks or LDAP-flags to sort customers and their domains to there > individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate > the path automatically. > > How do you handle >> 10 TB mailstore? > > I'm very interested in the feature "alternative mailstore" with mdbox, > because that makes it very easy to use at least TWO filesystems without any > tricky configuration. > > I think I'd love to have alternative mailstores. Why does dbox doesn't > look for its m.*-files in more then two directorys? Sure, looking in 4 > directorys would lead to four disc operations, but maybe it could be very > helpful. > > Peer > -- > > Heinlein Professional Linux Support GmbH > Linux: Akademie - Support - Hosting > http://www.heinlein-support.de > > Tel: 030/405051-42 > Fax: 030/405051-19 > > Zwangsangaben lt. ?35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From thomas at koch.ro Wed Nov 9 11:29:37 2011 From: thomas at koch.ro (Thomas Koch) Date: Wed, 9 Nov 2011 10:29:37 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT Message-ID: <201111091029.37963.thomas@koch.ro> Hi, somebody from the kolab groupware project recently explained me that the Bus- Factor[1] of around 1 would be one of their primary reasons not to use Dovecot and stick with Cyrus. What do you think about that? Is the bus factor much higher then 1? [1] http://en.wikipedia.org/wiki/Bus_factor Somehow related: Since the free software world[2] slowly converges towards GIT as the "one and only" VCS, have you thought about a switch? It's much more likely that somebody checks out your code, looks into it and provides patches if the VCS is already installed. [2] Android, Debian, Drupal, Eclipse, Fedora, Gnome, KDE, Linux Kernel, Perl, PHP (ongoing), PostgreSQL, Qt, Ruby on Rails, X.org (freedesktop.org) Best regards, Thomas Koch, http://www.koch.ro From sandro.tosi at register.it Wed Nov 9 11:45:12 2011 From: sandro.tosi at register.it (Sandro Tosi) Date: Wed, 09 Nov 2011 10:45:12 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <4EBA4BA8.1080403@register.it> On 11/09/2011 10:29 AM, Thomas Koch wrote: > Somehow related: Since the free software world[2] slowly converges towards GIT > as the "one and only" VCS, have you thought about a switch? It's much more > likely that somebody checks out your code, looks into it and provides patches > if the VCS is already installed. > > [2] Android, Debian, Drupal, Eclipse, Fedora, Gnome, KDE, Linux Kernel, Perl, > PHP (ongoing), PostgreSQL, Qt, Ruby on Rails, X.org (freedesktop.org) I don't know where you inferred that, but for sure Debian is not converging to git; we have that VCS as many others and none is the preferred/superior. Please try to balance what you say with actual facts: backing your reasoning with partial data is misleading for others not knowing the env you're talking about. Regards, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From p.heinlein at heinlein-support.de Wed Nov 9 11:47:04 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Wed, 9 Nov 2011 10:47:04 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT Message-ID: <201111091047.04570.p.heinlein@heinlein-support.de> Am Mittwoch, 9. November 2011, 10:29:37 schrieb Thomas Koch: Hi, > somebody from the kolab groupware project recently explained me that the > Bus- Factor[1] of around 1 would be one of their primary reasons not to > use Dovecot and stick with Cyrus. The problem is: Even it Timo will be hitten by a bus, everybody could stay on the actual Dovecot-version and everything's fine. Cyrus will IMO never reach the level, that Dovecot already has. :-) Peer P.S.: Timo! Don't leave your house. It's too dangerous for you. -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From david.ocana at bsc.es Wed Nov 9 11:54:28 2011 From: david.ocana at bsc.es (David Ocana) Date: Wed, 09 Nov 2011 10:54:28 +0100 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox Message-ID: <4EBA4DD4.4080505@bsc.es> Hello, I've been trying to set up dovecot 2.0.15, everything seems to work pretty well except for the quota feature. I would like to set a quota limit only for the Inbox folder. I configured two namespaces, according to some posts from Timo Sirainen (http://dovecot.org/list/dovecot/2006-July/014530.html) & (http://www.dovecot.org/list/dovecot/2011-January/056131.html) but I can't configure the quota plugin to act the way I pretend. Here's my namespace configuration: namespace { separator = / prefix = INBOX/ location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n inbox = yes hidden = yes } namespace { separator = / prefix = inbox = no location = mbox:/mail/%d/MAILBOXES/%n:INDEX=/var/dovecot/%d/%n } Quota config: plugin { quota = dirsize:User quota # I've tried with: quota_rule = INBOX:storage=819200K quota_rule = INBOX/*:storage=819200K quota_rule = INBOX/Inbox:storage=819200K # Works with: quota_rule = *:storage=819200K quota_rule = ?:storage=819200K (Same behavior as '*') } Using '*' and '?', is the only way I get it to work, but obviously, quota is calculated summing up all the mailbox folders. I don't know what else to change, any ideas? Is the two namespace configuration correct? Thanks in advance. David -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4439 bytes Desc: S/MIME Cryptographic Signature URL: From thomas at koch.ro Wed Nov 9 12:05:42 2011 From: thomas at koch.ro (Thomas Koch) Date: Wed, 9 Nov 2011 11:05:42 +0100 Subject: [Dovecot] GIT in Debian In-Reply-To: <4EBA4BA8.1080403@register.it> References: <201111091029.37963.thomas@koch.ro> <4EBA4BA8.1080403@register.it> Message-ID: <201111091105.42746.thomas@koch.ro> Hi Sandro, I've been asking myself whether I should include Debian in the list. You're right that there are also other VCS' used in Debian. Zack's statistics say: arch 22 bzr 271 cvs 31 darcs 382 git 5230 hg 63 mtn 13 svn 4843 http://upsilon.cc/~zack/stuff/vcs-usage/ The only open question is, towards which DVCS system the 4843 subversion packages will eventually migrate. I'd bet several rounds of beer on GIT. I believe that the recent migration of 2110 packages from the perl team from svn to git is not yet fully represented in the above numbers. But we shouldn't annoy the dovecot list with Debian details any longer. Regards, Thomas Koch, http://www.koch.ro From adrian.minta at gmail.com Wed Nov 9 14:59:17 2011 From: adrian.minta at gmail.com (Adrian M) Date: Wed, 9 Nov 2011 14:59:17 +0200 Subject: [Dovecot] Quota BUG ? Message-ID: Hello list, I believe I found a bug with quota. The dovecot quota replies with incorrect values when the mailbox is above 2GB. du -ks returns 2005764bytes dovecot IMAP on the same user/directory: 1 getquotaroot inbox * QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1096 2000000)* QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1096 2000000) 1 OK Getquotaroot completed. Is there any way to fix this ? --- linux system: dovecot 1.2.15-7 linux 2.6.32-5-amd64 dovecot.conf: dict { quotadict= mysql:/etc/dovecot/dovecot-dict-quota.conf } plugin { quota = dict:user::proxy::quotadict } dovecot-dict-quota.conf : user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' From maria_arrea at gmx.com Wed Nov 9 15:19:54 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Wed, 09 Nov 2011 14:19:54 +0100 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs Message-ID: <20111109131954.26480@gmx.com> Hello We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy cheap sata-based iscsi storage / sata-NFS. We want to evaluate how much space would be needed in alternate storage BEFORE buying it. I have been reading about doveadm altmove. We want to move ALL messages older than a year to alternate storage, what doveadm-script-fu could I do to identify how much space would go from a user to alternate storage? Regards Maria From tss at iki.fi Wed Nov 9 15:53:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 15:53:08 +0200 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <1320846789.21919.361.camel@hurina> On Wed, 2011-11-09 at 10:29 +0100, Thomas Koch wrote: > somebody from the kolab groupware project recently explained me that the Bus- > Factor[1] of around 1 would be one of their primary reasons not to use Dovecot > and stick with Cyrus. > > What do you think about that? Is the bus factor much higher then 1? I've created a company for Dovecot support, and if all goes well we should have at least one other coder in not too distant future (anyone want a job? :) Also besides me there's already at least Stephan Bosch who has written Dovecot's Sieve/ManageSieve implementations. I don't know if he'd take care of the whole Dovecot if I happened to die right now, but at least he knows the code pretty well. There are also a few big companies that have some people who have done some Dovecot coding. Also the Bus-Factor of Cyrus doesn't seem to be much higher than 1 to me. AFAIK there's only a single person currently developing it actively (plus I guess a few more not-very-active developers from CMU). > Somehow related: Since the free software world[2] slowly converges towards GIT > as the "one and only" VCS, have you thought about a switch? It's much more > likely that somebody checks out your code, looks into it and provides patches > if the VCS is already installed. I'm not as much against git anymore as I was when I switched to hg, but I don't see much benefits in switching to git either. I highly doubt I'd get even a single patch more if I used git instead of hg. The biggest problem with lack of patches is that few people are interested in coding a mail server. You can see the same with all open source IMAP/SMTP servers (and probably commercial ones too). Nearly always there's only a single guy who has written almost all of it. From achekalin at lazurit.com Wed Nov 9 15:57:11 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 09 Nov 2011 16:57:11 +0300 Subject: [Dovecot] Please advise on very fast search Message-ID: <4EBA86B7.2080201@lazurit.com> Hello, I try to create some kind of mail backup system. What I need is system that will store mail for the whole domain, and allow me to restore messages from/to specified email at that domain. The scheme is pretty simple: on our main mail server the SMTP server itself has a rule to send a copy of every message to 'backup at backupserver.host', and the backupserver.host domain is placed nearby on second server. The SMTP on second server do simple 'catchall' redirect of all messages to the single box. There is also a Dovecot that takes care for remote IMAP access to that box. And, finally, I've create some scripts to sort all messages in INBOX to folders named after message's date. So I have a lot of mailboxes inside the catchall box: INBOX 2011.11.03 2011.11.04 2011.11.05 2011.11.06 ...etc... and each folder holds messages for that day. Simply, and works perfectly. The problem is that when my archive become big (several years), it appears to be painful to find specified message(s). When someone suddenly needs to find his/her old message, it is mostly guesses like 'I think the message was between june and july of 2009, or maybe month or two before that', so I need to search all mailboxes (with 1000's messages in each). And it takes really long time. I tried to play with Dovecot indexes, but it won't help too much. The bad part is that I need to search for all emails in each message headers, not only for "From" or "To", since some messages are sent to maillists soe "To" = list address, not person's personal email. Then I tried to index messages on my own, storing info on emails into MySQL database ('email' -> 'mailbox', 'message filename'), but soon I find out that message files can be renamed by Dovecot. Could you please advice me how to speed up message search? Sorry for such a long question, hope you can help! Yours, Alexander Chekalin From tss at iki.fi Wed Nov 9 15:58:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 15:58:42 +0200 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs In-Reply-To: <20111109131954.26480@gmx.com> References: <20111109131954.26480@gmx.com> Message-ID: <1320847122.21919.365.camel@hurina> On Wed, 2011-11-09 at 14:19 +0100, Maria Arrea wrote: > We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy > cheap sata-based iscsi storage / sata-NFS. We want to evaluate how > much space would be needed in alternate storage BEFORE buying it. I > have been reading about doveadm altmove. We want to move ALL messages > older than a year to alternate storage, what doveadm-script-fu could I > do to identify how much space would go from a user to alternate > storage? doveadm -A -f tab fetch size.physical before 1y would print all the message sizes. You could then sum them up with some scripting to produce the total. Unfortunately they are the uncompressed sizes, but maybe simply dividing by 2 would give approximate size? :) (You could get the compressed sizes from dovecot.map.index files with doveadm dump and mapping their contents to the older than 1 year message UIDs, but this gets difficult.) From robert at schetterer.org Wed Nov 9 16:14:21 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 09 Nov 2011 15:14:21 +0100 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBA86B7.2080201@lazurit.com> References: <4EBA86B7.2080201@lazurit.com> Message-ID: <4EBA8ABD.8010804@schetterer.org> Am 09.11.2011 14:57, schrieb Alexander Chekalin: > Hello, > > I try to create some kind of mail backup system. What I need is system > that will store mail for the whole domain, and allow me to restore > messages from/to specified email at that domain. > > The scheme is pretty simple: on our main mail server the SMTP server > itself has a rule to send a copy of every message to > 'backup at backupserver.host', and the backupserver.host domain is placed > nearby on second server. > > The SMTP on second server do simple 'catchall' redirect of all messages > to the single box. There is also a Dovecot that takes care for remote > IMAP access to that box. And, finally, I've create some scripts to sort > all messages in INBOX to folders named after message's date. > > So I have a lot of mailboxes inside the catchall box: > INBOX > 2011.11.03 > 2011.11.04 > 2011.11.05 > 2011.11.06 > ...etc... > > and each folder holds messages for that day. Simply, and works perfectly. > > The problem is that when my archive become big (several years), it > appears to be painful to find specified message(s). When someone > suddenly needs to find his/her old message, it is mostly guesses like 'I > think the message was between june and july of 2009, or maybe month or > two before that', so I need to search all mailboxes (with 1000's > messages in each). And it takes really long time. > > > I tried to play with Dovecot indexes, but it won't help too much. The > bad part is that I need to search for all emails in each message > headers, not only for "From" or "To", since some messages are sent to > maillists soe "To" = list address, not person's personal email. > > Then I tried to index messages on my own, storing info on emails into > MySQL database ('email' -> 'mailbox', 'message filename'), but soon I > find out that message files can be renamed by Dovecot. > > Could you please advice me how to speed up message search? > > > Sorry for such a long question, hope you can help! > > Yours, > Alexander Chekalin > guess youre searching over imap ? perhaps compression will help for speed up, and many other speed related stuff, or you need some other idea of indexing at last if its maildir how fast is "grep" etc...and so on some ideas here http://wiki.dovecot.org/HowTo/ReadOnlyArchive etc anyway , i think you really need another kind of archive solution in Germany there is a law that you need to archive some kind of business mails up to 10 years for finance and other review, so there are a lot of "you can by" solutions now, these have solved the problems you discovered ( indexing etc ) i was shown i.e http://www.bytstormail.de which looked fine to me or perhaps you might have a look http://www.archiveopteryx.org/ here too -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From maria_arrea at gmx.com Wed Nov 9 16:24:14 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Wed, 09 Nov 2011 15:24:14 +0100 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs Message-ID: <20111109142414.26470@gmx.com> Timo, maybe you have mispelled the command? > We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy > cheap sata-based iscsi storage / sata-NFS. We want to evaluate how > much space would be needed in alternate storage BEFORE buying it. I > have been reading about doveadm altmove. We want to move ALL messages > older than a year to alternate storage, what doveadm-script-fu could I > do to identify how much space would go from a user to alternate > storage? doveadm -A -f tab fetch size.physical before 1y would print all the message sizes. You could then sum them up with some scripting to produce the total. Unfortunately they are the uncompressed sizes, but maybe simply dividing by 2 would give approximate size? :) (You could get the compressed sizes from dovecot.map.index files with doveadm dump and mapping their contents to the older than 1 year message UIDs, but this gets difficult.) From tss at iki.fi Wed Nov 9 17:15:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 17:15:23 +0200 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs In-Reply-To: <20111109142414.26470@gmx.com> References: <20111109142414.26470@gmx.com> Message-ID: <1320851723.21919.367.camel@hurina> On Wed, 2011-11-09 at 15:24 +0100, Maria Arrea wrote: > Timo, maybe you have mispelled the command? Minor changes, I tested that this actually works: doveadm -f tab fetch -A size.physical before 365d The -A parameter requires that you use a userdb that supports iteration. With SQL you need to provide the iteration_query. From tss at iki.fi Wed Nov 9 17:17:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 17:17:31 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBA86B7.2080201@lazurit.com> References: <4EBA86B7.2080201@lazurit.com> Message-ID: <1320851851.21919.369.camel@hurina> On Wed, 2011-11-09 at 16:57 +0300, Alexander Chekalin wrote: > The problem is that when my archive become big (several years), it > appears to be painful to find specified message(s). When someone > suddenly needs to find his/her old message, it is mostly guesses like 'I > think the message was between june and july of 2009, or maybe month or > two before that', so I need to search all mailboxes (with 1000's > messages in each). And it takes really long time. > > > I tried to play with Dovecot indexes, but it won't help too much. They'll help with the dates. > The > bad part is that I need to search for all emails in each message > headers, not only for "From" or "To", since some messages are sent to > maillists soe "To" = list address, not person's personal email. Headers only, not message body? Anyway, some of the full text search backends would support searching from both. I'd recommend using either Solr or with Dovecot v2.1 you can also use Lucene: http://wiki2.dovecot.org/Plugins/FTS From odhiambo at gmail.com Wed Nov 9 17:34:45 2011 From: odhiambo at gmail.com (Odhiambo Washington) Date: Wed, 9 Nov 2011 18:34:45 +0300 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320791738.21919.350.camel@hurina> References: <1320791738.21919.350.camel@hurina> Message-ID: On Wed, Nov 9, 2011 at 01:35, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig > > Here's the first beta release of Dovecot v2.1. This version has already > been tested quite a lot, so I'm not expecting any major bugs. So please > upgrade and see if you can find any problems. I'm optimistic about > getting rc1 released this year and perhaps even v2.1.0. > > Since alpha2 there have been a lot of fixes, especially to imapc > backend. There are probably also some small new features, but nothing > huge. > > As a reminder, here's the largest changes since v2.0: > > * Plugins now use UTF-8 mailbox names rather than mUTF-7: > acl, autocreate, expire, trash, virtual > * auth_username_format default changed to %Lu. If you really want > case sensitive usernames, set it back to empty. > * Solr full text search backend changed to use mailbox GUIDs > instead of > mailbox names, requiring reindexing everything. solr_old backend > can > be used with old indexes to avoid reindexing, but it doesn't > support > some newer features. > > + imapc (= IMAP client) storage allows using a remote IMAP server to > be used as storage. This allows using Dovecot as a smart (caching) > proxy or using dsync to do migration from remote IMAP server. > + Mailbox indexing via queuing indexer service (required for Lucene) > + Lucene full text search (FTS) backend rewritten with support for > different languages > + FTS finally supports "OR" search operation > + FTS supports indexing attachments via external programs > + IMAP FUZZY extension, supported by Lucene and Solr FTS backends > + Mailbox list indexes > + Statistics tracking via stats service. Exported via doveadm stats. > + Autocreate plugin creates/subscribes mailboxes physically only > when > the mailbox is opened for the first time. Mailbox listing shows > the > autocreated mailboxes even if they don't physically exist. > + Password and user databases now support default_fields and > override_fields settings to specify template defaults/overrides. > - listescape plugin works perfectly now > > I have been running the alpha for some time now. I have today tried compiling this -beta, but I saw so many warning about linking libraries, which I ignored, but it did fail with: Making all in wiki make: don't know how to make 2b. Stop *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1/doc. *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1. *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From achekalin at lazurit.com Wed Nov 9 18:16:39 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 09 Nov 2011 19:16:39 +0300 Subject: [Dovecot] Please advise on very fast search Message-ID: <4EBAA767.6060609@lazurit.com> Thanks, Robert, will take a look at. What I'm afraid for is how database storage should be planned (storage, CPU, RAM, scaling when will be over-filled). When dealing with files (I'm using maildir), it is much easy to understand and to fix just about everything. Adding database involves tune it up too, and I'll have more points of 'tune it a bit' In fact work with Dovecot is pretty nice, but I think I can tune it to work faster. I now run it on FreeBSD (on UFS2), maybe I should change OS + FS, but need to test (really hope ZFS disks on SAS drives will help; still find no benchmarks on such a setup). Will also try to use full text search, but afraid of index size (and I need no search on body, just on headers). Anyway thank your for pointing me in right directions! Yours, Alexander From tss at iki.fi Wed Nov 9 18:37:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 18:37:14 +0200 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: References: <1320791738.21919.350.camel@hurina> Message-ID: <1320856634.21919.371.camel@hurina> On Wed, 2011-11-09 at 18:34 +0300, Odhiambo Washington wrote: > I have today tried compiling this -beta, but I saw so many warning about > linking libraries, which I ignored, Probably plugin warnings which you can safely ignore. > Making all in wiki > make: don't know how to make 2b. Stop Looks like there's some weird file went into wiki docs, which some makes don't like. You could just find the line containing "2b" from doc/wiki/Makefile and delete it. From tss at iki.fi Wed Nov 9 18:40:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 18:40:04 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBAA767.6060609@lazurit.com> References: <4EBAA767.6060609@lazurit.com> Message-ID: <1320856804.21919.372.camel@hurina> On Wed, 2011-11-09 at 19:16 +0300, Alexander Chekalin wrote: > Will also try to use full text search, > but afraid of index size (and I need no search on body, just on headers). It wouldn't be difficult to patch Dovecot to skip indexing message bodies. Of course then you'd need to remember to keep applying the patch when updating. From tss at iki.fi Wed Nov 9 19:56:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 19:56:18 +0200 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <4EBA4DD4.4080505@bsc.es> References: <4EBA4DD4.4080505@bsc.es> Message-ID: <1320861378.21919.376.camel@hurina> On Wed, 2011-11-09 at 10:54 +0100, David Ocana wrote: > I've been trying to set up dovecot 2.0.15, everything seems to work > pretty well except for the quota feature. I would like to set a quota > limit only for the Inbox folder. I configured two namespaces, > according to some posts from Timo Sirainen > > namespace { > separator = / > prefix = INBOX/ > location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n > inbox = yes > hidden = yes > } > > plugin { > quota = dirsize:User quota quota = dirsize:User quota:ns=INBOX/ This limits the quota only to mailboxes in INBOX/ namespace. > # I've tried with: > quota_rule = INBOX:storage=819200K > quota_rule = INBOX/*:storage=819200K > quota_rule = INBOX/Inbox:storage=819200K Quota rules don't work in this way. There are no per-mailbox quotas really, at least in the way you're thinking about. From tss at iki.fi Wed Nov 9 19:59:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 19:59:42 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: References: Message-ID: <1320861585.21919.378.camel@hurina> On Wed, 2011-11-09 at 14:59 +0200, Adrian M wrote: > Hello list, > I believe I found a bug with quota. > The dovecot quota replies with incorrect values when the mailbox is above 2GB. > > du -ks returns 2005764bytes > > dovecot IMAP on the same user/directory: > 1 getquotaroot inbox > * QUOTAROOT "INBOX" "user" > * QUOTA "user" (STORAGE 1096 2000000)* QUOTAROOT "INBOX" "user" > * QUOTA "user" (STORAGE 1096 2000000) > 1 OK Getquotaroot completed. So it wraps. > dovecot.conf: > dict { > quotadict= mysql:/etc/dovecot/dovecot-dict-quota.conf > } > plugin { > quota = dict:user::proxy::quotadict > } > > dovecot-dict-quota.conf : > user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as > mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS > quota_rule FROM mailbox WHERE username = '%u' AND active = '1' This is dovecot-sql.conf, not dovecot-dict-quota.conf (or alternatively you've named them in a "wrong" way). Anyway, here's a guess: the quota is stored in mysql in "integer", which has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also use "integer". Replace that with "bigint" and recalculate quota and it should work. From adrian.minta at gmail.com Wed Nov 9 20:35:42 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 20:35:42 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320861585.21919.378.camel@hurina> References: <1320861585.21919.378.camel@hurina> Message-ID: <4EBAC7FE.5060207@gmail.com> On 11/09/11 19:59, Timo Sirainen wrote: > Anyway, here's a guess: the quota is stored in mysql in "integer", which > has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also > use "integer". Replace that with "bigint" and recalculate quota and it > should work. > Actually the mysql part is OK: quota | bigint(20) As you can see the value readed from mysql ( 2000000bytes) is also OK: * QUOTA "user" (STORAGE 1096 2000000) The problem is the 1096 number calculated internally by dovecot. By tweaking the SQL querry I coud put any value instead of 2000000, but I was unable to change 1096 part. -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From tss at iki.fi Wed Nov 9 20:41:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 20:41:23 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <4EBAC7FE.5060207@gmail.com> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> Message-ID: <1320864083.21919.379.camel@hurina> On Wed, 2011-11-09 at 20:35 +0200, Adrian Minta wrote: > On 11/09/11 19:59, Timo Sirainen wrote: > > Anyway, here's a guess: the quota is stored in mysql in "integer", which > > has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also > > use "integer". Replace that with "bigint" and recalculate quota and it > > should work. > > > Actually the mysql part is OK: > quota | bigint(20) > > As you can see the value readed from mysql ( 2000000bytes) is also OK: 2000000 kilobytes is the quota limit. > * QUOTA "user" (STORAGE 1096 2000000) > > The problem is the 1096 number calculated internally by dovecot. By > tweaking the SQL querry I coud put any value instead of 2000000, but I > was unable to change 1096 part. Is the quota in the database 1096 or >2GB? i.e. is the problem with reading it, or updating it? In general the quota code uses 64bit integers everywhere so this shouldn't be happening. From adrian.minta at gmail.com Wed Nov 9 20:44:31 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 20:44:31 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320864083.21919.379.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> Message-ID: <4EBACA0F.1010200@gmail.com> > Is the quota in the database 1096 or>2GB? i.e. is the problem with > reading it, or updating it? In general the quota code uses 64bit > integers everywhere so this shouldn't be happening. > > In the database quota field is 2048000000. -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From tss at iki.fi Wed Nov 9 20:46:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 20:46:02 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <4EBACA0F.1010200@gmail.com> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> Message-ID: <1320864362.21919.380.camel@hurina> On Wed, 2011-11-09 at 20:44 +0200, Adrian Minta wrote: > > Is the quota in the database 1096 or>2GB? i.e. is the problem with > > reading it, or updating it? In general the quota code uses 64bit > > integers everywhere so this shouldn't be happening. > > > > > In the database quota field is 2048000000. In the dict quota value? That looks more like the quota limit, not the current quota usage. From toddr at cpanel.net Wed Nov 9 20:57:39 2011 From: toddr at cpanel.net (Todd Rinaldo) Date: Wed, 9 Nov 2011 12:57:39 -0600 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <1320846789.21919.361.camel@hurina> References: <201111091029.37963.thomas@koch.ro> <1320846789.21919.361.camel@hurina> Message-ID: <629183B4-622D-4058-B706-D5FFF91FC366@cpanel.net> On Nov 9, 2011, at 7:53 AM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 10:29 +0100, Thomas Koch wrote: > >> Somehow related: Since the free software world[2] slowly converges towards GIT >> as the "one and only" VCS, have you thought about a switch? It's much more >> likely that somebody checks out your code, looks into it and provides patches >> if the VCS is already installed. > > I'm not as much against git anymore as I was when I switched to hg, but > I don't see much benefits in switching to git either. I highly doubt I'd > get even a single patch more if I used git instead of hg. > Hg / Git. Either seems fine to me. My big concern would be that the source history is in multiple accessible places around the world. This is the biggest limitation of SVN to my mind. Could I suggest a google code clone as an additional VCS backup location? From adrian.minta at gmail.com Wed Nov 9 21:01:43 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 21:01:43 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320864362.21919.380.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> <1320864362.21919.380.camel@hurina> Message-ID: <4EBACE17.4070508@gmail.com> On 11/09/11 20:46, Timo Sirainen wrote: > On Wed, 2011-11-09 at 20:44 +0200, Adrian Minta wrote: >>> Is the quota in the database 1096 or>2GB? i.e. is the problem with >>> reading it, or updating it? In general the quota code uses 64bit >>> integers everywhere so this shouldn't be happening. >>> >>> >> In the database quota field is 2048000000. > In the dict quota value? That looks more like the quota limit, not the > current quota usage. In the mailbox table. The 2000000 value commes from: user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' This value is easy to change by modifying the above mysql query, but the 1096 value I don't know how to change. Maybe to "instruct" dovecot to work internal with Kilobytes ? For instance by altering the query like this: user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', quota, 'K') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' ... I get ... * QUOTA "user" (STORAGE 1096 2048000000) -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From mrvjtod at gmail.com Wed Nov 9 21:04:12 2011 From: mrvjtod at gmail.com (Chris Young) Date: Wed, 9 Nov 2011 14:04:12 -0500 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: References: Message-ID: I am familiar with this article http://wiki.dovecot.org/LDA#multipleuids I want to disable setuid-root, but I don't know the default group and permissions. sudo chgrp root /usr/libexec/dovecot/dovecot-lda sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda but my delivery is still failing Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) Thoughts? Thanks Chris From e-frog at gmx.de Wed Nov 9 21:08:48 2011 From: e-frog at gmx.de (e-frog) Date: Wed, 09 Nov 2011 20:08:48 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <4EB6AFE3.3020001@gmx.de> References: <4EB6AFE3.3020001@gmx.de> Message-ID: <4EBACFC0.6060703@gmx.de> On 06.11.2011 17:03, wrote e-frog: > 1 lsub "" "virtual/*" > * LSUB () "/" "virtual" > * LSUB () "/" "virtual/Lists" > * LSUB () "/" "virtual/unread" > 1 OK Lsub completed. Just tested this again with 2.1beta1 with the same result. From apm at one.com Wed Nov 9 21:25:57 2011 From: apm at one.com (Peter Mogensen) Date: Wed, 09 Nov 2011 20:25:57 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EBAD3C5.7030006@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: >> Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted >> transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: >> indexid changed 1320419300 -> 1320419441 (sync_offset=0) > > Session A had TestMails open and created with index file whose ID was > 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 > 17:08:20 EET 2011). > > Session B came and recreated the index files 141 seconds later with ID > 1320419441. Either it didn't see A's original index files for some > reason or it simply decided to recreate them for some reason. Either way > this shouldn't have happened. Turns out this is expected to confuse Session A. The client in question sometimes start the session (B) with this command sequence: DELETE folder CREATE folder APPEND... Any Session A having opened "folder" of course would be surprised that there's a new index file (makes we wish for an IMAP5 where folderID!=displayname) This can be reproduced by hand speaking IMAP with two telnets. Only question left is, why does Dovecot end the log sequence by saying: "Disconnected: IMAP session state is inconsistent, please relogin." ... when it is capable of detecting this and returning "BYE folder deleted under us" and logging the same. /Peter From rpalmarin at yahoo.com Wed Nov 9 18:57:44 2011 From: rpalmarin at yahoo.com (rpalmarin) Date: Wed, 9 Nov 2011 16:57:44 +0000 (UTC) Subject: [Dovecot] LDAP expired password References: <116877.53915.qm@web29503.mail.ird.yahoo.com> <57o6m4h3miv8@mids.svenhartge.de> <4D95A998.9080701@noa.gr> <97o72ha3miv8@mids.svenhartge.de> Message-ID: Sven Hartge svenhartge.de> writes: > > Nikolaos Milas noa.gr> wrote: > > On 1/4/2011 11:09 ??, Sven Hartge wrote: > > >> Have a look at the ppolicy slapd.overlay. This will solve your > >> problem. hello all Sorry for the delay in the response I checked the ppolicy overlay but without success. This overlay does not have a single "password expired" attribute to put in the user_filter. currently my dovecot-ldap.conf contain: user_filter (&(|( = uid =% u) (employeeNumber =% u)) (objectClass = inetOrgPerson) (accountStatus = active)) > At my university we introduced our own attribute gifb-status which > contains a "1" if an account is valid, a "0" if it is not (and several > others for different purposes) and our ldap-filters all contain > something like "(&(ou=foobar)(gifb-status=1))". is possible that the only way to do this is to manage a new attribute? how can understand all the people that have configured the mail client to authenticate with imap-dovecot that their passoword has expired? thanks / roby From user+dovecot at localhost.localdomain.org Wed Nov 9 23:41:45 2011 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Wed, 09 Nov 2011 22:41:45 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <4EBAF399.7090706@localhost.localdomain.org> On 11/09/2011 10:29 AM Thomas Koch wrote: > Hi, > > somebody from the kolab groupware project recently explained me that the Bus- > Factor[1] of around 1 would be one of their primary reasons not to use Dovecot > and stick with Cyrus. And what is their primary reason for the lame excuse? Too many cooks spoil the broth. just my 2? Regards, Pascal -- The trapper recommends today: fabaceae.1131322 at localdomain.org From pch0317 at gmail.com Thu Nov 10 00:33:59 2011 From: pch0317 at gmail.com (pch0317) Date: Wed, 09 Nov 2011 22:33:59 +0000 Subject: [Dovecot] Can't delete sub-folder Message-ID: <4EBAFFD7.8020201@gmail.com> Hi list, I have problem with deleting sub-folder. I use thunderbird 8.0 as a client and dovecot 2.0.9 with imap protocol as a server. I use dovecot with tb-extra-mailbox-sep parameter. I use mbox mailboxes and thunderbird with *not checked* option 'server supports folders that contain sub-folders and messages' so when I create sub-folder I can choose between 'folders only' and 'messages only'. When I try to delete 'parent' folder which contain 'messages only' folder, error message appear: [ALREADYEXISTS] Mailbox has children, delete them first. Ok, so I delete 'message only' folder first and try to delete parent folder, but the same error appear: [ALREADYEXISTS] Mailbox has children, delete them first. When I list mailboxes from console I see inside 'parent' folder ".imap" folder. When I delete .imap folder from console and then delete parent folder from thunderbird, parent folder is deleted correctly. Which application create this ".imap" hidden folder and how solve this problem. Thanks From stan at hardwarefreak.com Thu Nov 10 02:42:33 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 18:42:33 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBAA767.6060609@lazurit.com> References: <4EBAA767.6060609@lazurit.com> Message-ID: <4EBB1DF9.4040208@hardwarefreak.com> On 11/9/2011 10:16 AM, Alexander Chekalin wrote: > Thanks, Robert, > > will take a look at. > > What I'm afraid for is how database storage should be planned (storage, > CPU, RAM, scaling when will be over-filled). When dealing with files > (I'm using maildir) Bingo. ^^^ Maildir is very likely a hug factor in your current slow search time. With a maildir search, every mail file must be opened and searched. How many total mail files are opened for each of your searches? Thousands? Tens of thousands? Maildir causes a massive disk IO bottleneck when searching so many files. Run iostat the next time you do one of these searches, and look at the %iowait value. It will likely be very high. If it is, this confirms maildir is a big part of the problem. mbox, and mdbox, would be many many times faster than maildir WRT searching as the total number of files is lower by orders of magnitude. Switching from maildir to mbox/mdbox shifts the workload burden from the disk subsystem to the processor/memory. And I'm sure as with everyone else on the planet today, you have massive spare CPU cycles, but extremely limited spindle throughput. And as Timo suggested, using one of the indexing search plugins would be much faster yet, as long as you keep the indexes updated. -- Stan From stan at hardwarefreak.com Thu Nov 10 02:50:07 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 18:50:07 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <1320856804.21919.372.camel@hurina> References: <4EBAA767.6060609@lazurit.com> <1320856804.21919.372.camel@hurina> Message-ID: <4EBB1FBF.9070401@hardwarefreak.com> On 11/9/2011 10:40 AM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 19:16 +0300, Alexander Chekalin wrote: >> Will also try to use full text search, >> but afraid of index size (and I need no search on body, just on headers). > > It wouldn't be difficult to patch Dovecot to skip indexing message > bodies. Of course then you'd need to remember to keep applying the patch > when updating. Also keep in mind that, in general, many/most message headers today are often as large, or larger than, the actual message body, especially for list mail. Just take a look at messages from this for evidence. Thus, I'd think that going out of your way to avoid indexing message bodies wouldn't be worth the effort/headaches involved. -- Stan From scott_the_musician at yahoo.com.au Thu Nov 10 06:09:02 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Wed, 9 Nov 2011 20:09:02 -0800 (PST) Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Message-ID: <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> ----- Forwarded Message ----- From: Scott Lewis To: "dovecot at dovecot.org" Sent: Thursday, 3 November 2011 4:31 PM Subject: patching dovecot for sieve/managesieve support, centos 5.6? Hi all, I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. Here's what happens: [root at mail ~]# whereis dovecot dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot ... checking whether to build static libraries... yes dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH to give path to compiled Dovecot sources or to a directory with the installed dovecot-config file. configure: error: dovecot-config not found -- I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: Avelsieve plugin details: backend = ManageSieve ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? thanks in advance! From achekalin at lazurit.com Thu Nov 10 06:37:05 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Thu, 10 Nov 2011 07:37:05 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB1DF9.4040208@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: Oh, that's the point to consider. But I must confess I'm in love with Maildir for maybe 10 years for that simple fact I can do anything with each and every single message even on disk (=much faster than via IMAP). If I would deal with mbox directly I'd need to parse huge files, brrrr. Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? 10.11.2011, ? 3:42, Stan Hoeppner ???????(?): > On 11/9/2011 10:16 AM, Alexander Chekalin wrote: >> Thanks, Robert, >> >> will take a look at. >> >> What I'm afraid for is how database storage should be planned (storage, >> CPU, RAM, scaling when will be over-filled). When dealing with files >> (I'm using maildir) > > Bingo. ^^^ > > Maildir is very likely a hug factor in your current slow search time. > With a maildir search, every mail file must be opened and searched. How > many total mail files are opened for each of your searches? Thousands? > Tens of thousands? Maildir causes a massive disk IO bottleneck when > searching so many files. Run iostat the next time you do one of these > searches, and look at the %iowait value. It will likely be very high. > If it is, this confirms maildir is a big part of the problem. > > mbox, and mdbox, would be many many times faster than maildir WRT > searching as the total number of files is lower by orders of magnitude. > Switching from maildir to mbox/mdbox shifts the workload burden from > the disk subsystem to the processor/memory. And I'm sure as with > everyone else on the planet today, you have massive spare CPU cycles, > but extremely limited spindle throughput. > > And as Timo suggested, using one of the indexing search plugins would be > much faster yet, as long as you keep the indexes updated. > > -- > Stan From achekalin at lazurit.com Thu Nov 10 07:35:11 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Thu, 10 Nov 2011 08:35:11 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB1DF9.4040208@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EBB628F.8060705@lazurit.com> Hello, Stan, in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway). You see, what I need is not only find message from|to someone on specified date, I also sometime need to restore that message back to user's original box. As far our mailserver and backup-mailserver are different machines, it is a bit tricky to copy messages between it fast enough. Say, if I need to find and restore all mails from user at domain.com within 2009 year, and search yields in some 1000's of messages, then use IMAP to copy it over to another server takes some time - and if you consider both search time and restore/copy time the whole process may take "ages". With maildir I can rsync/scp needed files to another host and that's fast way - that's why I stick with maildir. FTS in my case can help (I can search for user at domain.com, for example), but it also return messages that contains such a string in message body (and that takes index space, too), so I'll need to filter it later, but surely it'll be faster than checking every message in the archive. Yours, Alexander > Maildir is very likely a hug factor in your current slow search time. > With a maildir search, every mail file must be opened and searched. How > many total mail files are opened for each of your searches? Thousands? > Tens of thousands? Maildir causes a massive disk IO bottleneck when > searching so many files. Run iostat the next time you do one of these > searches, and look at the %iowait value. It will likely be very high. > If it is, this confirms maildir is a big part of the problem. > > mbox, and mdbox, would be many many times faster than maildir WRT > searching as the total number of files is lower by orders of magnitude. > Switching from maildir to mbox/mdbox shifts the workload burden from > the disk subsystem to the processor/memory. And I'm sure as with > everyone else on the planet today, you have massive spare CPU cycles, > but extremely limited spindle throughput. > > And as Timo suggested, using one of the indexing search plugins would be > much faster yet, as long as you keep the indexes updated. > -- ? ?????????, ????????? ??????? ??????? ??????????? +7 909 799 2549 achekalin at lazurit.com From research at cruzio.com Thu Nov 10 10:30:31 2011 From: research at cruzio.com (Mark Hanford) Date: Thu, 10 Nov 2011 00:30:31 -0800 Subject: [Dovecot] indexes permissions problem Message-ID: <4EBB8BA7.9070603@cruzio.com> Hey all, I did a search but didn't find the answer to my problem, so here goes. I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box with user home directories NFS mounted), my index files are setup to be in /u/indexes/ On the Dovecot 1.1.1 installation, the perms on the indexes directory is 777 with root:mail ownership. The same thing on the Dovecot 2 / Centos server results in a 'permission denied' error when Dovecot tries to create files. So, I guess my main question is, what perms and ownership should /u/indexes be set to? I've tried several different things before this cry for help... Thanks. Mark From ms at mike2k.de Thu Nov 10 11:19:34 2011 From: ms at mike2k.de (Michael Stilkerich) Date: Thu, 10 Nov 2011 10:19:34 +0100 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320442758.21919.222.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> <1320442758.21919.222.camel@hurina> Message-ID: Hi, On Nov 4, 2011, at 10:39 PM, Timo Sirainen wrote: >> Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking >> failed for file /home/dovecot/isa/dovecot.index.log: No such file or >> directory >> Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with >> file /home/dovecot/isa/dovecot.index.log: No such file or directory > > These simply shouldn't happen. I'd say it's a kernel bug. You're running > a default Ubuntu kernel? I wonder if other Ubuntu users have this > problem. It may be an apparmor issue. I noticed plenty of apparmor log entries on these accesses, though apparmor should only log but not disallow them. I have unloaded the dovecot apparmor profiles and not seen any of these errors since then. I got a new issue, however: I migrated from Maildir to mdbox. Since then, my shared mailboxes don't fully work anymore. I have given another user full rights to a shared mailbox (getacl returns akxeilprwtscd for that folder/user). The user sees the mailbox an can perform some operations including reading and deleting messages on it. If she tries to insert a new message, however, it fails and the error log shows: dovecot: imap(isa): Error: fcntl(write-lock) locking failed for file /home/dovecot/michael/storage/dovecot.map.index.log: Bad file descriptor dovecot: imap(isa): Error: mail_index_wait_lock_fd() failed with file /home/dovecot/michael/storage/dovecot.map.index.log: Bad file descriptor All my mail locations are owned by the respective system user and the mail group, and writeable by both. In particular, I checked that both the storage directory as well as the dovecot.map.index.log are writeable by the mail group. The users are not regular members of the mail group, but my dovecot config contains mail_access_groups = mail Any idea how to resolve this issue? -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4813 bytes Desc: not available URL: From alexdosh at gmail.com Thu Nov 10 11:27:36 2011 From: alexdosh at gmail.com (Alexander KIper) Date: Thu, 10 Nov 2011 13:27:36 +0400 Subject: [Dovecot] Exim thru through Dovecot deliver to spec IMAP-folder Message-ID: Hello All! How can I "post" some mails from Exim trnasport through Dovecot deliver to IMAP MailBox in specific folder (for example: "Junk")? -- With best regards, Alexander KIper! From usenet at schani.com Thu Nov 10 11:40:56 2011 From: usenet at schani.com (wwl) Date: Thu, 10 Nov 2011 10:40:56 +0100 Subject: [Dovecot] Limit the number of pop/imap requests Message-ID: Hello Group, is it possible to Limit the number of pop/imap requests to Dovecot? I read the Wiki and found max_mail_processes = 512 is this the way to limit this ? dovecot --version 1.2.15 Thanx for help Christian From david.ocana at bsc.es Thu Nov 10 13:14:50 2011 From: david.ocana at bsc.es (David Ocana) Date: Thu, 10 Nov 2011 12:14:50 +0100 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <1320861378.21919.376.camel@hurina> References: <4EBA4DD4.4080505@bsc.es> <1320861378.21919.376.camel@hurina> Message-ID: <4EBBB22A.90501@bsc.es> On 09/11/11 18:56, Timo Sirainen wrote: > On Wed, 2011-11-09 at 10:54 +0100, David Ocana wrote: > >> I've been trying to set up dovecot 2.0.15, everything seems to work >> pretty well except for the quota feature. I would like to set a quota >> limit only for the Inbox folder. I configured two namespaces, >> according to some posts from Timo Sirainen >> >> namespace { >> separator = / >> prefix = INBOX/ >> location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n >> inbox = yes >> hidden = yes >> } >> >> plugin { >> quota = dirsize:User quota > > quota = dirsize:User quota:ns=INBOX/ Actually I forgot to mention that I also tried that, but I got the following error: Error: Initialization failed: Failed to initialize quota: Quota root User quota: Unknown parameter for backend dirsize: ns=INBOX/ That's why I was trying to change quota settings by using the quota_rule directive. > > This limits the quota only to mailboxes in INBOX/ namespace. > >> # I've tried with: >> quota_rule = INBOX:storage=819200K >> quota_rule = INBOX/*:storage=819200K >> quota_rule = INBOX/Inbox:storage=819200K > > Quota rules don't work in this way. There are no per-mailbox quotas > really, at least in the way you're thinking about. > I see, I guess they're per-namespace quotas, right? I got the wrong idea after watching the following, which was exactly what I wanted to do :p quota_rule = : May be that, using dirsize backend lets you no other option than calculating quota for the whole user's mailbox? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4439 bytes Desc: S/MIME Cryptographic Signature URL: From stan at hardwarefreak.com Thu Nov 10 14:46:05 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 06:46:05 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EBBC78D.2060809@hardwarefreak.com> On 11/9/2011 10:37 PM, Alexander Chekalin wrote: > Oh, that's the point to consider. > > But I must confess I'm in love with Maildir for maybe 10 years This love affair may be coming to and end. >...for that simple fact I can do anything with each and every single message even on disk (=much faster than via IMAP). If I would deal with mbox directly I'd need to parse huge files, brrrr. Mbox is an excellent mailbox format for archived mail *because of* the fact that searching it is very fast and the disk subsystem overhead is low. For example, on my decade+ old 550MHz x86 SOHO server with only 384MB RAM and a single 7.2k SATA disk, after dropping caches, we'll search my debian-users mbox archive (my largest) for total message count by searching a known header of every message: -rw------- 1 stan stan 133M Nov 10 06:03 1-Debian-Users ~/mail$ time grep -c Content-Length 1-Debian-Users 22817 real 0m1.731s user 0m0.328s sys 0m0.852s Now let's search for posts from me (after dropping caches again): ~/mail$ time grep -c "From: Stan Hoeppner" 1-Debian-Users 536 real 0m1.657s user 0m0.216s sys 0m0.896s Nested greps will obviously take longer, as will those using perl expressions, but this gives some indication of the kind of speed we're talking about: less than seconds to search 22,000+ messages for a specific single header. So that's ~20 seconds for an mbox containing 220K+ messages, again on 10+ year old hardware. > Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? You should probably take a look at Enkive. I'm not sure what mail storage format it uses, and I've not used it personally, so I can't vouch for its speed, but it's pretty complete feature-wise. Take the test drive--nice search interface. http://www.enkive.org/ -- Stan From stan at hardwarefreak.com Thu Nov 10 15:17:42 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 07:17:42 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB628F.8060705@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EBB628F.8060705@lazurit.com> Message-ID: <4EBBCEF6.5040502@hardwarefreak.com> On 11/9/2011 11:35 PM, Alexander Chekalin wrote: > Hello, Stan, > > in fact the only thing I miss even with my current scheme is permanent > ID assigned to the message so I can easily find it despite the IMAP > mailbox it is now (so if someone moved the message from one > mailbox/folder to another, the ID allows to retrieve it fast anyway). > > You see, what I need is not only find message from|to someone on > specified date, I also sometime need to restore that message back to > user's original box. As far our mailserver and backup-mailserver are > different machines, it is a bit tricky to copy messages between it fast > enough. Say, if I need to find and restore all mails from > user at domain.com within 2009 year, and search yields in some 1000's of > messages, then use IMAP to copy it over to another server takes some > time - and if you consider both search time and restore/copy time the > whole process may take "ages". Apparently I didn't fully understand all of your requirements. Moving the archived mail to mbox/mdbox and/or getting a good indexing search engine installed will cut the search time down tremendously. Whether that would make up for the time consumed with an IMAP copy of many emails I don't know. If your servers aren't old and slow, and are not already overloaded, I would think the IMAP message copying over GbE would be pretty quick, even for the 1000 messages scenario. There may be some Dovecot tweaks that might make this copy process faster. Timo would need to chime in on that. Do you perform the IMAP transfers with a GUI IMAP client on your management PC? Or are you using imapsync or some other util directly on the servers? If the former you may be able to tweak your IMAP client to speed up the transfers as well. Try using IMAP and not IMAPS for the transfers. What is the network infrastructure between the servers and your management workstation? Is it all GbE with jumbo frames enabled? > With maildir I can rsync/scp needed files to another host and that's > fast way - that's why I stick with maildir. There is definitely some flexibility here. > FTS in my case can help (I can search for user at domain.com, for example), > but it also return messages that contains such a string in message body > (and that takes index space, too), so I'll need to filter it later, but > surely it'll be faster than checking every message in the archive. Sure. So you're concerned with your poor performance, but also with disk space. Unfortunately there's no free lunch to be had. You'll have to make sacrifices somewhere. You could go with mdbox and use compression, trading that saved space for search index files space. -- Stan From stan at hardwarefreak.com Thu Nov 10 15:19:57 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 07:19:57 -0600 Subject: [Dovecot] Exim thru through Dovecot deliver to spec IMAP-folder In-Reply-To: References: Message-ID: <4EBBCF7D.1080904@hardwarefreak.com> On 11/10/2011 3:27 AM, Alexander KIper wrote: > Hello All! > > How can I "post" some mails from Exim trnasport through Dovecot deliver to > IMAP MailBox in specific folder (for example: "Junk")? Dovecot 1.x: http://wiki.dovecot.org/LDA/Sieve Dovecot 2.x: http://wiki2.dovecot.org/Pigeonhole/Sieve -- Stan From tss at iki.fi Thu Nov 10 17:29:35 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 10 Nov 2011 17:29:35 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: On 10.11.2011, at 6.37, Alexander Chekalin wrote: > Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? See doveadm fetch / doveadm search. > in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway). Dovecot has message GUIDs (with maildir it's filename), but there's no quick lookup for them, even though doveadm can fetch them easily: doveadm fetch text guid 12312312 From adrian.minta at gmail.com Thu Nov 10 18:03:14 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Thu, 10 Nov 2011 18:03:14 +0200 Subject: [Dovecot] Quota BUG - fixed In-Reply-To: <1320864362.21919.380.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> <1320864362.21919.380.camel@hurina> Message-ID: <4EBBF5C2.7020403@gmail.com> After some deep investigations I manage to solve the problem. I was only reading quota in user_querry. Now I read it in user_querry and in password_query and all seems fine: --dovecot-sql.conf--- user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', quota) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' password_query = SELECT username as user, password, '/home/%d/%n' as userdb_home, 'maildir:/home/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid, CONCAT('*:bytes=', quota) as userdb_quota_rule FROM mailbox WHERE username = '%u' AND active = '1' --dovecot.conf--- plugin { quota = dict:user::proxy::quotadict quota_rule2 = Trash:storage=10%% quota_rule3 = SPAM:storage=10%% } the result is fine now: 2 getquotaroot inbox * QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1997999 2000000) 2 OK Getquotaroot completed. Only one "cosmetic" bug remains when an empty mailbox appear as a small negative number in quota2 table, but this is fixable in postfixadmin. -- Best regards, Adrian Minta MA3173-RIPE,www.minta.ro From micah at riseup.net Thu Nov 10 20:59:32 2011 From: micah at riseup.net (Micah Anderson) Date: Thu, 10 Nov 2011 13:59:32 -0500 Subject: [Dovecot] dovecot-lda quota rule Message-ID: <874nybztuz.fsf@algae.riseup.net> I really like the feature where you can define quota rules with percents which trigger off of the default values[0] (so you can set the Trash to allow for 10% more of the user's quota for example). What I would really love in dovecot would be for the ability to configure a quota rule for dovecot-lda. I would like to configure things so we don't bounce emails for users until they are well over quota, the IMAP quota plugin is a really great way to notify people that they are over quota because it fails to write to other folders.... that should be enough to get people's attention that they need to deal with things, but bouncing is harsh. Is there a way to do this now that I haven't seen? thanks! micah 0. http://wiki2.dovecot.org/Quota/Configuration -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From carloswill at gmail.com Thu Nov 10 21:17:15 2011 From: carloswill at gmail.com (Carlos Mennens) Date: Thu, 10 Nov 2011 14:17:15 -0500 Subject: [Dovecot] TLS Authentication Confusion Message-ID: I asked a user today to make sure his incoming and outgoing email was using TLS. He told me it wasn't possible because my Dovecot / Postfix daemons were only listening on TCP 25 & 143 according to a port scan he did. He told me the only way I could enable encrypted secure sessions between the client & server is to enable port 993 (IMAPs). I told him that TLS is supported on my mail server over the default ports TCP 25 / 143 and that many consider IMAPs to be legacy. I sent him a telnet session of my PC communicating with my server & it shows TLS is available. I just wanted to be sure I was correct with the information above or am I completely wrong and I do indeed need TCP port 993? I know this is the Dovecot mailing list but since Dovecot and Postfix both use and support TLS in their configuration files, I figured I would ask here for your help! carloss at pc1:~$ telnet mail.holyghost.org 25 Trying 192.168.4.100... Connected to mail.holyghost.org. Escape character is '^]'. 220 mail.holyghost.org ESMTP Postfix EHLO pc1.holyghost.org 250-mail.holyghost.org 250-PIPELINING 250-SIZE 20480000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN Below is a snip from my mail logs showing TLS: Nov 9 10:26:39 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=::ffff:192.168.4.100, lip=::ffff:192.168.4.100, TLS The above snip from my log means that I'm connecting to Dovecot via TLS, correct? From dick at fouter.net Thu Nov 10 21:28:55 2011 From: dick at fouter.net (Dick Middleton) Date: Thu, 10 Nov 2011 19:28:55 +0000 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: References: Message-ID: <4EBC25F7.1040700@fouter.net> On 11/10/11 19:17, Carlos Mennens wrote: > I asked a user today to make sure his incoming and outgoing email was > using TLS. He told me it wasn't possible because my Dovecot / Postfix > daemons were only listening on TCP 25 & 143 according to a port scan > he did. He told me the only way I could enable encrypted secure > sessions between the client & server is to enable port 993 (IMAPs). Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same port as plain. The difference between SSH and TLS is that with SSH the encryption is set up before any application communication takes place. i.e all application packets are contained in the encrypted payload. With TLS the application starts communication and then the application sets up encryption of its payload. Dick From stephan at rename-it.nl Thu Nov 10 21:37:56 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 10 Nov 2011 20:37:56 +0100 Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> Message-ID: <4EBC2814.2090200@rename-it.nl> This mail was answered before. Don't repost your question unless you have acted on the information provided, got new information or have additional questions. Re-posting the exact same message makes no sense. Regards, Stephan. On 11/10/2011 5:09 AM, Scott Lewis wrote: > > > ----- Forwarded Message ----- > From: Scott Lewis > To: "dovecot at dovecot.org" > Sent: Thursday, 3 November 2011 4:31 PM > Subject: patching dovecot for sieve/managesieve support, centos 5.6? > > > Hi all, > > I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. > > Here's what happens: > > [root at mail ~]# whereis dovecot > dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz > > [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot > > ... > > checking whether to build static libraries... yes > dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH > to give path to compiled Dovecot sources or to a directory with the > installed dovecot-config file. configure: error: > dovecot-config not found > > -- > > I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. > > I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: > > Avelsieve plugin details: backend = ManageSieve > ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? > > thanks in advance! From frank at moltke28.B.Shuttle.DE Thu Nov 10 21:56:50 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Thu, 10 Nov 2011 20:56:50 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC25F7.1040700@fouter.net> References: <4EBC25F7.1040700@fouter.net> Message-ID: (auto-added) On Thu, 10 Nov 2011 19:28:55 +0000 Dick Middleton wrote: > On 11/10/11 19:17, Carlos Mennens wrote: > > I asked a user today to make sure his incoming and outgoing email was > > using TLS. He told me it wasn't possible because my Dovecot / Postfix > > daemons were only listening on TCP 25 & 143 according to a port scan > > he did. He told me the only way I could enable encrypted secure > > sessions between the client & server is to enable port 993 (IMAPs). > > Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same > port as plain. > > The difference between SSH and TLS is that with SSH the encryption is set up > before any application communication takes place. i.e all application packets > are contained in the encrypted payload. With TLS the application starts > communication and then the application sets up encryption of its payload. :%s/SSH/SSL/g --Frank From tom at whyscream.net Thu Nov 10 22:11:01 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 10 Nov 2011 21:11:01 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC25F7.1040700@fouter.net> References: <4EBC25F7.1040700@fouter.net> Message-ID: <4EBC2FD5.2060602@whyscream.net> On 10-11-11 20:28, Dick Middleton wrote: > On 11/10/11 19:17, Carlos Mennens wrote: >> I asked a user today to make sure his incoming and outgoing email was >> using TLS. He told me it wasn't possible because my Dovecot / Postfix >> daemons were only listening on TCP 25 & 143 according to a port scan >> he did. He told me the only way I could enable encrypted secure >> sessions between the client & server is to enable port 993 (IMAPs). > > Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same > port as plain. > > The difference between SSH and TLS is that with SSH the encryption is set up > before any application communication takes place. i.e all application packets > are contained in the encrypted payload. With TLS the application starts > communication and then the application sets up encryption of its payload. > You're contributing to the confusion. SSL and TLS are practically the same, just another name for the same beast. The only difference is that SSL is the old name, and newer versions of the standard are labeled TLS. The term SSH is not in the scope of this question. There are 2 ways of using SSL/TLS to encrypt sessions: 1) Setup a dedicated port where a SSL/TLS session can be setup before the actual data is transferred. This is what happens for IMAPS/993 and SMTPS/465. 2) Extend an existing protocol to enable SSL/TLS during an open session. This is called STARTTLS in several protocols, SMTP and IMAP being among them. And this is what happens on SMTP/25, Submission/587 and IMAP/143. Note that although the second option is *named* STARTTLS, you probably could implement any server to *use* SSL 1.0 for the actual encryption (not recommended though). The OP is offering STARTTLS for both services, which is good. -- Regards, Tom From noeldude at gmail.com Thu Nov 10 22:21:49 2011 From: noeldude at gmail.com (Noel) Date: Thu, 10 Nov 2011 14:21:49 -0600 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC2FD5.2060602@whyscream.net> References: <4EBC25F7.1040700@fouter.net> <4EBC2FD5.2060602@whyscream.net> Message-ID: <4EBC325D.60404@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11/10/2011 2:11 PM, Tom Hendrikx wrote: > On 10-11-11 20:28, Dick Middleton wrote: >> On 11/10/11 19:17, Carlos Mennens wrote: >>> I asked a user today to make sure his incoming and outgoing email was >>> using TLS. He told me it wasn't possible because my Dovecot / Postfix >>> daemons were only listening on TCP 25 & 143 according to a port scan >>> he did. He told me the only way I could enable encrypted secure >>> sessions between the client & server is to enable port 993 (IMAPs). >> >> Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same >> port as plain. >> >> The difference between SSH and TLS is that with SSH the encryption is set up >> before any application communication takes place. i.e all application packets >> are contained in the encrypted payload. With TLS the application starts >> communication and then the application sets up encryption of its payload. >> > > You're contributing to the confusion. > > SSL and TLS are practically the same, just another name for the same > beast. The only difference is that SSL is the old name, and newer > versions of the standard are labeled TLS. The term SSH is not in the > scope of this question. > > There are 2 ways of using SSL/TLS to encrypt sessions: > > 1) Setup a dedicated port where a SSL/TLS session can be setup before > the actual data is transferred. This is what happens for IMAPS/993 and > SMTPS/465. > > 2) Extend an existing protocol to enable SSL/TLS during an open session. > This is called STARTTLS in several protocols, SMTP and IMAP being among > them. And this is what happens on SMTP/25, Submission/587 and IMAP/143. > > Note that although the second option is *named* STARTTLS, you probably > could implement any server to *use* SSL 1.0 for the actual encryption > (not recommended though). > > The OP is offering STARTTLS for both services, which is good. > > -- > Regards, > Tom The confusion is caused by the way some client software differentiate these services in their configuration, often referring to wrappermode smtps/imaps as "SSL", and STARTTLS as "TLS". -- Noel Jones -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOvDJcAAoJEHIluGOd3V4F6foH/16+xq91/j4hgXufdnAsxwW1 N2ZXf1fby7TjR4BpaYNdH6PsN5/UqFSZItVYkeDXWgGG/wYCTRC+LHdks/EeQKgR 1ondUL2iorQ7bGy25m3526DGShFmcEh7P+Z6WWwdFeOTLBS57LIgwvFHBg4niYHq 3ZbPOjzI+d7kbz8tT8ATb+Ju+uJlV2rpbZKHQ90qlOR9tRl6bUOEeW32yPf5hjpI gs89o66Ud+mb9kkH9vgrhnutxsWjVxWNWM1ba43S1bh4Jg9YneIdsHdQVQSPrFUz EPy5Tgz3b+LZC6lwe6czFrhYgv/GUiJutS34qRHLSMAQGY+fgOcZBSZQHKP7NC4= =TdNE -----END PGP SIGNATURE----- From sven at svenhartge.de Thu Nov 10 23:15:05 2011 From: sven at svenhartge.de (Sven Hartge) Date: Thu, 10 Nov 2011 22:15:05 +0100 Subject: [Dovecot] LDAP expired password References: <116877.53915.qm@web29503.mail.ird.yahoo.com> <57o6m4h3miv8@mids.svenhartge.de> <4D95A998.9080701@noa.gr> <97o72ha3miv8@mids.svenhartge.de> Message-ID: rpalmarin wrote: > Sven Hartge svenhartge.de> writes: >> Nikolaos Milas noa.gr> wrote: >>> On 1/4/2011 11:09 ??, Sven Hartge wrote: >>>> Have a look at the ppolicy slapd.overlay. This will solve your >>>> problem. > Sorry for the delay in the response I checked the ppolicy overlay but > without success. This overlay does not have a single "password > expired" attribute to put in the user_filter. I think you misunderstood the usage of the overlay. There is _no_ additional attribute to check. With ppolicy any authentication will fail if some previously defined conditions are met (or no longer met) like the max age of a password. Documentation is contained in "man slapo-ppolicy", which as bit hard to understand, I must admit. Also look at http://www.openldap.org/doc/admin24/overlays.html "12.10 Password Policies" has a nice example. With this overlay you don't need any additional attributes and no maintenance or houskeeping script to invalidate expired passwords. >> At my university we introduced our own attribute gifb-status which >> contains a "1" if an account is valid, a "0" if it is not (and >> several others for different purposes) and our ldap-filters all >> contain something like "(&(ou=foobar)(gifb-status=1))". > is possible that the only way to do this is to manage a new attribute? > how can understand all the people that have configured the mail > client to authenticate with imap-dovecot that their passoword has > expired? Well, either way (using ppolicy or an additional attribute): they will call the support desk, if they are unable to understand the message from their mail client. No way to fix _this_ problem, I am afraid ;) S? -- Sigmentation fault. Core dumped. From frank at moltke28.B.Shuttle.DE Fri Nov 11 13:01:05 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 12:01:05 +0100 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop Message-ID: (auto-added) Hi, running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA I've found a strange error in log which I've nerver observed before when startung dovecot: auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop Dovecot seems to work normal despite the error message. Any enlightment welcome. --Frank Elsner From tss at iki.fi Fri Nov 11 13:10:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 11 Nov 2011 13:10:57 +0200 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop In-Reply-To: (auto-added) References: (auto-added) Message-ID: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> On 11.11.2011, at 13.01, Frank Elsner wrote: > running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA .. > auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop authdb_imap is from Dovecot v2.1. It doesn't exist in v2.0.14.. From peter at newton.cx Fri Nov 11 09:41:21 2011 From: peter at newton.cx (Peter Williams) Date: Thu, 10 Nov 2011 23:41:21 -0800 Subject: [Dovecot] Invalid "doveadm search" queries not rejected Message-ID: <1320997281.2261.116.camel@iota.localdomain> Hello, I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS 6). According to the doveadm-search-query(7) manual page, a "mailbox" specification can only come at the beginning of a search query. However, if I put a mailbox specification in the middle of a query, not only do I not get an error message, I get surprising results: $ doveadm search mailbox A |wc -l 6979 $ doveadm search mailbox B |wc -l 206 $ doveadm search mailbox A OR mailbox B |wc -l 42492 If it became possible to do boolean logic on mailbox queries, that'd be cool. But barring that, these invalid queries should be rejected. Relatedly, if I can only do simple mailbox filtering, does anyone know of a clever way to efficiently emulate more complex boolean logic? Please CC me on any replies as I'm not subscribed to the list. Thanks, Peter -- Peter Williams / peter at newton.cx From frank at moltke28.B.Shuttle.DE Fri Nov 11 16:28:44 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 15:28:44 +0100 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop In-Reply-To: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> References: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> Message-ID: (auto-added) On Fri, 11 Nov 2011 13:10:57 +0200 Timo Sirainen wrote: > On 11.11.2011, at 13.01, Frank Elsner wrote: > > > running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA > .. > > auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop > > authdb_imap is from Dovecot v2.1. It doesn't exist in v2.0.14.. Nov 11 15:22:13 seymour dovecot: master: Dovecot v2.0.15 starting up (core dumps disabled) Nov 11 15:22:13 seymour dovecot: auth: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop This is definitly version 2.0.15 starting. But ... /usr/local/dovecot/lib/dovecot/auth has date Nov 9 10:17 usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so has date Nov 9 10:17 This was the time I've tested dovecot-2.1. --Frank Elsner From frank at moltke28.B.Shuttle.DE Fri Nov 11 16:37:17 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 15:37:17 +0100 Subject: [Dovecot] Re(2): undefined symbol: o_stream_switch_ioloop In-Reply-To: References: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> Message-ID: (auto-added) On Fri, 11 Nov 2011 15:28:44 +0100 Frank Elsner wrote: [ ... ] > But ... > > /usr/local/dovecot/lib/dovecot/auth has date Nov 9 10:17 > usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so has date Nov 9 10:17 > > This was the time I've tested dovecot-2.1. After removing all files from Nov 9 10:17 in /usr/local/dovecot/lib/dovecot version 2.0.15 start without the error message :-) --Frank Elsner From dovecot-20110531 at billmail.scconsult.com Fri Nov 11 18:00:47 2011 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Fri, 11 Nov 2011 11:00:47 -0500 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: References: Message-ID: <4EBD46AF.40507@billmail.scconsult.com> Carlos Mennens wrote, On 11/10/11 2:17 PM: > I asked a user today to make sure his incoming and outgoing email was > using TLS. He told me it wasn't possible because my Dovecot / Postfix > daemons were only listening on TCP 25& 143 according to a port scan > he did. He told me the only way I could enable encrypted secure > sessions between the client& server is to enable port 993 (IMAPs). I > told him that TLS is supported on my mail server over the default > ports TCP 25 / 143 and that many consider IMAPs to be legacy. I sent > him a telnet session of my PC communicating with my server& it shows > TLS is available. I just wanted to be sure I was correct with the > information above or am I completely wrong and I do indeed need TCP > port 993? You are correct. To convince your user, point him at http://en.wikipedia.org/wiki/STARTTLS which has links to the relevant RFC's. > > I know this is the Dovecot mailing list but since Dovecot and Postfix > both use and support TLS in their configuration files, I figured I > would ask here for your help! > > carloss at pc1:~$ telnet mail.holyghost.org 25 > Trying 192.168.4.100... > Connected to mail.holyghost.org. > Escape character is '^]'. > 220 mail.holyghost.org ESMTP Postfix > EHLO pc1.holyghost.org > 250-mail.holyghost.org > 250-PIPELINING > 250-SIZE 20480000 > 250-VRFY > 250-ETRN > 250-STARTTLS > 250-ENHANCEDSTATUSCODES > 250-8BITMIME > 250 DSN > > Below is a snip from my mail logs showing TLS: > > Nov 9 10:26:39 mail dovecot: imap-login: Login: user=, > method=PLAIN, rip=::ffff:192.168.4.100, lip=::ffff:192.168.4.100, TLS > > The above snip from my log means that I'm connecting to Dovecot via > TLS, correct? Yes From james.page at ubuntu.com Fri Nov 11 18:32:32 2011 From: james.page at ubuntu.com (James Page) Date: Fri, 11 Nov 2011 16:32:32 +0000 Subject: [Dovecot] Restarting dovecot periodically fails with "Socket already exists: /var/run/dovecot/anvil" In-Reply-To: <1320513622.21919.260.camel@hurina> References: <4E97000D.7030908@ubuntu.com> <1320513622.21919.260.camel@hurina> Message-ID: <4EBD4E20.50800@ubuntu.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 05/11/11 17:20, Timo Sirainen wrote: > http://hg.dovecot.org/dovecot-2.1/rev/6894298ae5fd should apply to > v2.0 as well, but since it's a bit large change I don't know if I > should backport it, at least without some more testing. Thanks for this pointer Timo I've patched this into dovecot in the latest Ubuntu development release - it appears to resolve the issue and we are a good 6 months from release so plenty of time to a) flush out any problems or b) upgrade to 2.1. Cheers James - -- James Page Ubuntu Core Developer -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJOvU4gAAoJEL/srsug59jD7fYQAIy+wnPiYuv58wazO23V4nyA ut2D6iaChsNG7XrySNtgY/RajOu9O8dIOnLuLTD9DmxbRZEaH2Tzd/7n6I0P3i7t h3PyI3tW7GPFyII/VOnAsTjfmncjF/ug7ohyUw2CYc1z2D7lQE0zTFlQeEWCqDCz mxKEHg37TvPlRZzfSvzgjn2hoTJ/vmhQPM11I8uEVYOnWWwahCZxkA2rgBmkjtEG mhCZfgoaJ6BZ5fH0BbjChN5Vr4zuQVEws8fkbGX/7/JPUxKHkz7jdxhVuZr7YN5o xy0HfGRr4sqa1v6qqTAFvFmZu6n3V6dVvn64SkjDLPECnOH80HxPzwMKN9+/cnYi d54/xIYH5N0pPhnSah+/caYCJp5FqDeAmim7xpMLExMzRdqvfbaI7ggGILecN3cH iJiWyj32jCArlAFRFKbxrin8VMRFhU4i+S7zigDp3jFuMotMnL5e1Wk+qrDjKRvt XhairFpT9po5rSvJp/J6qJxWrb+1vznBQkKgl91o570AjBBbRfmYvhxtnU5j2FZ9 MwA8pAD9+n7dc5aGY3o8ABa+5/0wqkAt8qGQ8HYuq+I86NwUbjTadvFLZcfy1Vy5 OOSoMPeruz2tQTCb0/PIHM35MFCmm45Lo0ZamqFmCi4iRKcKwx4tVesJHZY51lGu f29+i9iDlLmofPvhUQkF =hTb+ -----END PGP SIGNATURE----- From dovecot at knutejohnson.com Fri Nov 11 18:33:09 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Fri, 11 Nov 2011 08:33:09 -0800 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBD46AF.40507@billmail.scconsult.com> References: <4EBD46AF.40507@billmail.scconsult.com> Message-ID: <4EBD4E45.4080402@knutejohnson.com> So is a STARTTLS on port 143 session somehow superior to using SSL on port 993? Can you still limit connections on port 143 to require STARTTLS? Thanks, -- Knute Johnson From tkrah at fachschaft.imn.htwk-leipzig.de Fri Nov 11 20:43:22 2011 From: tkrah at fachschaft.imn.htwk-leipzig.de (Torsten Krah) Date: Fri, 11 Nov 2011 19:43:22 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBD4E45.4080402@knutejohnson.com> References: <4EBD46AF.40507@billmail.scconsult.com> <4EBD4E45.4080402@knutejohnson.com> Message-ID: <1321037002.2772.1.camel@sf045> Am Freitag, den 11.11.2011, 08:33 -0800 schrieb Knute Johnson: > So is a STARTTLS on port 143 session somehow superior to using SSL on > port 993? Can you still limit connections on port 143 to require STARTTLS? Yes you can disable authentication until starttls is issued. > > Thanks, > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5253 bytes Desc: not available URL: From bambero at gmail.com Sat Nov 12 06:32:31 2011 From: bambero at gmail.com (Bambero) Date: Sat, 12 Nov 2011 05:32:31 +0100 Subject: [Dovecot] submission_host problem Message-ID: Hello, I configured dovecot to use submission smtp host becouse of chroot. submission_host = 127.0.0.1 Unfortunatelly: Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next input="MAIL FROM:\\r\\n" Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next input="MAIL FROM:\\r\\n" Nov 12 05:11:15 myhost dovecot: lda(alias at apisoft.pl): Error: smtp(127.0.0.1): RCPT TO failed: 554 SMTP synchronization error Nov 12 05:11:15 myhost dovecot: lda(alias at apisoft.pl): Error: sieve: msgid=: failed to redirect message to (refer to server log for more information) Does anyone know how to fix it ? Regards, Bambero From rajeev at teamsg.in Sat Nov 12 09:28:42 2011 From: rajeev at teamsg.in (Rajeev Sharma) Date: Sat, 12 Nov 2011 12:58:42 +0530 Subject: [Dovecot] Migration From Courier Imap. Message-ID: <653a51f07131e61c52bf983359f4710c@192.168.0.251> Sir, I am using vpopmail,qmail along with courier-imap 4.0 i want to migrate from courier to dovecot but i have 100 shared folder under /home/vpopmail/domains/domain.com/Malidir and user shared them on the basis of rights assign. I am very much clear how to migrate from courier to dovecot but i am unable how to migrate shared folder and assign rights. Rajeev Sharma. -- Rajeev Sharma From mnbbrown at gmail.com Sun Nov 13 11:49:21 2011 From: mnbbrown at gmail.com (Matthew Brown) Date: Sun, 13 Nov 2011 19:49:21 +1000 Subject: [Dovecot] Dovecot v2.0.13 unix_listeners Message-ID: <604BA92C-C1D5-4694-B17E-E678C5FFFC0D@gmail.com> Could somebody be able to explain to me what the various unix_listeners do in the Dovecot 2.x configuration (specifically in 10-master.conf). Currently, for postfix to use for local delivery, I have: service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } and for auth I have: service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0666 user = vmail } } So what does each one specifically do? Also, does somebody know of a resource that can explain the mode setting? I couldn't seem to find anything on the wiki.. Sorry if these questions seem simple, Im still trying to get a grips with dovecots workings. MDA: Dovecot v2.0.13 MTA: SMTP User & Virtual Domain Management: MySQL Cheers, Matthew From scott_the_musician at yahoo.com.au Sun Nov 13 12:40:20 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Sun, 13 Nov 2011 02:40:20 -0800 (PST) Subject: [Dovecot] [re] patching dovecot for managesieve/sieve Message-ID: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> My apologies for the repost yesterday; it was not intentional. I was attempting to post a followup from my mobile, when something went awry with the editing options on my phone. Anyway.. ?I have managed to download and compile my own patched dovecot/dovecot-sieve/dovecot-managesieve modules from scratch, with Thanks to this page hardc0l2e.wordpress.com/2010/02/26/managesieve-with-dovecot-on-centos-5-4/ But when I run http://mydomain.com/src/configtest.php , I am still getting the following rather non-descriptive error: "I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=ManageSieve) is bad?" Where else can I look to resolve this? should managesieve show up as its own /etc/init.d service? Or does dovecot simply control it? How can I confirm that deliver is being used as my LDA? Thanks again, Scott From andreas at kado-web.de Sun Nov 13 16:48:49 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Sun, 13 Nov 2011 15:48:49 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive Message-ID: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> Hi List, i am trying to restore mails from an backup-archive which was made with rsnapshot. All the mails in the archive look similar like the following: -rw------- 2 1014 1015 3308 8. Aug 13:42 1312803768.V809I266a00aM505178.server:2,ST Now I copy the files from the archive into the Maildir/cur directory of the user. Then I set the permissions etc. and log in with the user account via Squirrelmail. The restored mails aren't in the INBOX, and when I go into the Maildir/cur dir of the user, the restored mails are gone. Possibly deleted after the login. Had anyone some similar experiences ? Any hints ? Kind regards Andreas From bambero at gmail.com Sun Nov 13 18:25:55 2011 From: bambero at gmail.com (Bambero) Date: Sun, 13 Nov 2011 17:25:55 +0100 Subject: [Dovecot] Chroot Message-ID: Hello, I have successfully setup dovecot with chroot but there are two things yet 1. submission_host problem (I posted message 2 days ago) 2. Problem with acl's and sharing: all maildirs are located in: /var/mail/jdoe at example.com/Maildir so: mail_chroot=/var/mail I setup acls with: 10-mail.conf ... namespace { type = private separator = / prefix =. inbox = yes hidden = no list = yes subscriptions = yes } namespace { type = shared separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = yes list = yes } ... 90-acl.conf ... plugin { acl = vfile:cache_secs=5 } plugin { acl_shared_dict = file:/%i.db } ... Setting acl is no problem there are dovecot-acl dovecot-acl-list in Maildir. and dictionary /var/mail/xxx.db contains list od shares I can also see shared folder in my IMAP client but when trying to subscribe i got error: Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No such file or directory Error: Namespace 'shared/': Could not lookup home for user jdoe at example.com /var/dovecot/auth-userdb is not accessible because chroot. But why userdb is necessary in this case ? I hope there is a solution without disabling chroot. Below my basic configuration. Regards, Bambero # 2.0.13: /etc/dovecot/dovecot.conf # OS: OpenBSD 5.0 i386 auth_mechanisms = plain login digest-md5 cram-md5 apop default_internal_user = _dovecot default_login_user = _dovenull first_valid_gid = 1000 first_valid_uid = 1000 last_valid_gid = 10000 last_valid_uid = 10000 mail_chroot = /var/mail mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date imapflags notify mbox_write_locks = fcntl mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir:User quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_global_path = /etc/sieve/default.sieve } protocols = imap pop3 sieve sieve service auth { unix_listener auth-client { mode = 0660 user = _exim } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/ca-bundle.crt ssl_cert = References: <604BA92C-C1D5-4694-B17E-E678C5FFFC0D@gmail.com> Message-ID: <20111113163741.Horde.s9ZFZpLnE6FOwDilUtzVjrA@kishi.patrickdk.com> Heh? they pretty much do exactly as described. the service lmtp is how you inject email into dovecot using the lmtp protocol. the service auth is how you can authenicate user/password against dovecot, normally used in postfix for sasl auth the mode setting is just that, the file mode to set for that file, 0666, all access, 0660 only user/group access. Quoting Matthew Brown : > Could somebody be able to explain to me what the various > unix_listeners do in the Dovecot 2.x configuration (specifically in > 10-master.conf). > > Currently, for postfix to use for local delivery, I have: > > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > > and for auth I have: > > service auth { > > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > > unix_listener auth-userdb { > mode = 0666 > user = vmail > } > } > > So what does each one specifically do? > > Also, does somebody know of a resource that can explain the mode > setting? I couldn't seem to find anything on the wiki.. > Sorry if these questions seem simple, Im still trying to get a grips > with dovecots workings. > > > MDA: Dovecot v2.0.13 > MTA: SMTP > User & Virtual Domain Management: MySQL > > Cheers, > Matthew From georg.martius at web.de Sun Nov 13 23:57:46 2011 From: georg.martius at web.de (Georg Martius) Date: Sun, 13 Nov 2011 22:57:46 +0100 Subject: [Dovecot] metadata plugin problems Message-ID: <3455528.juJhefTHFy@chugul> Hi, I have trouble with the metadata plugin. I compiled dovecot 2.0.15 from source and also the plugin [2] separately. I work on debian/lenny. I had some trouble compiling it (autoconf somehow didn't insert the DOVECOT_IMAP_INCLUDE variable, I have not idea why) Anyway, I did as described in [1]. Do I have to create: the file /var/lib/dovecot/shared-metadata ? The server starts without errors, but I get for instance the following error message: "Failed to write some annotations for 'Contacts' on the IMAP server. SetMetaData failed, server replied: A000076 NO Parsing entry failed" from my KMail. How can I test that the plugin is actually active and working? Any help would be appreciated. Regards, Georg [1]http://www.mail-archive.com/dovecot at dovecot.org/msg38887.html [2] http://hg.dovecot.org/dovecot-metadata-plugin From dovecot-20110531 at billmail.scconsult.com Mon Nov 14 00:27:02 2011 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Sun, 13 Nov 2011 17:27:02 -0500 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> References: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> Message-ID: <4EC04436.3090905@billmail.scconsult.com> Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From pw at wk-serv.de Mon Nov 14 01:31:24 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 14 Nov 2011 00:31:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EC0534C.8070409@wk-serv.de> Peer Heinlein schrieb: > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. You have 11 TB of mails on a non cluster filesystem? Is it only accessed from one server or how does it work? From andreas at kado-web.de Mon Nov 14 08:09:01 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Mon, 14 Nov 2011 07:09:01 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <4EC04436.3090905@billmail.scconsult.com> Message-ID: <0MHQzX-1RdoD93uYl-00Dcge@mrelayeu.kundenserver.de> Thanks for that information bill. The T was the point which was important for me, because the user didn't tell me that the messages were already marked for deletion. - Anyway, in outlook the mails were visible (marked for deletion). -----Urspr?ngliche Nachricht----- Von: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Bill Cole Gesendet: Sonntag, 13. November 2011 23:27 An: Dovecot Mailing List Betreff: Re: [Dovecot] Restoring Mails from Backup Archive Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From andreas at kado-web.de Mon Nov 14 08:09:20 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Mon, 14 Nov 2011 07:09:20 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <4EC04436.3090905@billmail.scconsult.com> Message-ID: <0M2TGj-1QaLdl2jLB-00sOsn@mrelayeu.kundenserver.de> Thanks for that information bill. The T was the point which was important for me, because the user didn't tell me that the messages were already marked for deletion. - Anyway, in outlook the mails were visible (marked for deletion). -----Urspr?ngliche Nachricht----- Von: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Bill Cole Gesendet: Sonntag, 13. November 2011 23:27 An: Dovecot Mailing List Betreff: Re: [Dovecot] Restoring Mails from Backup Archive Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From p.heinlein at heinlein-support.de Mon Nov 14 11:34:02 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Mon, 14 Nov 2011 10:34:02 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EC0534C.8070409@wk-serv.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> Message-ID: <201111141034.03060.p.heinlein@heinlein-support.de> Am Montag, 14. November 2011, 00:31:24 schrieb Patrick Westenberg: > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > HP EVA. > > You have 11 TB of mails on a non cluster filesystem? Yes. I don't believe a clustered filesystem would have more performance and would be more rock solid. I don't have a problem on my frontend server. Why should I have two or more of them? I have a problem in my backend. My SAN has too much to do. Why should a cluster filesystem be better for my SAN? > Is it only accessed from one server or how does it work? Yes. peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From arif at team.de.inter.net Mon Nov 14 11:51:20 2011 From: arif at team.de.inter.net (Kai Arif) Date: Mon, 14 Nov 2011 10:51:20 +0100 Subject: [Dovecot] patch for UID 0 problem Message-ID: <4EC0E498.7000900@team.de.inter.net> Hi all, attached you'll find a patch for cmd-thread.c which addresses a problem with certain clients in conjunction with the use of the UID THREAD REFS command: TAG5 UID THREAD REFS us-ascii SINCE 14-May-2011 * THREAD (0)(246)(247)(248)(249)(250)(252)(253)(254)(255)(256)(257)(258)(259)(260)(261)(262)(263)(264)(265)(266)(267)(268) TAG5 OK Thread completed. The first UID delivered (0) is invalid, some clients - @mail for instance - then try to fetch the invalid UID 0 subsequently and run into problems. The patch actually disables the return of (0) and seems to work for us. Please review and - if OK - incorporate it in the next version. Best Regards Kai -- Kai Arif - System Administrator Inter.net Germany GmbH Knesebeckstra?e 59-61 10719 Berlin Germany Fon +49 30 25430 0 Fax +49 30 25430 499 arif at team.de.inter.net www.de.inter.net Legal-Information: Inter.net Germany GmbH, HRB #79136, Amtsgericht Berlin Charlottenburg, UST-IdNr:: DE 813 165 159 FA f?r K?rperschaften I Berlin, Gesch?ftsf?hrer: J?rn Lubkoll Zust?ndige Regulierungsbeh?rde: Bundesnetzagentur Chemnitz, Liselotte-Herrmann-Str. 20a, 09127 Chemnitz, Registriernummer: 06/164 -------------- next part -------------- A non-text attachment was scrubbed... Name: cmd-thread.c.diff Type: text/x-diff Size: 644 bytes Desc: not available URL: From stephan at rename-it.nl Mon Nov 14 12:06:22 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 14 Nov 2011 11:06:22 +0100 Subject: [Dovecot] [re] patching dovecot for managesieve/sieve In-Reply-To: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> References: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> Message-ID: <4EC0E81E.2090401@rename-it.nl> On 13-11-2011 11:40, Scott Lewis wrote: > My apologies for the repost yesterday; it was not intentional. I was attempting to post a followup from my mobile, when something went awry with the editing options on my phone. > Anyway.. > > I have managed to download and compile my own patched dovecot/dovecot-sieve/dovecot-managesieve modules from scratch, with Thanks to this page > > hardc0l2e.wordpress.com/2010/02/26/managesieve-with-dovecot-on-centos-5-4/ > > But when I run http://mydomain.com/src/configtest.php , I am still getting the following rather non-descriptive error: > > "I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=ManageSieve) is bad?" You can test whether ManageSieve is working properly using this procedure: http://wiki.dovecot.org/ManageSieve/Troubleshooting > Where else can I look to resolve this? should managesieve show up as its own /etc/init.d service? Or does dovecot simply control it? > How can I confirm that deliver is being used as my LDA? The Dovecot LDA produces characteristic log lines in your system log (http://wiki1.dovecot.org/Logging). For instance, with Sieve enabled, the following is reported upon delivery to INBOX: Nov 14 10:53:41 host dovecot: deliver(stephan): sieve: msgid=: stored mail into mailbox 'INBOX' Something similar is logged without Sieve. All such messages are prefixed with 'deliver(): '. With mail_debug=yes, more verbose messages are logged as well. Regards, Stephan. From janfrode at tanso.net Mon Nov 14 12:27:19 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 14 Nov 2011 11:27:19 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111141034.03060.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> Message-ID: <20111114102719.GA2330@dibs.tanso.net> On Mon, Nov 14, 2011 at 10:34:02AM +0100, Peer Heinlein wrote: > > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > > HP EVA. > > > > You have 11 TB of mails on a non cluster filesystem? > > Yes. > > I don't believe a clustered filesystem would have more performance and > would be more rock solid. > > I don't have a problem on my frontend server. Why should I have two or > more of them? I have a problem in my backend. My SAN has too much to do. > Why should a cluster filesystem be better for my SAN? Agree. A non-clustered fs should give you better performance, and probably also be more reliable, if you can live with the SPoF and full downtime during patching/upgrades/maintenance. But I would expect xfs to be a better choice than ext*. We have about the same storage size as you (12TB/115M-inodes), with the backup-process almost biting itself in the tail every day, but I can't quite imagine running it all on a single local fs with no scale-out options if we should want/need more processing power for dovecot. I'm looking forward to moving to mdbox soonish.. to reduce the number of files and speed up the backup process. -jf From tlx at leuxner.net Mon Nov 14 13:30:08 2011 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 14 Nov 2011 12:30:08 +0100 Subject: [Dovecot] Managing public folder ACL files In-Reply-To: <1302268626.24579.79.camel@hurina> References: <20110406115434.3049ba70@kae.tiger-computing.wbp> <1302268626.24579.79.camel@hurina> Message-ID: <20111114113008.GA5126@nihlus.leuxner.net> On Fri, Apr 08, 2011 at 04:17:06PM +0300, Timo Sirainen wrote: > > doveadm mailbox list -u admin at domain "sales/CustA/*" | > while read mailbox; do > doveadm acl set -u user at domain "$mailbox" user at domain +lrw > done Have the +lrw modifiers been implemented at all? I can only get it to work with syntax like: $ doveadm acl set -u user at domain "Shops/ACME" owner insert lookup post read write write-seen This is a bit painful to type and does not allow to add or remove rights... Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From achekalin at lazurit.com Mon Nov 14 16:35:07 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 14 Nov 2011 17:35:07 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EC1271B.4060604@lazurit.com> Timo, Stan, I've just tested mdbox and find it pretty nice for me, but now I got some questions for you: 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if 2Mb. Looks like not even every message can fit into such storage container volume (nowadays we used to see messages of 20Mb and even more). Should I tune it (at least mdbox_rotate_size and mdbox_rotate_interval) or its size is on purpose? As for now I store each day's messages in separate IMAP folders (mailboxes), which gives me 2000-6000 messages and 2-5 Gb (on disk) per folder. 2. I can use no compression, gz and bz2 - which one will be better for storing archive messages? I've just tested mdbox by copying 5800+ msgs from maildir to compressed mdbox, and it took exactly the same size (2.8 G) in 100+ small m.* files. No good as far. 3. What if I use maildir as I do now but turn on compression, will this speed things up? I'd like to use mdbox as storage but for now it is very new for me and I simple afraid what should I do if I'll need to manually fix the storage (maildir is really good for that, surely). After all, I simple need to speed up the search and restore process in archive. Yours, Alexander From stan at hardwarefreak.com Mon Nov 14 20:23:14 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 14 Nov 2011 12:23:14 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC1271B.4060604@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> Message-ID: <4EC15C92.4000707@hardwarefreak.com> On 11/14/2011 8:35 AM, Alexander Chekalin wrote: > Timo, Stan, > > I've just tested mdbox and find it pretty nice for me, but now I got > some questions for you: > > 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if > 2Mb. Looks like not even every message can fit into such storage > container volume (nowadays we used to see messages of 20Mb and even > more). Should I tune it (at least mdbox_rotate_size and > mdbox_rotate_interval) or its size is on purpose? As for now I store > each day's messages in separate IMAP folders (mailboxes), which gives me > 2000-6000 messages and 2-5 Gb (on disk) per folder. mdbox_rotate_size of 2MB is too small for your needs. Test 32MB and 64MB. > 2. I can use no compression, gz and bz2 - which one will be better for > storing archive messages? I've just tested mdbox by copying 5800+ msgs > from maildir to compressed mdbox, and it took exactly the same size (2.8 > G) in 100+ small m.* files. No good as far. bzip2 may give you a little better compression but at the cost of much lower de/compression speed and higher CPU and memory consumption. gzip will be faster all around, between 4x-8x, with lower mem usage, but with less compression resulting in slightly larger file sizes than bzip2. > 3. What if I use maildir as I do now but turn on compression, will this > speed things up? No. Maildir performance is limited by the disk head actuator speed, which is between 150-300 seeks per second depending on your disk (7.2k vs 15k RPM). Compressing the files doesn't change the seek physics of the disk drives. You're still reading tens of thousands of files when doing your searches thus bouncing the heads tens of thousands of times. mbox uses a single file, so head speed isn't a factor, as it may only move a few times when reading an entire mailbox file. Thus, bandwidth becomes the potential bottleneck. Using compression with large mbox files can substantially increase search performance as effective bandwidth is increased by ~4x using gzip and 6x using bzip2. This assumes you have plenty of excess CPU power. mdbox should see similar compression speedups if you use file sizes much larger than the 2MB default. Doing so should keep your IOPS well below the drive's head saturation point as you're reading only a fraction of the file count compared to maildir. > I'd like to use mdbox as storage but for now it is very new for me and I > simple afraid what should I do if I'll need to manually fix the storage > (maildir is really good for that, surely). Doveadm handles such tasks pretty well. Just make sure you keep good backups of your mdbox files. > After all, I simple need to speed up the search and restore process in > archive. The only way to accomplish this with maildir is with much bigger, faster, more expensive storage hardware. And the gain will still be much less than simply switching to a larger file format such as mbox or mdbox. As with many things some computer technologies come full circle over time. One of the reasons the creators of the UNIX mbox mail file format decided upon a single file many decades ago was the horribly limited seek performance of the slow SCSI disks of that period. Doing something like the maildir format was simply impossible at that time. In the early days of the public internet, disk became faster than the average load and maildir was born to fix the locking and corruption shortcomings of mbox. Today many sites are hitting the seek problem of a few decades ago because boxes are oversubscribed with users, emails now frequently contain attachments, everyone is storing more email, and the total volume of email is a few orders of magnitude greater. IIRC, this is one of the reasons Timo created mdbox--to decrease the massive IOPS load, and thus slow performance, of large maildir stores. -- Stan From moseleymark at gmail.com Mon Nov 14 20:23:37 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 14 Nov 2011 10:23:37 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: Message-ID: On Mon, Sep 26, 2011 at 10:11 AM, Mark Moseley wrote: > On Fri, Sep 23, 2011 at 3:35 PM, Mark Moseley wrote: >> I've been goofing with this all day with 2.0.15 and I'm starting to >> realize that either a) I'm not that smart, b) it's been so long since >> I messed with namespaces that I'm going about it completely wrong, or >> c) it's just not possible. I haven't posted 'doveconf -n' and other >> details, because mainly I'm just looking for 'yes, this is possible' >> or 'no, you're smoking crack' before posting further details. At this >> point, it's all maildir and moving to mdbox, while highly desirable in >> the future, is not possible in the near- to medium-term. >> >> I'm trying to glue a namespace underneath INBOX: >> >> namespace INBOX { >> ? ? ? ?type = private >> ? ? ? ?separator = . >> ? ? ? ?prefix = INBOX. ? ?# Yes, this used to be on Courier >> ? ? ? ?inbox = yes >> ? ? ? ?list = yes >> ? ? ? ?hidden = no >> ? ? ? ?subscriptions = yes >> ? ? ? ?location = maildir:~/Maildir >> } >> namespace archive { >> ? ? ? ?type = private >> ? ? ? ?separator = . >> ? ? ? ?prefix = INBOX.Archives. >> ? ? ? ?inbox = no >> ? ? ? ?list = children >> ? ? ? ?subscriptions = yes >> ? ? ? ?location = maildir:~/Maildir-Archive >> } >> >> >> I've tried putting namespace archive's 'prefix' as just "Archives", >> but Tbird doesn't seem to see this namespace, regardless of how much I >> futz with the imap settings in tbird. >> >> With the above setup, it actually seems to work correctly (provided >> ~/Maildir-Archive exists), though I'm sure a big gotcha is waiting in >> the wings. I can move messages around, create subfolders, subscribe to >> folders in ~/Maildir-Archive). The only thing I can't seem to get >> working is quotas. With my password_query like: >> >> password_query = ... >> CONCAT( '*:bytes=', "1M" ) AS 'userdb_quota_rule', \ >> CONCAT( '*:messages=10' ) AS 'userdb_quota_rule2', \ >> CONCAT( 'INBOX.Archives:bytes=+4900M' ) AS 'userdb_quota_rule3', \ >> CONCAT( 'INBOX.Archives:messages=+3900' ) AS 'userdb_quota_rule4' >> ... >> >> only the default quota seems to be in place for any subfolder of >> INBOX.Archives and for INBOX.Archives itself, i.e. *:bytes still >> applies to INBOX.Archives. The debug log show that: >> >> Debug: Quota root: name=User quota backend=maildir args= >> Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=0 >> Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=10 >> Debug: Quota rule: root=User quota mailbox=INBOX.Archives >> bytes=+5138022400 messages=0 >> Debug: Quota rule: root=User quota mailbox=INBOX.Archives >> bytes=+5138022400 messages=+3900 >> >> These are wildly stupid quotas but they're just there to test. With >> INBOX already at capacity (byte-wise; only set to a meg), copying >> large messages inside INBOX.Archives fails (only copying a 800k >> message but the quota should be 5gig now). >> >> Again, before I post configs, I'm just curious if what I'm trying to >> do isn't remotely possible, or that I'm approaching this entirely >> wrongly. Thanks! >> > > Thanks to a fortuitously unrelated thread ("how to disable quota for > second namespace"), I got the quota part figured out and that seems to > be working: Add a second entry to plugin {}, e.g. "quota2 = > maildir:Archive quota:ns=INBOX.Archives." and add rules for > userdb_quota2_rule, userdb_quota2_rule2, etc. > > My real question now is: Are there any fatal gotchas in this that I'm > just not thinking of? > Haven't had a chance to try this large-scale yet. Anybody have any thoughts on it? From stan at hardwarefreak.com Mon Nov 14 20:42:30 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 14 Nov 2011 12:42:30 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111114102719.GA2330@dibs.tanso.net> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> Message-ID: <4EC16116.1000902@hardwarefreak.com> On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > Agree. A non-clustered fs should give you better performance, and > probably also be more reliable, if you can live with the SPoF and > full downtime during patching/upgrades/maintenance. But I would expect > xfs to be a better choice than ext*. Depends on the workload characteristics and how well the XFS filesystem is tuned to the storage hardware. If setup properly, using many allocation groups with fast spindles, a decent amount of BBWC, and a high concurrency maildir workload (dozens to hundreds of delivery and IMAP operations), XFS will runs circles around EXTx as it can create/write/read to every AG in parallel. Much of EXT4's operation is still serialized. This is why XFS outruns all other filesystems in the highly parallel mail workload benchmarks I posted previously, EXTx by a factor of 2-3. For smaller hosts that don't see parallelism, for example SOHO servers, XFS will likely be slower than EXTx as the workload will be serialized. -- Stan From achekalin at lazurit.com Mon Nov 14 23:16:53 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 15 Nov 2011 00:16:53 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC15C92.4000707@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> Message-ID: <43B341A1-9245-4863-932A-887E21970319@lazurit.com> Locking issues on mbox is the reason for my long-lasting love affair with maildir, and it's lasts long years. Ok, the life's lessons are like this, learn something and move on with it ;) even if it's "new old thing". Thank you for pointing that! What I was doubt about is default rotate size of 2M, since I used to see pretty reasonable default settings in all Dovecot config. 32 or 64 are much close to the ones I'd personally prefer. I also about to choose now is the OS and FS for the archive. I seriously think about ZFS with compression (in fact it will be stripes over couple of mirrors = software equivalent of RAID 10 on SATA drives, with compression on FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox itself. I see pros and contras for both, so that's the question to answer! Yours, Alexander > On 11/14/2011 8:35 AM, Alexander Chekalin wrote: >> Timo, Stan, >> >> I've just tested mdbox and find it pretty nice for me, but now I got >> some questions for you: >> >> 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if >> 2Mb. Looks like not even every message can fit into such storage >> container volume (nowadays we used to see messages of 20Mb and even >> more). Should I tune it (at least mdbox_rotate_size and >> mdbox_rotate_interval) or its size is on purpose? As for now I store >> each day's messages in separate IMAP folders (mailboxes), which gives me >> 2000-6000 messages and 2-5 Gb (on disk) per folder. > > mdbox_rotate_size of 2MB is too small for your needs. Test 32MB and 64MB. > >> 2. I can use no compression, gz and bz2 - which one will be better for >> storing archive messages? I've just tested mdbox by copying 5800+ msgs >> from maildir to compressed mdbox, and it took exactly the same size (2.8 >> G) in 100+ small m.* files. No good as far. > > bzip2 may give you a little better compression but at the cost of much > lower de/compression speed and higher CPU and memory consumption. gzip > will be faster all around, between 4x-8x, with lower mem usage, but with > less compression resulting in slightly larger file sizes than bzip2. > >> 3. What if I use maildir as I do now but turn on compression, will this >> speed things up? > > No. Maildir performance is limited by the disk head actuator speed, > which is between 150-300 seeks per second depending on your disk (7.2k > vs 15k RPM). Compressing the files doesn't change the seek physics of > the disk drives. You're still reading tens of thousands of files when > doing your searches thus bouncing the heads tens of thousands of times. > > mbox uses a single file, so head speed isn't a factor, as it may only > move a few times when reading an entire mailbox file. Thus, bandwidth > becomes the potential bottleneck. Using compression with large mbox > files can substantially increase search performance as effective > bandwidth is increased by ~4x using gzip and 6x using bzip2. This > assumes you have plenty of excess CPU power. mdbox should see similar > compression speedups if you use file sizes much larger than the 2MB > default. Doing so should keep your IOPS well below the drive's head > saturation point as you're reading only a fraction of the file count > compared to maildir. > >> I'd like to use mdbox as storage but for now it is very new for me and I >> simple afraid what should I do if I'll need to manually fix the storage >> (maildir is really good for that, surely). > > Doveadm handles such tasks pretty well. Just make sure you keep good > backups of your mdbox files. > >> After all, I simple need to speed up the search and restore process in >> archive. > > The only way to accomplish this with maildir is with much bigger, > faster, more expensive storage hardware. And the gain will still be > much less than simply switching to a larger file format such as mbox or > mdbox. > > As with many things some computer technologies come full circle over > time. One of the reasons the creators of the UNIX mbox mail file format > decided upon a single file many decades ago was the horribly limited > seek performance of the slow SCSI disks of that period. Doing something > like the maildir format was simply impossible at that time. In the > early days of the public internet, disk became faster than the average > load and maildir was born to fix the locking and corruption shortcomings > of mbox. > > Today many sites are hitting the seek problem of a few decades ago > because boxes are oversubscribed with users, emails now frequently > contain attachments, everyone is storing more email, and the total > volume of email is a few orders of magnitude greater. > > IIRC, this is one of the reasons Timo created mdbox--to decrease the > massive IOPS load, and thus slow performance, of large maildir stores. > > -- > Stan From steveh at itbasecamp.com.au Tue Nov 15 02:08:33 2011 From: steveh at itbasecamp.com.au (Steve H) Date: Tue, 15 Nov 2011 00:08:33 +0000 Subject: [Dovecot] XUIDL Migration problems Message-ID: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> Hello All, I am new to Dovecot and the Dovecot mailing list. I am trying to migrate an existing Exim + qpopper - mbox (pop3 only) installation over to Exim + Dovecot using mdir (pop3 and imap) And am running into problems with preserving the XUIDL headers. I am running Dovecot 2.0.14 I have used the mb2md.pl script : http://dovecot.org/tools/mb2md.pl from this page http://wiki2.dovecot.org/Migration/MailFormat to migrate and import the mboxes And have the following pop3 uidl settings in my configuration . pop3_reuse_xuidl = yes pop3_save_uidl = yes protocols = imap pop3 service imap-login { inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } protocol pop3 { pop3_reuse_xuidl = yes pop3_save_uidl = yes pop3_uidl_format = %08Xu%08Xv } The problem specifically is that when I test the migration and import a mailbox, I am able to connect via IMAP using Thunderbird and Outlook 2007 and the Read / Unread status is preserved. However when I connect via POP3 the Read/Unread status is not preserved. I am probably missing something simple, but have been unable to find anything that seems relevant on the Dovecot wiki or searching the mailing lists. Any suggestions would be appreciated. Regards, Steven Holloway IT Technical Consultant I.T. Basecamp P/L [http://www.itbasecamp.com.au/signature.gif] Address: Office 1 Level 1/50 Owen St Huskisson NSW 2540 Phone: +61 2 44418555 ext 308 Fax: +61 2 44418551 Web: http://www.itbasecamp.com.au Subscribe to our newsletter! Visit: IT Basecamp to manage your subscription. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 2602 bytes Desc: image001.gif URL: From tss at iki.fi Tue Nov 15 02:19:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:19:17 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC1271B.4060604@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> Message-ID: <23D65BA4-912A-43EE-8480-8586DEAD76FA@iki.fi> On 14.11.2011, at 16.35, Alexander Chekalin wrote: > 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if 2Mb. Looks like not even every message can fit into such storage container volume (nowadays we used to see messages of 20Mb and even more). The messages are never split into multiple files. So if you have a 20 MB message, it gets stored into its own m.* file. > Should I tune it (at least mdbox_rotate_size and mdbox_rotate_interval) or its size is on purpose? As for now I store each day's messages in separate IMAP folders (mailboxes), which gives me 2000-6000 messages and 2-5 Gb (on disk) per folder. The main problem with larger mdbox files is that if you expunge messages, there's more data to write when packing the data into a new file. I don't really know the "best" value for mdbox_rotate_size setting. But even a 2 MB mdbox file can contain thousands of small mails, so it's not too bad.. From tss at iki.fi Tue Nov 15 02:21:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:21:09 +0200 Subject: [Dovecot] XUIDL Migration problems In-Reply-To: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> References: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> Message-ID: <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> On 15.11.2011, at 2.08, Steve H wrote: > I am trying to migrate an existing Exim + qpopper - mbox (pop3 only) installation over to Exim + Dovecot using mdir (pop3 and imap) > And am running into problems with preserving the XUIDL headers. Are you sure the UIDLs are the problem? Did you connect to old and new POP3 server and compare their UIDL lists? > However when I connect via POP3 the Read/Unread status is not preserved. Another possibility would be simply that if you changed the hostname in the client config, it might have thought that it's a completely different server. From tss at iki.fi Tue Nov 15 02:38:15 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:38:15 +0200 Subject: [Dovecot] patch for UID 0 problem In-Reply-To: <4EC0E498.7000900@team.de.inter.net> References: <4EC0E498.7000900@team.de.inter.net> Message-ID: On 14.11.2011, at 11.51, Kai Arif wrote: > attached you'll find a patch for cmd-thread.c which addresses a problem with certain clients in > conjunction with the use of the UID THREAD REFS command: > > TAG5 UID THREAD REFS us-ascii SINCE 14-May-2011 > * THREAD > (0)(246)(247)(248)(249)(250)(252)(253)(254)(255)(256)(257)(258)(259)(260)(261)(262)(263)(264)(265)(266)(267)(268) > TAG5 OK Thread completed. > > The first UID delivered (0) is invalid, some clients - @mail for instance - then try to fetch the > invalid UID 0 subsequently and run into problems. Yes, it is. It should never happen. > The patch actually disables the return of (0) and seems to work for us. Please review and - if OK - > incorporate it in the next version. Could you instead send me such a mailbox where you can reproduce this problem? Probably sending dovecot.index, dovecot.index.log and dovecot.index.thread files would be enough. None of those contain any sensitive information. From steveh at itbasecamp.com.au Tue Nov 15 03:14:26 2011 From: steveh at itbasecamp.com.au (Steve H) Date: Tue, 15 Nov 2011 01:14:26 +0000 Subject: [Dovecot] XUIDL Migration problems In-Reply-To: <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> References: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> Message-ID: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8DB2@ITBEXCH.office.itb.net.au> Hi Timo, Thanks for the quick reply. > Are you sure the UIDLs are the problem? Did you connect to old and new POP3 server and compare their UIDL lists? No not 100% sure that the UIDLs are the issue; but running into dead ends with what I have tried. I will compare the UIDL lists from the old / new server and report back. > However when I connect via POP3 the Read/Unread status is not preserved. > Another possibility would be simply that if you changed the hostname in the client config, it might have thought that it's a completely different server. The new server is a completely different host and currently had a different IP, but it will end up with the same name and IP as the existing host when we cut over. At this time I have manually adjusted my lost hosts file to point towards the new host for testing. Would this be a problem even though the IMAP appears to be working were as the POP is not.?? I'll report back on the UIDLs Regards, Steven From mysqlstudent at gmail.com Tue Nov 15 04:30:35 2011 From: mysqlstudent at gmail.com (Alex) Date: Mon, 14 Nov 2011 21:30:35 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail Message-ID: Hi, I have a fedora15 system with dovecot-v2.0.15, dovecot-pigeonhole and and amavisd-new and would like to implement it on my system using postfix. I've read the basic wiki documentation, but I really can't figure out the proper starting point in my case. I've enabled "protocol lda" in 15-lda.conf and added "sieve" to the list of mail_plugins. I'm trying to adapt postfix to use deliver as its LDA for multiple virtual domains. I've added the following to master.cf: dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient} -a "${RECIPIENT}" Is that the proper format? Is it okay to use the dovecot user here, or should I create a new user? What permissions does this user need to possess? I've added the following to main.cf: dovecot_destination_recipient_limit = 1 virtual_transport = dovecot I have a number of procmail scripts and I've converted them to sieve, but they didn't convert very well. Can someone show me how to convert the following: :0 { :0 fhW | /home/alex/bin/process-mail.pl } The script skipped right over that. I'm also trying to convert the following: :0 * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) * !USER_IN_WHITELIST * !BAYES_00 * !URIBL_ * !KHOP_RCVD_UNTRUST * !RCVD_IN_DNSWL_MED * !SPF_SOFTFAIL * SPF_HELO_PASS * SPF_PASS { LOG="DNSWL: " :0: whitelist/dnswl } The script only produced the following: if true { if true { if true { if true { if true { if true { if true { if true { if true { if true { fileinto "{"; fileinto " LOG="DNSWL: ""; fileinto " :0:"; fileinto " whitelist/dnswl"; fileinto "}"; } Any ideas greatly appreciated. Thanks, Alex From JTRUTWIN at CSBSJU.EDU Tue Nov 15 05:34:55 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Tue, 15 Nov 2011 03:34:55 +0000 Subject: [Dovecot] getenv() in plugin not working Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> Hi - new to the list, can't find much on this using google. I'm trying to setup the dovecot DRAC plugin as described here: http://wiki.dovecot.org/HowTo/PopBSMTPAndDovecot#DRAC DRAC is installed and running using this startup command: /usr/local/sbin/rpc.dracd -i -e 5 /etc/postfix/dracd.db I downloaded the drac.c file linked on the above URL: http://www.dovecot.org/patches/1.1/drac.c and built it using this command: gcc -Wall -W -shared -fPIC -DHAVE_CONFIG_H drac.c -I/usr/include/dovecot -L/usr/lib64/dovecot -o drac.so -ldrac Then I copied drac.so to /usr/lib64/dovecot (this is an x86_64 SL 6.1 system - dovecot 2.0.9) and modified my /etc/dovecot/conf.d/20-imap.conf file: mail_plugin_dir = /usr/lib64/dovecot mail_plugins = drac Restart dovecot and try to login and getting these messages for ANY login attempts: dovecot: imap-login: Login: user=, method=PLAIN, rip=, lip=, mpid=4018, TLS server dovecot: imap(: Error: DRAC: IP environment not given I traced the "IP environment not given" to the drac.c plugin file, and the code is pretty simple: void drac_init(void) { const char *ip_str; struct ip_addr ip; ip_str = getenv("IP"); if (ip_str == NULL) i_error("DRAC: IP environment not given"); Is there anything special that needs to be done for getenv("IP") to report the login IP? I tried to put getenv("RIP") for remote ip and even getenv("USER") as well but always get NULL. Is this API only for dovecot 1.x? If so is there a 2.0 equivalent? The rest of the plugin is very basic... I think this is the last step in getting this setup running. I'd have to have this one last piece force me to run that icky perl program instead. Thank you, Let me know if anything else is needed. Josh From JTRUTWIN at CSBSJU.EDU Tue Nov 15 05:58:55 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Tue, 15 Nov 2011 03:58:55 +0000 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> > I'm trying to setup the dovecot DRAC plugin as described here: > Is there anything special that needs to be done for getenv("IP") to report the > login IP? I tried to put getenv("RIP") for remote ip and even getenv("USER") > as well but always get NULL. Is this API only for dovecot 1.x? If so is there > a 2.0 equivalent? The rest of the plugin is very basic... Appears this is the problem - I found this for dovecot v2 and it seems to be working fine: http://sourceforge.jp/projects/dovecot2-drac/ Might wanna make a note on the wiki page that the DRAC setup is only for dovecot 1.x? Thanks, Josh From stephan at rename-it.nl Tue Nov 15 09:41:37 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 15 Nov 2011 08:41:37 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: Message-ID: <4EC217B1.2040809@rename-it.nl> On 11/15/2011 3:30 AM, Alex wrote: > I have a number of procmail scripts and I've converted them to sieve, > but they didn't convert very well. Can someone show me how to convert > the following: > > :0 > { > :0 fhW > | /home/alex/bin/process-mail.pl > } > > The script skipped right over that. The Sieve language doesn't support invoking external programs. You can execute this program before Sieve, for instance by wrapping the LDA invocation. Also, there is a non-standard (an still pretty much experimental) language extension for this, as provided by the pipe plugin: http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe > I'm also trying to convert the following: > > :0 > * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) > * !USER_IN_WHITELIST > * !BAYES_00 > * !URIBL_ > * !KHOP_RCVD_UNTRUST > * !RCVD_IN_DNSWL_MED > * !SPF_SOFTFAIL > * SPF_HELO_PASS > * SPF_PASS > { > LOG="DNSWL: " > :0: > whitelist/dnswl > } > > The script only produced the following: [...] > Any ideas greatly appreciated. I'm not sure what these rules are supposed to mean (most conditions are obscured in variables), but by the looks of things these should also be executed outside Sieve as well. If all those conditions map to simple header and envelope tests, you can provide a Sieve alternative. However, I think USER_IN_WHITELIST queries some external list and that too is not (currently) possible from Pigeonhole Sieve. Also, it is not possible to change the log prefix from Sieve (as suggested by LOG=). Regards, Stephan From CMarcus at Media-Brokers.com Tue Nov 15 13:22:53 2011 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 15 Nov 2011 06:22:53 -0500 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> Message-ID: <4EC24B8D.9010401@Media-Brokers.com> On 2011-11-14 10:58 PM, Trutwin, Joshua wrote: > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? Why? You were on the wiki for v1, so obviously it is (or may be) only meant for v1. -- Best regards, Charles From bind at enas.net Tue Nov 15 17:02:22 2011 From: bind at enas.net (Urban Loesch) Date: Tue, 15 Nov 2011 16:02:22 +0100 Subject: [Dovecot] Question about pop3_reuse_xuidl Message-ID: <4EC27EFE.1060504@enas.net> Hi, we are in migration progress from Communigate Pro 5.0.x to Dovecot 2.0.15 with mdbox. We still migrated about 25.000 IMAP accounts from CGP to Dovecot. Also there were still added about 2000 new POP3 Accounts on Dovecot. Now we must migrate about 10.000 POP3 accounts from CGP to Dovecot. At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, or does Dovecot only change the uidl value for new received and still not downloaded mails? Many thanks and regards Urban Loesch From mysqlstudent at gmail.com Tue Nov 15 17:38:44 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 10:38:44 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <4EC217B1.2040809@rename-it.nl> References: <4EC217B1.2040809@rename-it.nl> Message-ID: Hi, >> I have a number of procmail scripts and I've converted them to sieve, >> but they didn't convert very well. Can someone show me how to convert >> the following: >> >> :0 >> { >> ? ? :0 fhW >> ? ? | /home/alex/bin/process-mail.pl >> } >> >> The script skipped right over that. > > The Sieve language doesn't support invoking external programs. You can > execute this program before Sieve, for instance by wrapping the LDA > invocation. Can you describe how I might do this? > Also, there is a non-standard (an still pretty much > experimental) language extension for this, as provided by the pipe plugin: Is there a way to just continue to use procmail? >> I'm also trying to convert the following: >> >> :0 >> * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) >> * !USER_IN_WHITELIST >> * !BAYES_00 >> * !URIBL_ >> * !KHOP_RCVD_UNTRUST >> * !RCVD_IN_DNSWL_MED >> * !SPF_SOFTFAIL >> * SPF_HELO_PASS >> * SPF_PASS >> { >> ? ? ? ? LOG="DNSWL: " >> ? ? ? ? :0: >> ? ? ? ? whitelist/dnswl >> } >> >> The script only produced the following: > > [...] >> >> Any ideas greatly appreciated. > > I'm not sure what these rules are supposed to mean (most conditions are > obscured in variables), but by the looks of things these should also be > executed outside Sieve as well. If all those conditions map to simple header > and envelope tests, you can provide a Sieve alternative. However, I think > USER_IN_WHITELIST queries some external list and that too is not (currently) > possible from Pigeonhole Sieve. Also, it is not possible to change the log > prefix from Sieve (as suggested by LOG=). No, these are all simple header matches. There's no external program being executed here. The LOG= is also a procmail built-in, but I can do without that. Thanks again, Alex From stephan at rename-it.nl Tue Nov 15 18:49:00 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 15 Nov 2011 17:49:00 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> Message-ID: <4EC297FC.8050405@rename-it.nl> Op 15-11-2011 16:38, Alex schreef: > Hi, > >> The Sieve language doesn't support invoking external programs. You can >> execute this program before Sieve, for instance by wrapping the LDA >> invocation. > Can you describe how I might do this? You can write a shell script that is executed from your MTA instead of the Dovecot LDA. Perform the process-mail task from within that script and then call the Dovecot LDA from the shell script (and pipe the message to it). >> Also, there is a non-standard (an still pretty much >> experimental) language extension for this, as provided by the pipe plugin: > Is there a way to just continue to use procmail? Yes, you can pipe the message to the Dovecot LDA from procmail. This would be similar to the solution described above, only using Procmail instead of a shell script. Regards, Stephan. From mysqlstudent at gmail.com Tue Nov 15 20:09:39 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 13:09:39 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <4EC297FC.8050405@rename-it.nl> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> Message-ID: Hi, >>> The Sieve language doesn't support invoking external programs. You can >>> execute this program before Sieve, for instance by wrapping the LDA >>> invocation. >> >> Can you describe how I might do this? > > You can write a shell script that is executed from your MTA instead of the > Dovecot LDA. Perform the process-mail task from within that script and then > call the Dovecot LDA from the shell script (and pipe the message to it). So you mean from master.cf, in the same way as deliver is being executed? dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient} -a "${RECIPIENT}" Replacing deliver here with a shell script? Then execute deliver from within the shell script, or just do the delivery from within a shell script altogether? >> Is there a way to just continue to use procmail? > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > be similar to the solution described above, only using Procmail instead of a > shell script. Are you referring to doing something like this: http://wiki2.dovecot.org/procmail I just don't understand how to invoke procmail from deliver. My main motivation for trying to continue to use procmail is that the sieve language seems far more complicated and involved than even a procmail recipe, and isn't as fully developed. I don't want to make this even more complicated, however. Perhaps you can explain how to filter a basic message based on something in the body or a header and sort it into a file? Thanks so much for your help. Best, Alex From stan at hardwarefreak.com Tue Nov 15 20:26:38 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 15 Nov 2011 12:26:38 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <43B341A1-9245-4863-932A-887E21970319@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> Message-ID: <4EC2AEDE.2080306@hardwarefreak.com> On 11/14/2011 3:16 PM, Alexander Chekalin wrote: > Locking issues on mbox is the reason for my long-lasting love affair with maildir, Same reason most others fell in love with it. Many now want to divorce maildir, as the cost of the storage to maintain acceptable performance is now too high. > and it's lasts long years. Ok, the life's lessons are like this, learn something and move on with it ;) even if it's "new old thing". Thank you for pointing that! Many old UNIX gurus still use mbox, not maildir, and never will. If you ask them why they'll likely say "you don't use a screwdriver to drive a nail do you?" > What I was doubt about is default rotate size of 2M, since I used to see pretty reasonable default settings in all Dovecot config. 32 or 64 are much close to the ones I'd personally prefer. Given the fact that we're talking about an archive server, you'd be better off using a very large mdbox file size, say 1GB. You're never deleting individual messages from this archive correct? No expunges? This is why I recommended mbox in the first place. If your only writes to these mailbox files are appends of new messages, mbox is the best format by far. It's faster at appending than any other format, and it's faster for searching than any other. > I also about to choose now is the OS and FS for the archive. I seriously think about ZFS with compression (in fact it will be stripes over couple of mirrors = software equivalent of RAID 10 on SATA drives, with compression on FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox itself. I see pros and contras for both, so that's the question to answer! It's an archive. You're not going to use maildir so you don't need random IOPS performance. Thus RAID5/6 are a much better fit for an archive as you get better read performance, with more than adequate write performance, and you use less disks. And as this is an archive, you don't need real time automatic/transparent compression. Thus I recommend something like: 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled 2.6.39 or later kernel 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 Specify a strip size of 256KB for the array Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading ahead 1024 sectors at a time instead of the default of 256. This will speed up your searches quite a bit. 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults 4. mbox w/zlib plugin. Compress daily files each night with a script 5. You don't need LVM with a good RAID card (or with mdraid). This controller can expand the RAID5 up to 8 drives (up to 32 drives max using SAS expanders) -- Stan From tss at iki.fi Tue Nov 15 21:02:43 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:02:43 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC2AEDE.2080306@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> Message-ID: <1321383763.21919.484.camel@hurina> On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. Just as long as you're not simultaneously trying to read and write the mbox file (or just write in 2+ sessions). Then there's a lot waiting on locks. (mdbox has no read locks, and its write locks are very short lived.) From micah at riseup.net Tue Nov 15 21:24:18 2011 From: micah at riseup.net (Micah Anderson) Date: Tue, 15 Nov 2011 14:24:18 -0500 Subject: [Dovecot] dsync problems Message-ID: <871ut9i3z1.fsf@algae.riseup.net> When a user renames their username, I am using dsync to copy their mail over to the new username's mail location[0]. Some of the dsyncs are failing with errors that I dont know how to work with, for example: dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=42fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=43fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=44fcd40303c8a64e432300007ef44c7a ERROR: dsync failed, exit code: 256 The errors cause a non-zero exit code from dsync, which causes my rename script to bail out. What are these errors, and how can I fix them? Thanks, micah 0. Why use dsync instead of a simple mv operation? This seems to be necessary for two corner cases: 1. dovecot creates the new mailbox automatically when the user logs in or receives a mail, so if the user changes their mail and logs in or receives an email before the move has been done, then the mailbox is created and then a move command will fail. 2. If there has been new mail created under the new name, we can't just simply remove the stuff that is automatically created there and replace it with the old things because we could potentially be removing mail that has been delivered in the mean time. I'd be really interested if people had suggestions for a better mechanism, or perhaps a way to have dovecot not create the new mail location automatically. -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Tue Nov 15 21:25:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:25:55 +0200 Subject: [Dovecot] Question about pop3_reuse_xuidl In-Reply-To: <4EC27EFE.1060504@enas.net> References: <4EC27EFE.1060504@enas.net> Message-ID: <1321385157.21919.497.camel@hurina> On Tue, 2011-11-15 at 16:02 +0100, Urban Loesch wrote: > At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. > Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? > > Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, Yes. > or does Dovecot only change the uidl value for new received and still not downloaded mails? Nope.. There is pop3_save_uidl=yes setting, which would do this for Maildir, assuming the POP3 clients had logged in while this option was enabled, but this isn't implemented for dbox. The biggest problem with implementing this for dbox would be that it has no variable length permanent per-mail storage (Maildir's dovecot-uidlist equivalent). One way to solve this would be by patching dsync a little bit, and then dsyncing existing mdbox to mdbox-new, which adds existing POP3 UIDLs as dbox metadata. Those will override X-UIDL: headers. The patching would be basically by making dsync/dsync-local-worker.c local_worker_msg_get_next() always set data.pop3_uidl variable to the correct UIDL (which would depend on your pop3_uidl_format setting - pop3/pop3-commands.c:pop3_get_uid() shows how). From tss at iki.fi Tue Nov 15 21:30:37 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:30:37 +0200 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> Message-ID: <1321385437.21919.498.camel@hurina> On Tue, 2011-11-15 at 03:58 +0000, Trutwin, Joshua wrote: > Appears this is the problem - I found this for dovecot v2 and it seems to be > working fine: > > http://sourceforge.jp/projects/dovecot2-drac/ > > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? I changed the wiki2 link to this URL. From mgoldey at goldey.net Tue Nov 15 21:31:50 2011 From: mgoldey at goldey.net (Mark Goldey) Date: Tue, 15 Nov 2011 14:31:50 -0500 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 Message-ID: <4EC2BE26.3030807@goldey.net> Folks: I've been running Dovecot 2.0.7 on a Qmail / Vpopmail installation built on Debian Squeeze, with no problems for about a year. Dovecot is used for IMAP/S auth only. Dovecot is built from source. I just upgraded to 2.0.15 but it segfaults whenever an IMAP auth is attempted, regardless of the user who tries to login, whether by webmail (Roundmail) Android phone (K-9 Mail) or iPad (Roundmail) I downgraded to 2.0.13 but the segfault persisted. Re-installed 2.0.7 and that cleared it right up. Here's the pertinent details from 2.0.15. Anything else I can provide? root at XXX:/# uname -a Linux XXX 2.6.32-5-686 #1 SMP Wed Jan 12 04:01:41 UTC 2011 i686 GNU/Linux root at XXX:/# dovecot -n # 2.0.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.3 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no first_valid_uid = XXXX last_valid_uid = XXXX log_path = /var/log/dovecot.log mail_debug = yes passdb { driver = vpopmail } protocols = imap service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 128 vsz_limit = 64 M } ssl_cert = , method=PLAIN, rip=174.79.xxx.xxx, lip=192.xxx.xxx.xxx, mpid=691, TLS Nov 15 14:08:43 imap-login: Warning: SSL alert: where=0x4008, ret=256: warning close notify [174.79.xxx.xxx] Nov 15 14:08:43 auth-worker: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth I ought to mention that /usr/local/lib/dovecot/auth is empty. Here's the backtrace: root at XXX:/# gdb /usr/local/libexec/dovecot/auth /usr/local/var/run/dovecot/core GNU gdb (GDB) 7.0.1-debian [snip] Reading symbols from /usr/local/libexec/dovecot/auth...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/i686/cmov/libcrypt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/libcrypt-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libcrypt.so.1 Reading symbols from /home/vpopmail/lib/libvpopmail.so...(no debugging symbols found)...done. Loaded symbols for /home/vpopmail/lib/libvpopmail.so Reading symbols from /lib/i686/cmov/libdl.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libdl-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libdl.so.2 Reading symbols from /lib/i686/cmov/librt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/librt-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/librt.so.1 Reading symbols from /lib/i686/cmov/libc.so.6...Reading symbols from /usr/lib/debug/lib/i686/cmov/libc-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libc.so.6 Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/i686/cmov/libpthread.so.0...Reading symbols from /usr/lib/debug/lib/i686/cmov/libpthread-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libpthread.so.0 Reading symbols from /home/vpopmail/etc/vpopmail.authmodule...(no debugging symbols found)...done. Loaded symbols for /home/vpopmail/etc/vpopmail.authmodule Core was generated by `dovecot/auth -w'. Program terminated with signal 11, Segmentation fault. #0 0x00000000 in ?? () (gdb) bt full #0 0x00000000 in ?? () No symbol table info available. #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at auth-worker-client.c:405 auth_request = 0x82b56e8 userdb_id = 1 #2 auth_worker_handle_line (client=0x82b34d0) at auth-worker-client.c:534 id = 2 #3 auth_worker_input (client=0x82b34d0) at auth-worker-client.c:614 _data_stack_cur_id = 3 line = ret = 232 #4 0xb785f021 in io_loop_call_io (io=0x82b3688) at ioloop.c:384 ioloop = 0x82a6398 t_id = 2 #5 0xb78600d3 in io_loop_handler_run (ioloop=0x82a6398) at ioloop-epoll.c:213 ctx = 0x82b1f38 event = 0x82b1f78 list = 0x82b36b0 io = 0x829e2b0 tv = {tv_sec = 2147483, tv_usec = 0} msecs = 1 ret = 1 i = 0 j = 0 call = 192 #6 0xb785efa0 in io_loop_run (ioloop=0x82a6398) at ioloop.c:405 No locals. #7 0xb784abaa in master_service_run (service=0x82a62e8, callback=0x805c470 ) at master-service.c:481 No locals. #8 0x0805c828 in main (argc=2, argv=0x82a61c0) at main.c:298 c = (gdb) --mhg From tss at iki.fi Tue Nov 15 21:35:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:35:03 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> Message-ID: <1321385703.21919.501.camel@hurina> On Tue, 2011-11-15 at 13:09 -0500, Alex wrote: > > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > > be similar to the solution described above, only using Procmail instead of a > > shell script. > > Are you referring to doing something like this: > > http://wiki2.dovecot.org/procmail > > I just don't understand how to invoke procmail from deliver. The idea was to invoke deliver from procmail. I still do this myself since I haven't bothered changing my procmail scripts to Sieve: DELIVER=/usr/local/libexec/dovecot/deliver :0 * ^List-Id.*dovecot.dovecot.org | $DELIVER -m dovecot :0 | $DELIVER etc. From tss at iki.fi Tue Nov 15 21:39:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:39:50 +0200 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: Message-ID: <1321385991.21919.503.camel@hurina> On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote: > > Thanks to a fortuitously unrelated thread ("how to disable quota for > > second namespace"), I got the quota part figured out and that seems to > > be working: Add a second entry to plugin {}, e.g. "quota2 = > > maildir:Archive quota:ns=INBOX.Archives." and add rules for > > userdb_quota2_rule, userdb_quota2_rule2, etc. > > > > My real question now is: Are there any fatal gotchas in this that I'm > > just not thinking of? > > > > Haven't had a chance to try this large-scale yet. Anybody have any > thoughts on it? The gotcha is that you have two completely independent quotas with independent usage/limits for the INBOX and Archive namespaces. If that is what you want, it should all be fine. From tss at iki.fi Tue Nov 15 21:44:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:44:20 +0200 Subject: [Dovecot] Chroot In-Reply-To: References: Message-ID: <1321386260.21919.505.camel@hurina> On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote: > all maildirs are located in: /var/mail/jdoe at example.com/Maildir So this is /var/mail/%u/Maildir template. > namespace { > type = shared > separator = / > prefix = shared/%%u/ > location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u You can replace this with: location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u Although if this is used outside chroot (e.g. lmtp, doveadm) it won't work correctly.. > Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No > such file or directory This is done because you used %%h, to look up home directory from userdb. If you don't use %%h, it's not done. From tss at iki.fi Tue Nov 15 21:49:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:49:20 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: Message-ID: <1321386560.21919.509.camel@hurina> On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > I configured dovecot to use submission smtp host becouse of chroot. > > submission_host = 127.0.0.1 > > Unfortunatelly: > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > synchronization error (next input sent too soon: pipelining was not > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > input="MAIL FROM:\\r\\n" This basically says that Dovecot's SMTP client is being noncompliant, because Exim doesn't advertise PIPELINING extension. But I'd rather not add more code to fix this, since PIPELINING gives you a little bit better performance anyway and you can most likely fix this by modifying Exim's configs in some way (I can't believe Exim wouldn't support PIPENING..). From tss at iki.fi Tue Nov 15 22:16:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:16:14 +0200 Subject: [Dovecot] Migration From Courier Imap. In-Reply-To: <653a51f07131e61c52bf983359f4710c@192.168.0.251> References: <653a51f07131e61c52bf983359f4710c@192.168.0.251> Message-ID: <1321388180.21919.515.camel@hurina> On Sat, 2011-11-12 at 12:58 +0530, Rajeev Sharma wrote: > I am using vpopmail,qmail along with courier-imap 4.0 i want to > migrate from courier to dovecot but i have 100 shared folder > under /home/vpopmail/domains/domain.com/Malidir and user shared them > on the basis of rights assign. > > I am very much clear how to migrate from courier to dovecot but i am > unable how to migrate shared folder and assign rights. I don't think there is any existing documentation about this. I've also not used Courier shared folders myself much. I think the ACL files' contents are compatible, although Dovecot names them differently. IIRC Courier used some kind of symlinked maildir files for shared folders to provide per-user flags? Migrating those to Dovecot would be a bit tricky, especially because Dovecot currently supports only per-user \Seen flags (maybe Courier does too? I'm not sure) and Dovecot stores the per-user \Seen flags only in per-user index files. Anyway, I don't really have time to look into details of this, so you'll just need to figure out the specifics yourself (or alternatively pay me to do that via my company). From tss at iki.fi Tue Nov 15 22:24:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:24:57 +0200 Subject: [Dovecot] Invalid "doveadm search" queries not rejected In-Reply-To: <1320997281.2261.116.camel@iota.localdomain> References: <1320997281.2261.116.camel@iota.localdomain> Message-ID: <1321388697.21919.518.camel@hurina> On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > 6). > > According to the doveadm-search-query(7) manual page, a "mailbox" > specification can only come at the beginning of a search query. It doesn't exactly say that, but yeah, it doesn't also include MAILBOX and MAILBOX-GUID as search keys. Added now to future versions. > However, > if I put a mailbox specification in the middle of a query, not only do I > not get an error message, I get surprising results: > > $ doveadm search mailbox A |wc -l > 6979 > $ doveadm search mailbox B |wc -l > 206 > $ doveadm search mailbox A OR mailbox B |wc -l > 42492 I'm pretty sure this was fixed in v2.0.14. > If it became possible to do boolean logic on mailbox queries, that'd be > cool. It is. From tss at iki.fi Tue Nov 15 22:27:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:27:20 +0200 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <874nybztuz.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> Message-ID: <1321388840.21919.520.camel@hurina> On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > I really like the feature where you can define quota rules with percents > which trigger off of the default values[0] (so you can set the Trash to > allow for 10% more of the user's quota for example). > > What I would really love in dovecot would be for the ability to > configure a quota rule for dovecot-lda. I would like to configure things > so we don't bounce emails for users until they are well over quota, the > IMAP quota plugin is a really great way to notify people that they are > over quota because it fails to write to other folders.... that should be > enough to get people's attention that they need to deal with things, but > bouncing is harsh. You can give LDA-specific quota settings. Like maybe: protocol lda { plugin { quota_rule2 = INBOX:storage=200% } } or even not enforcing quota at all: protocol lda { plugin { quota = maildir:User quota:noenforcing } } From tss at iki.fi Tue Nov 15 22:28:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:28:57 +0200 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <4EBBB22A.90501@bsc.es> References: <4EBA4DD4.4080505@bsc.es> <1320861378.21919.376.camel@hurina> <4EBBB22A.90501@bsc.es> Message-ID: <1321388937.21919.521.camel@hurina> On Thu, 2011-11-10 at 12:14 +0100, David Ocana wrote: > >> namespace { > >> separator = / > >> prefix = INBOX/ > >> location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n > >> inbox = yes > >> hidden = yes > >> } > >> > >> plugin { > >> quota = dirsize:User quota > > > > quota = dirsize:User quota:ns=INBOX/ > > Actually I forgot to mention that I also tried that, but I got the > following error: > > Error: Initialization failed: Failed to initialize quota: Quota root > User quota: Unknown parameter for backend dirsize: ns=INBOX/ Oh. Yeah, dirsize backend doesn't have all the features that other backends have, because it's so little used. You could switch to dict file quota.. From tss at iki.fi Tue Nov 15 22:30:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:30:16 +0200 Subject: [Dovecot] Limit the number of pop/imap requests In-Reply-To: References: Message-ID: <1321389016.21919.522.camel@hurina> On Thu, 2011-11-10 at 10:40 +0100, wwl wrote: > Hello Group, > > is it possible to Limit the number of pop/imap requests to Dovecot? > > I read the Wiki and found > > max_mail_processes = 512 > > is this the way to limit this ? This limits the number of simultaneous IMAP and POP3 connections. Is that what you mean with "requests"? If so, yeah, you can reduce this value. I think the counts are tracked separately for IMAP and POP3 (so you can have 512 IMAP and 512 POP3 connections). From tss at iki.fi Tue Nov 15 22:32:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:32:08 +0200 Subject: [Dovecot] indexes permissions problem In-Reply-To: <4EBB8BA7.9070603@cruzio.com> References: <4EBB8BA7.9070603@cruzio.com> Message-ID: <1321389128.21919.523.camel@hurina> On Thu, 2011-11-10 at 00:30 -0800, Mark Hanford wrote: > I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). > For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box > with user home directories NFS mounted), my index files are setup to be > in /u/indexes/ > > On the Dovecot 1.1.1 installation, the perms on the indexes directory is > 777 with root:mail ownership. > > The same thing on the Dovecot 2 / Centos server results in a 'permission > denied' error when Dovecot tries to create files. What exactly is the error message? Copy&paste it entirely. > So, I guess my main question is, what perms and ownership should > /u/indexes be set to? I've tried several different things before this > cry for help... Those permissions look ok to me. It's possible that this was a bug in 2.0.beta6 and upgrading might help (but might not). From tss at iki.fi Tue Nov 15 22:34:21 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:34:21 +0200 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: References: Message-ID: <1321389261.21919.524.camel@hurina> On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > I am familiar with this article > http://wiki.dovecot.org/LDA#multipleuids > > I want to disable setuid-root, but I don't know the default group and > permissions. > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > but my delivery is still failing > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, > orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) dovecot-lda's group should be the same as you specify to the pipe in Postfix's master.cf. From tss at iki.fi Tue Nov 15 22:35:36 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:35:36 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <20111105194350.GA3782@state-of-mind.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <20111105194350.GA3782@state-of-mind.de> Message-ID: <1321389336.21919.525.camel@hurina> On Sat, 2011-11-05 at 20:43 +0100, Patrick Ben Koetter wrote: > * Timo Sirainen : > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > > > is there a way to use Dovecot SASL Authentication for submission on a > > > different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > > > > Dovecot machine <-> postfix machine(for submission) > > > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. > > Document it and Wietse will support it on the mailing list. If its not > documented support will be declined. You mean document in Dovecot wiki or for Postfix? > Should I? Please do :) From tss at iki.fi Tue Nov 15 22:39:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:39:13 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC2BE26.3030807@goldey.net> References: <4EC2BE26.3030807@goldey.net> Message-ID: <1321389553.21919.527.camel@hurina> On Tue, 2011-11-15 at 14:31 -0500, Mark Goldey wrote: Could you do a bit more with gdb and this core: > #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at > auth-worker-client.c:405 > auth_request = 0x82b56e8 > userdb_id = 1 fr 1 p *auth_request->userdb->userdb p *auth_request->userdb->userdb->iface From tss at iki.fi Tue Nov 15 22:43:24 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:43:24 +0200 Subject: [Dovecot] dsync problems In-Reply-To: <871ut9i3z1.fsf@algae.riseup.net> References: <871ut9i3z1.fsf@algae.riseup.net> Message-ID: <1321389804.21919.530.camel@hurina> On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > When a user renames their username, I am using dsync to copy their mail > over to the new username's mail location[0]. > > Some of the dsyncs are failing with errors that I dont know how to work > with, for example: > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a These shouldn't really happen. Something's going internally wrong with dsync. Can you reproduce this reliably somehow? > The errors cause a non-zero exit code from dsync, which causes my rename > script to bail out. What are these errors, and how can I fix them? Does a second dsync on error succeed? :) > 0. Why use dsync instead of a simple mv operation? This seems to be > necessary for two corner cases: > > 1. dovecot creates the new mailbox automatically when the user logs in > or receives a mail, so if the user changes their mail and logs in or > receives an email before the move has been done, then the mailbox is > created and then a move command will fail. > > 2. If there has been new mail created under the new name, we can't just > simply remove the stuff that is automatically created there and replace > it with the old things because we could potentially be removing mail > that has been delivered in the mean time. You could temporarily change the permissions for the home directory so that no new mailboxes/mails could be created during the move (e.g. 0700 root). From mrvjtod at gmail.com Tue Nov 15 23:13:13 2011 From: mrvjtod at gmail.com (Chris Young) Date: Tue, 15 Nov 2011 16:13:13 -0500 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: <1321389261.21919.524.camel@hurina> References: <1321389261.21919.524.camel@hurina> Message-ID: Thanks for the response. I was able to get it working with chmod 755 On Tue, Nov 15, 2011 at 3:34 PM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > > I am familiar with this article > > http://wiki.dovecot.org/LDA#multipleuids > > > > I want to disable setuid-root, but I don't know the default group and > > permissions. > > > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > > > but my delivery is still failing > > > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=< > user at domain.com>, > > orig_to=, relay=dovecot, delay=0.05, > delays=0.04/0/0/0.01, > > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: > fatal: > > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied > ) > > dovecot-lda's group should be the same as you specify to the pipe in > Postfix's master.cf. > > > From mysqlstudent at gmail.com Wed Nov 16 00:14:50 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 17:14:50 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321385703.21919.501.camel@hurina> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> I just don't understand how to invoke procmail from deliver. > > The idea was to invoke deliver from procmail. I still do this myself > since I haven't bothered changing my procmail scripts to Sieve: > > DELIVER=/usr/local/libexec/dovecot/deliver > > :0 > * ^List-Id.*dovecot.dovecot.org > | $DELIVER -m dovecot > > :0 > | $DELIVER How do I tell dovecot to use procmail so that it may invoke deliver? Thanks again, Alex From tss at iki.fi Wed Nov 16 00:27:12 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 00:27:12 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 0.14, Alex wrote: >>> I just don't understand how to invoke procmail from deliver. >> >> The idea was to invoke deliver from procmail. I still do this myself >> since I haven't bothered changing my procmail scripts to Sieve: >> >> DELIVER=/usr/local/libexec/dovecot/deliver >> >> :0 >> * ^List-Id.*dovecot.dovecot.org >> | $DELIVER -m dovecot >> >> :0 >> | $DELIVER > > How do I tell dovecot to use procmail so that it may invoke deliver? You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc. From nick.z.edwards at gmail.com Wed Nov 16 00:48:19 2011 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 16 Nov 2011 08:48:19 +1000 Subject: [Dovecot] Excessive imap-logins, never die Message-ID: Hiya, Today I note there are 38 imap-logins that seem to be stale on our staff imap server, they do not seem to die off, new imap-logins are created with new logins and exit accordingly. Now, these are from our monitoring software that runs on another server on our LAN. Our corporate link was taken down for 2 hours for urgent relocation of fibre (demolishing old adjoining building, and just in case, they needed to move it which we were glad about), so, this begs the question, why would imap-logins be there I guess in some sort of wait state, even with a still working internal DNS (internal view as well) the only difference is no actual internet connection in this time. A week ago when they first started to working and we lost internet for 45 minutes I also saw a few extra stale imap-logins, but after todays events, I decided this is a bigger issue that needs to be looked into, as dovecot required a restart to clear it all (I wasnt going to sit there killing individual PIDs all evening). Dovecot version 1.2.17, built from source From tss at iki.fi Wed Nov 16 00:50:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 00:50:25 +0200 Subject: [Dovecot] Excessive imap-logins, never die In-Reply-To: References: Message-ID: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> On 16.11.2011, at 0.48, Nick Edwards wrote: > Today I note there are 38 imap-logins that seem to be stale on our staff > imap server, > they do not seem to die off, new imap-logins are created with new logins > and exit accordingly. .. > Dovecot version 1.2.17, built from source All this code has been rewritten in v2.0 and there's not much point in spending time on figuring out why things aren't working perfectly in v1.x. But you could most likely avoid this problem by switching to "high-performance mode": http://wiki.dovecot.org/LoginProcess From nick.z.edwards at gmail.com Wed Nov 16 01:28:07 2011 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 16 Nov 2011 09:28:07 +1000 Subject: [Dovecot] Excessive imap-logins, never die In-Reply-To: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> References: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> Message-ID: On Wed, Nov 16, 2011 at 8:50 AM, Timo Sirainen wrote: > On 16.11.2011, at 0.48, Nick Edwards wrote: > > > Today I note there are 38 imap-logins that seem to be stale on our staff > > imap server, > > they do not seem to die off, new imap-logins are created with new logins > > and exit accordingly. > .. > > Dovecot version 1.2.17, built from source > > All this code has been rewritten in v2.0 and there's not much point in > spending time on figuring out why things aren't working perfectly in v1.x. > But you could most likely avoid this problem by switching to > "high-performance mode": http://wiki.dovecot.org/LoginProcess > > Okay, we do use high security mode specifically, and we require SSL on that server, I might just leave it as is, if there is no timeout value for these which I couldn't see when googling earlier today. I want to look at v2, but need to wait until all staff on are Christmas holidays, so if upgrade goes pair shaped, I can restore and not get bombarded with phone calls from management who can't get their mail. From mysqlstudent at gmail.com Wed Nov 16 02:00:13 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 19:00:13 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >>>> I just don't understand how to invoke procmail from deliver. >>> >>> The idea was to invoke deliver from procmail. I still do this myself >>> since I haven't bothered changing my procmail scripts to Sieve: >>> >>> DELIVER=/usr/local/libexec/dovecot/deliver >>> >>> :0 >>> * ^List-Id.*dovecot.dovecot.org >>> | $DELIVER -m dovecot >>> >>> :0 >>> | $DELIVER >> >> How do I tell dovecot to use procmail so that it may invoke deliver? > > You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc. Okay, I'm making progress. I disabled the virtual_domain stuff in main.cf for postfix and just enabled mail_command=procmail. I think there's something wrong with the deliver command. It's now delivering, but my deliver command must be wrong. This is my ~/.procmailrc: # Log an abstract of all messages LOGABSTRACT=all PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/lib:/usr/local/bin LOGFILE=$HOME/.maillog MAILDIR=$HOME/mail PROCHOME=$HOME/.procmail SENDMAIL=/usr/sbin/sendmail VERBOSE=yes DELIVER=/usr/libexec/dovecot/deliver :0 fhW | $DELIVER -m xspamtest :0 fhw | $DELIVER When a mail is received, it prints this to .maillog: procmail: [23382] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: [23383] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Assigning "LASTFOLDER=/var/spool/mail/alexuser" procmail: Opening "/var/spool/mail/alexuser" procmail: Acquiring kernel-lock procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Unlocking "/var/spool/mail/alexuser.lock" >From prvs=293a46ea3=mm at mydomain.com Tue Nov 15 18:56:04 2011 Subject: Grand Manor, Corvallis, OR, USA, AUTOMATED Extrane Folder: /var/spool/mail/bcc-user 17025 It delivers to the mail spool, but executing deliver as its printed in the procmailrc: # # /usr/libexec/dovecot/deliver -m spamtest lda(root): Fatal: destination user parameter (-d user) not given Am I doing something wrong? Thanks, Alex From tss at iki.fi Wed Nov 16 02:03:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 02:03:16 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 2.00, Alex wrote: > It's now delivering, but my deliver command must be wrong. This is my > ~/.procmailrc: .. > # # /usr/libexec/dovecot/deliver -m spamtest > lda(root): Fatal: destination user parameter (-d user) not given > > Am I doing something wrong? Is it trying to deliver mails to root user? That's not supported. From moseleymark at gmail.com Wed Nov 16 02:04:46 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 15 Nov 2011 16:04:46 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: <1321385991.21919.503.camel@hurina> References: <1321385991.21919.503.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 11:39 AM, Timo Sirainen wrote: > On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote: > >> > Thanks to a fortuitously unrelated thread ("how to disable quota for >> > second namespace"), I got the quota part figured out and that seems to >> > be working: Add a second entry to plugin {}, e.g. "quota2 = >> > maildir:Archive quota:ns=INBOX.Archives." and add rules for >> > userdb_quota2_rule, userdb_quota2_rule2, etc. >> > >> > My real question now is: Are there any fatal gotchas in this that I'm >> > just not thinking of? >> > >> >> Haven't had a chance to try this large-scale yet. Anybody have any >> thoughts on it? > > The gotcha is that you have two completely independent quotas with > independent usage/limits for the INBOX and Archive namespaces. If that > is what you want, it should all be fine. Nope, that's totally fine. The idea is to put Archive on cheaper (slower) storage and then grant more generous quotas there to make it worth their while to use, without slowing down their Inbox. Another application would be to put their Spam in another namespace (for people who choose to have it put in a separate folder) with a lower quota, again to offload it onto cheaper storage, since hardly anyone actually looks at it. Or is this something that I could be doing more transparently in 2.1 with imapc? From mysqlstudent at gmail.com Wed Nov 16 02:08:34 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 19:08:34 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> It's now delivering, but my deliver command must be wrong. This is my >> ~/.procmailrc: > .. >> # # /usr/libexec/dovecot/deliver -m spamtest >> lda(root): Fatal: destination user parameter (-d user) not given >> >> Am I doing something wrong? > > Is it trying to deliver mails to root user? That's not supported. No, I was just running as root from the command-line for an example. The maillog shows error 75: procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded This was for a test user named "alexuser" I created. Thanks again, Alex From tss at iki.fi Wed Nov 16 02:16:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 02:16:33 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 2.08, Alex wrote: > No, I was just running as root from the command-line for an example. > The maillog shows error 75: > > procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" > procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" > procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > procmail: Executing "/usr/libexec/dovecot/deliver" > procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > > This was for a test user named "alexuser" I created. deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging From micah at riseup.net Wed Nov 16 03:04:42 2011 From: micah at riseup.net (Micah Anderson) Date: Tue, 15 Nov 2011 20:04:42 -0500 Subject: [Dovecot] Deleting a mailbox and all its children Message-ID: <87pqgsho7p.fsf@algae.riseup.net> I'm looking at scripting a mechanism to delete a mailbox, which is easy with 'doveadm mailbox delete' -- however it gets complicated when there are children of arbitrary depth, for example, I may have this: restored.daily1.INBOX restored.daily1.Sent restored.daily1.Mystuff restored.daily1.Mystuff.foo I would like to delete the 'restored' mailbox and all its children (and unsubscribe the folders), but the 'doveadm mailbox delete' command wont delete any children. I'm surprised that there isn't a -r (recursive) option to 'doveadm mailbox delete', but perhaps I am missing the easier way to handle this? thanks for any ideas or suggestions! micah -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Wed Nov 16 03:15:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 03:15:02 +0200 Subject: [Dovecot] Deleting a mailbox and all its children In-Reply-To: <87pqgsho7p.fsf@algae.riseup.net> References: <87pqgsho7p.fsf@algae.riseup.net> Message-ID: <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> On 16.11.2011, at 3.04, Micah Anderson wrote: > I'm looking at scripting a mechanism to delete a mailbox, which is easy > with 'doveadm mailbox delete' -- however it gets complicated when there > are children of arbitrary depth, for example, I may have this: > > restored.daily1.INBOX > restored.daily1.Sent > restored.daily1.Mystuff > restored.daily1.Mystuff.foo > > I would like to delete the 'restored' mailbox and all its children (and > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > delete any children. > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > mailbox delete', but perhaps I am missing the easier way to handle this? Well, how about something like: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm mailbox delete restored That's what it would have to do internally anyway.. From mysqlstudent at gmail.com Wed Nov 16 03:50:56 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 20:50:56 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" >> procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" >> procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> procmail: Executing "/usr/libexec/dovecot/deliver" >> procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> >> This was for a test user named "alexuser" I created. > > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging Yes, looks like permissions. I have the following now in syslog: Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): Operation not permitted Where is it trying to write that it can't? Does alexuser need to be a member of group mail? Thanks, Alex From micah at riseup.net Wed Nov 16 04:23:47 2011 From: micah at riseup.net (micah anderson) Date: Tue, 15 Nov 2011 21:23:47 -0500 Subject: [Dovecot] Deleting a mailbox and all its children In-Reply-To: <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> References: <87pqgsho7p.fsf@algae.riseup.net> <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> Message-ID: <87lirghkjw.fsf@algae.riseup.net> On Wed, 16 Nov 2011 03:15:02 +0200, Timo Sirainen wrote: > On 16.11.2011, at 3.04, Micah Anderson wrote: > > > I'm looking at scripting a mechanism to delete a mailbox, which is easy > > with 'doveadm mailbox delete' -- however it gets complicated when there > > are children of arbitrary depth, for example, I may have this: > > > > restored.daily1.INBOX > > restored.daily1.Sent > > restored.daily1.Mystuff > > restored.daily1.Mystuff.foo > > > > I would like to delete the 'restored' mailbox and all its children (and > > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > > delete any children. > > > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > > mailbox delete', but perhaps I am missing the easier way to handle this? > > Well, how about something like: > > doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete > doveadm mailbox delete restored That wont work because 'restored.daily1.Mystuff.foo' wont get removed before 'restored.daily1.Mystuff' is attempted, resulting in the following: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm(micah): Error: Can't delete mailbox restored.daily1.Mystuff: Mailbox has children, delete them first The problem is the subfolders need to be removed first and then walk up the tree. perhaps I can do: doveadm mailbox list 'restored.*' | sort -r | xargs -n1 doveadm mailbox delete doveadm mailbox delete restored although I'm not sure if the reverse sort is going to work in all cases. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From achekalin at lazurit.com Wed Nov 16 08:16:04 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 16 Nov 2011 09:16:04 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC2AEDE.2080306@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> Message-ID: <4EC35524.30803@lazurit.com> Hello, Stan, > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. I now seriously consider to use mdbox due to its nice self-regulation. After all it I believe mdbox should do file compression on its own, no cron scripts required. > It's an archive. You're not going to use maildir so you don't need > random IOPS performance. Thus RAID5/6 are a much better fit for an > archive as you get better read performance, with more than adequate > write performance, and you use less disks. And as this is an archive, > you don't need real time automatic/transparent compression. Thus I > recommend something like: > > 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled > 2.6.39 or later kernel > 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors > e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 > Specify a strip size of 256KB for the array > Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading > ahead 1024 sectors at a time instead of the default of 256. This > will speed up your searches quite a bit. > 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults > 4. mbox w/zlib plugin. Compress daily files each night with a script > 5. You don't need LVM with a good RAID card (or with mdraid). This > controller can expand the RAID5 up to 8 drives (up to 32 drives max > using SAS expanders) We are considering to get HP DL180G6 server for 8 or 14 drives bays (base model price is somewhat equal, but additional drives adds up cost) with HP Smart Array P410 RAID controller (some servers are equipped with this controller by default) with 256 Mb battery-backed cache, but I'll check your suggestions! What memory size should I plan in the server? You're talking about AMD64 OS image, and 64-bit software are like to consume more memory that 32-bit, so looks like your talking about pretty huge RAM, and I don't believe it's necessary, or maybe I'm wrong? Problem is I have no experience with XFS and not sure I can tune it in the best way, so I'll go with mkfs.xfs defaults, I think. Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins and 2.1 handle that much better, but I don't like the idea of use unstable in production. Thank you for taking your time on my case, yours, Alexander From peter at newton.cx Wed Nov 16 11:00:50 2011 From: peter at newton.cx (Peter Williams) Date: Wed, 16 Nov 2011 01:00:50 -0800 Subject: [Dovecot] Invalid "doveadm search" queries not rejected In-Reply-To: <1321388697.21919.518.camel@hurina> References: <1320997281.2261.116.camel@iota.localdomain> <1321388697.21919.518.camel@hurina> Message-ID: <1321434050.2493.5.camel@iota.localdomain> On Tue, 2011-11-15 at 22:24 +0200, Timo Sirainen wrote: > On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > > > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > > 6). > > > > According to the doveadm-search-query(7) manual page, a "mailbox" > > specification can only come at the beginning of a search query. > > It doesn't exactly say that, but yeah, it doesn't also include MAILBOX > and MAILBOX-GUID as search keys. Added now to future versions. Thanks for explaining the situation. As a small improvement I would argue that the manpage was, and still is, a little bit misleading -- here's the bit of text that I was focusing on: Use either the keyword _mailbox_ and the name of the mailbox or the keyword _mailbox-guid_ and the mailbox's globally unique identifier ***at the beginning of the search_query.*** Peter -- Peter Williams / peter at newton.cx From odhiambo at gmail.com Wed Nov 16 11:26:54 2011 From: odhiambo at gmail.com (Odhiambo Washington) Date: Wed, 16 Nov 2011 12:26:54 +0300 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320856634.21919.371.camel@hurina> References: <1320791738.21919.350.camel@hurina> <1320856634.21919.371.camel@hurina> Message-ID: On Wed, Nov 9, 2011 at 19:37, Timo Sirainen wrote: > On Wed, 2011-11-09 at 18:34 +0300, Odhiambo Washington wrote: > > I have today tried compiling this -beta, but I saw so many warning about > > linking libraries, which I ignored, > > Probably plugin warnings which you can safely ignore. > > > Making all in wiki > > make: don't know how to make 2b. Stop > > Looks like there's some weird file went into wiki docs, which some makes > don't like. You could just find the line containing "2b" from > doc/wiki/Makefile and delete it. > > That did it. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 16:00:38 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 15:00:38 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321386560.21919.509.camel@hurina> References: <1321386560.21919.509.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 8:49 PM, Timo Sirainen wrote: > On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > >> I configured dovecot to use submission smtp host becouse of chroot. >> >> submission_host = 127.0.0.1 >> >> Unfortunatelly: >> >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol >> synchronization error (next input sent too soon: pipelining was not >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next >> input="MAIL FROM:\\r\\n" > > This basically says that Dovecot's SMTP client is being noncompliant, > because Exim doesn't advertise PIPELINING extension. But I'd rather not > add more code to fix this, since PIPELINING gives you a little bit > better performance anyway and you can most likely fix this by modifying > Exim's configs in some way (I can't believe Exim wouldn't support > PIPENING..). > > Thanks for your reply. Indeed exim supports PIPELINING by default: # telnet mail.apisoft.pl 25 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. 220 zeus.apisoft.pl ESMTP Exim 4.76 Wed, 16 Nov 2011 14:55:25 +0100 EHLO test 250-zeus.apisoft.pl Hello localhost [127.0.0.1] 250-SIZE 52428800 250-PIPELINING 250-AUTH LOGIN PLAIN CRAM-MD5 DIGEST-MD5 250-STARTTLS 250 HELP and PIPELINING seems to work fine. But maybe there is a problem with smtp dialog between dovecot and exim. From jester at arcor.de Wed Nov 16 16:02:51 2011 From: jester at arcor.de (Oliver Eales) Date: Wed, 16 Nov 2011 15:02:51 +0100 Subject: [Dovecot] Usage of director without LMTP Message-ID: <4EC3C28B.7040700@arcor.de> Hello, we want to migrate our dovecot 1.2 installation to verison 2.0 and want to use the director feature of version 2. Currently we have a two tier archtiecture, with dovecot proxys and a mailstore togehter with indexes on NFS. (With the common problems of this setup...) For the final setup we want to utilize the director for all protocols (IMAP,POP,LMTP,SIEVE). During the transition phase we will not be able to use LMTP for mail delivery, but we are still stuck with deliver as LDA. So for mail delivery we will not have the fixed user/server assignment, but rather a random assignment. What NFS- and locking settings should be used on the backend servers in this case. And should i use different settings for the host the deliver is executed than the IMAP/POP3 hosts which are accessed through a director proxy. Thanks in advance, Oliver Eales From mh+dovecot at zugschlus.de Wed Nov 16 16:11:25 2011 From: mh+dovecot at zugschlus.de (Marc Haber) Date: Wed, 16 Nov 2011 15:11:25 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321386560.21919.509.camel@hurina> References: <1321386560.21919.509.camel@hurina> Message-ID: <20111116141125.GC4966@torres.zugschlus.de> On Tue, Nov 15, 2011 at 09:49:20PM +0200, Timo Sirainen wrote: > On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > > I configured dovecot to use submission smtp host becouse of chroot. > > > > submission_host = 127.0.0.1 > > > > Unfortunatelly: > > > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > > synchronization error (next input sent too soon: pipelining was not > > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > > input="MAIL FROM:\\r\\n" > > This basically says that Dovecot's SMTP client is being noncompliant, > because Exim doesn't advertise PIPELINING extension. But I'd rather not > add more code to fix this, since PIPELINING gives you a little bit > better performance anyway and you can most likely fix this by modifying > Exim's configs in some way (I can't believe Exim wouldn't support > PIPENING..). I have always interpreted the standard in the way that a client MUST NOT assume that the server supports pipelining before it has advertised PIPELINING. Since PIPELINING is only advertised after the client has identified itself as being ESMTP compliant by saying EHLO instead of HELO, I believe that the client MUST wait with his EHLO until the server has shown its banner. Forcing synchronization is a very effective means of spam protection since most spam bots just blast away with EHLO, MAIL FROM without bothering to wait for the server's banner. Greetings Marc -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 31958061 Nordisch by Nature | How to make an American Quilt | Fax: *49 621 31958062 From llonchj at gmail.com Wed Nov 16 16:13:44 2011 From: llonchj at gmail.com (Jordi Llonch) Date: Thu, 17 Nov 2011 01:13:44 +1100 Subject: [Dovecot] Creating a password_scheme Message-ID: Hello, I am trying to develop a password_scheme for the Django passwords. After building and installing the module in $module_dir/auth/password/djangopass.so, also tried libdjangopass.so and libdjangopass_plugin.so, Dovecot does not recognize the scheme: doveadm pw -l CRYPT MD5 MD5-CRYPT SHA SHA1 SHA256 SHA512 SMD5 SSHA SSHA256 SSHA512 PLAIN CLEARTEXT CRAM-MD5 HMAC-MD5 DIGEST-MD5 PLAIN-MD4 PLAIN-MD5 LDAP-MD5 LANMAN NTLM OTP SKEY RPA SHA256-CRYPT SHA512-CRYPT Please can you help me? Thanks -- Jordi Llonch # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.40.3-0.fc15.x86_64 x86_64 Fedora release 15 (Lovelock) ext4 ----------password-scheme-django.c-------------------------- #include #include #include #include #include #include #include #define uchar unsigned char static bool djangopass_verify(const char *plaintext, const char *user, const unsigned char *raw_password, size_t size) { return TRUE; } void djangopass_generate(const char *plaintext, const char *user, const unsigned char **raw_password_r, size_t *size_r) { } struct password_scheme djangopass_scheme = { "DJANGO", PW_ENCODING_BASE64, 0, djangopass_verify, djangopass_generate }; struct module; void djangopass_init(struct module *module) { password_scheme_register(&djangopass_scheme); } void djangopass_deinit(void) { password_scheme_unregister(&djangopass_scheme); } --------------------------------------------------------------------------- From david at blue-labs.org Wed Nov 16 16:14:54 2011 From: david at blue-labs.org (David Ford) Date: Wed, 16 Nov 2011 09:14:54 -0500 Subject: [Dovecot] submission_host problem In-Reply-To: <20111116141125.GC4966@torres.zugschlus.de> References: <1321386560.21919.509.camel@hurina> <20111116141125.GC4966@torres.zugschlus.de> Message-ID: <4EC3C55E.8040401@blue-labs.org> this and several other features are tools i use with tremendous success at battling spam. every MTA connection that violates protocol by making an assumption or posts invalid data for the SMTP phase, gets kicked off with a 421. -david On 11/16/2011 09:11 AM, Marc Haber wrote: > I have always interpreted the standard in the way that a client MUST > NOT assume that the server supports pipelining before it has > advertised PIPELINING. Since PIPELINING is only advertised after the > client has identified itself as being ESMTP compliant by saying EHLO > instead of HELO, I believe that the client MUST wait with his EHLO > until the server has shown its banner. Forcing synchronization is a > very effective means of spam protection since most spam bots just > blast away with EHLO, MAIL FROM without bothering to wait for the > server's banner. Greetings Marc From tss at iki.fi Wed Nov 16 16:37:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 16:37:46 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> Message-ID: <1321454266.21919.545.camel@hurina> On Wed, 2011-11-16 at 15:00 +0100, Bambero wrote: > >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > >> synchronization error (next input sent too soon: pipelining was not > >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > >> input="MAIL FROM:\\r\\n" > > > > This basically says that Dovecot's SMTP client is being noncompliant, > > because Exim doesn't advertise PIPELINING extension. But I'd rather not > > add more code to fix this, since PIPELINING gives you a little bit > > better performance anyway and you can most likely fix this by modifying > > Exim's configs in some way (I can't believe Exim wouldn't support > > PIPENING..). > > > > > Thanks for your reply. > Indeed exim supports PIPELINING by default: Oh. Well, see if the attached patch helps? If it does, I'll add it to next version. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 1493 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 16:54:43 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 15:54:43 +0100 Subject: [Dovecot] Chroot In-Reply-To: <1321386260.21919.505.camel@hurina> References: <1321386260.21919.505.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 8:44 PM, Timo Sirainen wrote: > On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote: > >> all maildirs are located in: /var/mail/jdoe at example.com/Maildir > > So this is /var/mail/%u/Maildir template. > >> namespace { >> ? type = shared >> ? separator = / >> ? prefix = shared/%%u/ >> ? location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u > > You can replace this with: > > location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u > > Although if this is used outside chroot (e.g. lmtp, doveadm) it won't > work correctly.. > >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >> such file or directory > > This is done because you used %%h, to look up home directory from > userdb. If you don't use %%h, it's not done. > Thanks but my db backend returns home dir in chroot like /test at example.com while the realpath with no chroot is: /var/mail/test at example.com So both locations in this case are the same location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u From tss at iki.fi Wed Nov 16 16:59:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 16:59:06 +0200 Subject: [Dovecot] Chroot In-Reply-To: References: <1321386260.21919.505.camel@hurina> Message-ID: <1321455546.21919.553.camel@hurina> On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: > >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No > >> such file or directory > > > > This is done because you used %%h, to look up home directory from > > userdb. If you don't use %%h, it's not done. > > > > Thanks but my db backend returns home dir in chroot like > /test at example.com > > while the realpath with no chroot is: > /var/mail/test at example.com > > So both locations in this case are the same > location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u > location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u Yes, they are the same, but the difference is that with the second one you shouldn't fail with the userdb lookup error. Right? From tss at iki.fi Wed Nov 16 17:02:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 17:02:50 +0200 Subject: [Dovecot] Usage of director without LMTP In-Reply-To: <4EC3C28B.7040700@arcor.de> References: <4EC3C28B.7040700@arcor.de> Message-ID: <1321455770.21919.555.camel@hurina> On Wed, 2011-11-16 at 15:02 +0100, Oliver Eales wrote: > For the final setup we want to utilize the director for all protocols > (IMAP,POP,LMTP,SIEVE). > During the transition phase we will not be able to use LMTP for mail > delivery, but we are still stuck with deliver as LDA. > So for mail delivery we will not have the fixed user/server assignment, > but rather a random assignment. > > What NFS- and locking settings should be used on the backend servers in > this case. > And should i use different settings for the host the deliver is executed > than the IMAP/POP3 hosts which are accessed through a director proxy. All the LDA/IMAP/POP3 servers should use these settings to minimize problems: mail_nfs_storage = yes mail_nfs_index = yes Once you're using LMTP proxying also you can remove those. Otherwise the settings are the same. From leo at strike.wu.ac.at Wed Nov 16 17:04:20 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Wed, 16 Nov 2011 16:04:20 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <1320441536.21919.206.camel@hurina> References: <4EAED444.70608@strike.wu.ac.at> <1320441536.21919.206.camel@hurina> Message-ID: <4EC3D0F4.7020204@strike.wu.ac.at> On 11/04/2011 10:18 PM, Timo Sirainen wrote: > On Mon, 2011-10-31 at 18:00 +0100, Alexander 'Leo' Bergolth wrote: >> NAMESPACE_2='maildir:/var/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep' >> NAMESPACE_2_PREFIX=Shared/spamrep/ > > The settings are now named. > http://wiki2.dovecot.org/MailLocation#Custom_namespace_location shows > the basics. So for example if you have: > > namespace foo { > .. > } > > You can change it via: > > NAMESPACE/FOO/PREFIX=Shared/spamrep/ > NAMESPACE/FOO/SEP=/ > ..etc.. > > The '/' character is a bit problematic with shells scripts though, see > the wiki link how to do it via "env" at least. > > Another problem is if you want to add more namespaces than you have > configured. IIRC it's possible to simply add: > > NAMESPACE=foo Thanks a lot, Timo! Changing to named, /-separated environment variables did the job. NAMESPACE/S-SPAMREP/LIST=yes NAMESPACE/S-SPAMREP/LOCATION=maildir:/var/spool/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep NAMESPACE/S-SPAMREP/PREFIX=Shared/spamrep/ NAMESPACE/S-SPAMREP/SEPARATOR=/ NAMESPACE/S-SPAMREP/TYPE=public NAMESPACE/S-TESTSHARED/LIST=yes NAMESPACE/S-TESTSHARED/LOCATION=maildir:/var/spool/mail/Shared/testshared:INDEX=~/Maildir/index/Shared/testshared NAMESPACE/S-TESTSHARED/PREFIX=Shared/testshared/ NAMESPACE/S-TESTSHARED/SEPARATOR=/ NAMESPACE=s-testshared s-spamrep NAMESPACE/S-TESTSHARED/TYPE=public USERDB_KEYS=SYSTEM_GROUPS_USER UID GID HOME NAMESPACE/S-SPAMREP/LIST NAMESPACE NAMESPACE/S-TESTSHARED/SEPARATOR NAMESPACE/S-TESTSHARED/TYPE NAMESPACE/S-TESTSHARED/PREFIX NAMESPACE/S-TESTSHARED/LIST NAMESPACE/S-TESTSHARED/LOCATION NAMESPACE/S-SPAMREP/SEPARATOR NAMESPACE/S-SPAMREP/TYPE NAMESPACE/S-SPAMREP/PREFIX NAMESPACE/S-SPAMREP/LOCATION > And more than one is probably either "foo bar" or "foo,bar". If you get > something to work, please add example to > http://wiki2.dovecot.org/PostLoginScripting :) I'll add my script to the Wiki. Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From tss at iki.fi Wed Nov 16 17:04:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 17:04:42 +0200 Subject: [Dovecot] Creating a password_scheme In-Reply-To: References: Message-ID: <1321455883.21919.556.camel@hurina> On Thu, 2011-11-17 at 01:13 +1100, Jordi Llonch wrote: > I am trying to develop a password_scheme for the Django passwords. > > After building and installing the module in > $module_dir/auth/password/djangopass.so, also tried libdjangopass.so and > libdjangopass_plugin.so, Dovecot does not recognize the scheme: There is no longer a separate password/ directory. Just put it to $module_dir/auth/. Where do you see the password/ directory documented? It should be changed.. From achekalin at lazurit.com Wed Nov 16 17:16:35 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 16 Nov 2011 18:16:35 +0300 Subject: [Dovecot] How to do SEARCH with doveadm Message-ID: <4EC3D3D3.7000601@lazurit.com> Hello, I'm still playing with search messages from/to specified email address. In fact I need to do that to take such messages and copy it to another mailbox (so that box will contain 'Search results' and will be available to examine by any IMAP client). I've put my mailings in mdbox storage format, and use latest 2.1 beta version of Dovecot to be sure I use the freshest approach in search (and plan to use Solr FTS as well). The setup is each, I use passdb { driver = passwd-file args = /etc/dovecot/auth/%Lu } and userdb { driver = static args = uid=dovecot gid=dovecot home=/MAIL/mail_storage/%Lu } and put user:password pair in file named after my mailbox in /etc/dovecot/auth/ (let's name it test at domain.com), and the messages are at /MAIL/mail_storage/test at domain.com/... While I use it from IMAP client it works nicely. Ok, I put somewhat 5800 messages in INBOX, and IMAP pretty fast in working with it. But as I try to use doveadm to search for messages I need I simple lost my way (doing it under root): # doveadm search ALL doveadm(root): Error: chdir(/root/) failed: Permission denied (euid=104(dovecot) egid=106(dovecot) missing +x perm: /root, dir owned by 0:0 mode=0700) doveadm(root): Error: chdir(/root) failed: Permission denied Whatever cli command I try doveadv try to find messages in /root dispute it is at /MAIL/mail_storage/%Lu. How can I make doveadv use right mail path? In fact, I don't want to use system users to work with mail, it'll be simple virtual mail user, so it is pointless to set up special system user to do that search. I think I lost myself in general concept of how Dovecot handles users and credentials, so please please please push me in right direction! Thank you, Alexander From micah at riseup.net Wed Nov 16 18:09:12 2011 From: micah at riseup.net (micah anderson) Date: Wed, 16 Nov 2011 11:09:12 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <1321388840.21919.520.camel@hurina> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> Message-ID: <87ehx8gic7.fsf@algae.riseup.net> On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen wrote: > On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > > I really like the feature where you can define quota rules with percents > > which trigger off of the default values[0] (so you can set the Trash to > > allow for 10% more of the user's quota for example). > > > > What I would really love in dovecot would be for the ability to > > configure a quota rule for dovecot-lda. I would like to configure things > > so we don't bounce emails for users until they are well over quota, the > > IMAP quota plugin is a really great way to notify people that they are > > over quota because it fails to write to other folders.... that should be > > enough to get people's attention that they need to deal with things, but > > bouncing is harsh. > > You can give LDA-specific quota settings. Like maybe: > > protocol lda { > plugin { > quota_rule2 = INBOX:storage=200% > } > } Aha! Just what I was looking for. I presume the number following 'quota_rule' must be an unique number across all quota_rules in my config? > or even not enforcing quota at all: > > protocol lda { > plugin { > quota = maildir:User quota:noenforcing > } > } I was not aware of the noenforcing option, that answered another question I had not asked yet, so thanks for that! :) micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 18:23:02 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:23:02 +0100 Subject: [Dovecot] Chroot In-Reply-To: <1321455546.21919.553.camel@hurina> References: <1321386260.21919.505.camel@hurina> <1321455546.21919.553.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 3:59 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: >> >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >> >> such file or directory >> > >> > This is done because you used %%h, to look up home directory from >> > userdb. If you don't use %%h, it's not done. >> > >> >> Thanks but my db backend returns home dir in chroot like >> /test at example.com >> >> while the realpath with no chroot is: >> /var/mail/test at example.com >> >> So both locations in this case are the same >> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u >> location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u > > Yes, they are the same, but the difference is that with the second one > you shouldn't fail with the userdb lookup error. Right? > Thanks a lot it almost helped. But now there is another problem: Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX ** test at apisoft.pl R=localuser T=local_delivery: Child process of local_delivery transport (running command "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain") was terminated by signal 11 (Segmentation fault) But it anly appear if user has .sieve filter with mail redirection (uses submission_host) From bambero at gmail.com Wed Nov 16 18:27:13 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:27:13 +0100 Subject: [Dovecot] Chroot In-Reply-To: References: <1321386260.21919.505.camel@hurina> <1321455546.21919.553.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 5:23 PM, Bambero wrote: > On Wed, Nov 16, 2011 at 3:59 PM, Timo Sirainen wrote: >> On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: >>> >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >>> >> such file or directory >>> > >>> > This is done because you used %%h, to look up home directory from >>> > userdb. If you don't use %%h, it's not done. >>> > >>> >>> Thanks but my db backend returns home dir in chroot like >>> /test at example.com >>> >>> while the realpath with no chroot is: >>> /var/mail/test at example.com >>> >>> So both locations in this case are the same >>> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u >>> location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u >> >> Yes, they are the same, but the difference is that with the second one >> you shouldn't fail with the userdb lookup error. Right? >> > > Thanks a lot it almost helped. But now there is another problem: > Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX > ** test at apisoft.pl R=localuser T=local_delivery: Child process of > local_delivery transport (running command > "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f > $sender_address -a $original_local_part@$original_domain") was > terminated by signal 11 (Segmentation fault) > > But it anly appear if user has .sieve filter with mail redirection > (uses submission_host) > Sorry wrong thread. I this case you were right now everything work fine. From tss at iki.fi Wed Nov 16 18:27:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 18:27:20 +0200 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <87ehx8gic7.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> Message-ID: <1321460847.21919.562.camel@hurina> On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote: > > protocol lda { > > plugin { > > quota_rule2 = INBOX:storage=200% > > } > > } > > Aha! Just what I was looking for. I presume the number following > 'quota_rule' must be an unique number across all quota_rules in my > config? No only unique, but all of the numbers must exist. For example quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10 isn't used, unless quota_rule2..9 also exist. From bambero at gmail.com Wed Nov 16 18:28:16 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:28:16 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321454266.21919.545.camel@hurina> References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 3:37 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 15:00 +0100, Bambero wrote: >> >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol >> >> synchronization error (next input sent too soon: pipelining was not >> >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next >> >> input="MAIL FROM:\\r\\n" >> > >> > This basically says that Dovecot's SMTP client is being noncompliant, >> > because Exim doesn't advertise PIPELINING extension. But I'd rather not >> > add more code to fix this, since PIPELINING gives you a little bit >> > better performance anyway and you can most likely fix this by modifying >> > Exim's configs in some way (I can't believe Exim wouldn't support >> > PIPENING..). >> > >> > >> Thanks for your reply. >> Indeed exim supports PIPELINING by default: > > Oh. Well, see if the attached patch helps? If it does, I'll add it to > next version. > > Thanks a lot it almost helped. But now there is another problem: Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX ** test at apisoft.pl R=localuser T=local_delivery: Child process of local_delivery transport (running command "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain") was terminated by signal 11 (Segmentation fault) But it anly appear if user has .sieve filter with mail redirection (uses submission_host) From micah at riseup.net Wed Nov 16 18:31:01 2011 From: micah at riseup.net (micah anderson) Date: Wed, 16 Nov 2011 11:31:01 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <1321460847.21919.562.camel@hurina> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> <1321460847.21919.562.camel@hurina> Message-ID: <87ty64f2re.fsf@algae.riseup.net> On Wed, 16 Nov 2011 18:27:20 +0200, Timo Sirainen wrote: > On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote: > > > protocol lda { > > > plugin { > > > quota_rule2 = INBOX:storage=200% > > > } > > > } > > > > Aha! Just what I was looking for. I presume the number following > > 'quota_rule' must be an unique number across all quota_rules in my > > config? > > No only unique, but all of the numbers must exist. For example > quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10 > isn't used, unless quota_rule2..9 also exist. Ah, interesting, good to know. I presume that 'quota_rule' stands in for 'quota_rule1' and one does not need to specify a quota_rule1 if one has a 'quota_rule'. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From stan at hardwarefreak.com Wed Nov 16 18:36:12 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 16 Nov 2011 10:36:12 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC35501.6050601@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> <4EC35501.6050601@lazurit.com> Message-ID: <4EC3E67C.4050107@hardwarefreak.com> On 11/16/2011 12:15 AM, Alexander Chekalin wrote: > Hello, Stan, > >> This is why I recommended mbox in the first place. If your only writes >> to these mailbox files are appends of new messages, mbox is the best >> format by far. It's faster at appending than any other format, and it's >> faster for searching than any other. > > I now seriously consider to use mdbox due to its nice self-regulation. > After all it I believe mdbox should do file compression on its own, no > cron scripts required. mbox and mdbox each has strengths and weaknesses. mbox will compress with a higher ratio than mdbox. You already have a nightly script that moves all mail from the day into a new file. Piping that through gzip or bzip2 is a no brainer. It'll add one line to your existing script, if that. Dovecot will decompress the file transparently when you access it via IMAP. And again since it's a single file searching it is much faster. With mbox you will have a single file for each day of emails. This seems ideal for archive purposes, one file per day. mdbox does fully transparent de/compression which is nice. The downside is that Dovecot does dbox compression on a per email basis, not a per file basis. So your compression ratio will be much less than with mbox, especially with bzip2 which works best on files over 900KB in size. Most emails are less than 8KB. Using mdbox will yield multiple files per day of emails instead of just one. Either format is much better than maildir for archiving. >> It's an archive. You're not going to use maildir so you don't need >> random IOPS performance. Thus RAID5/6 are a much better fit for an >> archive as you get better read performance, with more than adequate >> write performance, and you use less disks. And as this is an archive, >> you don't need real time automatic/transparent compression. Thus I >> recommend something like: >> >> 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled >> 2.6.39 or later kernel >> 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors >> e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 >> Specify a strip size of 256KB for the array >> Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading >> ahead 1024 sectors at a time instead of the default of 256. This >> will speed up your searches quite a bit. >> 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults >> 4. mbox w/zlib plugin. Compress daily files each night with a script >> 5. You don't need LVM with a good RAID card (or with mdraid). This >> controller can expand the RAID5 up to 8 drives (up to 32 drives max >> using SAS expanders) > > We are considering to get HP DL180G6 server for 8 or 14 drives bays The P410 tops out at 8 drives, so get the 8 drive model. Start with 4 x 2TB drives in RAID5. Add 4 more drives when you need the capacity, and when drive prices are back down to normal (see below). http://h18004.www1.hp.com/products/quickspecs/13248_na/13248_na.html > (base model price is somewhat equal, but additional drives adds up cost) Especially right now in 2011. Flooding in Thailand, where 25% of the world's drives are produced, has doubled the cost of all hard drives worldwide. Now is a horrible time to buy spinning drives. I've read it may be 12 months before prices start coming back down... > with HP Smart Array P410 RAID controller (some servers are equipped with > this controller by default) with 256 Mb battery-backed cache, but I'll > check your suggestions! The P410 should be fine for a dedicated archive server. > What memory size should I plan in the server? You're talking about AMD64 > OS image, and 64-bit software are like to consume more memory that > 32-bit, so looks like your talking about pretty huge RAM, and I don't > believe it's necessary, or maybe I'm wrong? The memory footprint of 64bit binaries is nothing to worry about. The additional amount consumed is more than offset by the performance gained with direct access to RAM above 4GB compared to the performance of PAE. Keep in mind that 90% of your memory will be eaten by Linux buffer cache. Your binaries will account for less than 5% of your RAM consumption. If I understand correctly how you will use this archive server, then 8GB should be plenty. 8GB is standard on the 8 drive DL180 G6. http://h18004.www1.hp.com/products/quickspecs/13248_na/13248_na.html > Problem is I have no experience with XFS and not sure I can tune it in > the best way, so I'll go with mkfs.xfs defaults, I think. With only 4 drives and using a P410 w/cache and RAID5, doing manual XFS tuning isn't necessary for good performance, especially for an archive application which is data heavy, not metadata heavy. Setting sunit/swidth to match the RAID5 layout may increase performance slightly due to stripe aligned writes, but not enough that I'd worry about it. Just use the mkfs.xfs defaults. If you get the BBWC for the P410, enable the controller write cache, and mount XFS with 'nobarrier'. This will increase write performance quite a bit as fsyncs will complete instantly. > Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins > and 2.1 handle that much better, but I don't like the idea of use > unstable in production. Me neither. Speaking of archive/search, did you take look at Enkive yet? http://www.enkive.org/ > Thank you for taking your time on my case, You're welcome Alexander. -- Stan P.S. You may wish to implement dnswl.org ;) From tss at iki.fi Wed Nov 16 18:38:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 18:38:07 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> Message-ID: <1321461498.21919.566.camel@hurina> On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: > Thanks a lot it almost helped. But now there is another problem: > Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX > ** test at apisoft.pl R=localuser T=local_delivery: Child process of > local_delivery transport (running command > "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f > $sender_address -a $original_local_part@$original_domain") was > terminated by signal 11 (Segmentation fault) > > But it anly appear if user has .sieve filter with mail redirection > (uses submission_host) It's difficult to debug crashes without gdb backtrace. But since you're running in a strange chrooted environment, it may be difficult to get a core file.. Probably the easiest way would be if you apply the attached patch, then try to deliver a mail that causes this crash (and no other mails), and attach gdb into the process during the 10 second wait: gdb -p `pidof dovecot-lda` cont bt full -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 286 bytes Desc: not available URL: From tss at iki.fi Wed Nov 16 20:02:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:02:18 +0200 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <4EBACFC0.6060703@gmx.de> References: <4EB6AFE3.3020001@gmx.de> <4EBACFC0.6060703@gmx.de> Message-ID: <1321466541.21919.576.camel@hurina> On Wed, 2011-11-09 at 20:08 +0100, e-frog wrote: > On 06.11.2011 17:03, wrote e-frog: > > 1 lsub "" "virtual/*" > > * LSUB () "/" "virtual" > > * LSUB () "/" "virtual/Lists" > > * LSUB () "/" "virtual/unread" > > 1 OK Lsub completed. > > Just tested this again with 2.1beta1 with the same result. Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/16024d03f66e From tss at iki.fi Wed Nov 16 20:15:44 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:15:44 +0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB72177.8090801@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> Message-ID: <1321467345.21919.580.camel@hurina> On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: > Ed W schrieb: > > > See the "sticky" in my reply. You use one of several techniques to > > ensure that users always end up on the server with the indexes on. That > > way much of the IO is served from that local machine and you only access > > the SAN for the (in theory much less frequent) access to the mail files > > themselves. > > I know you can afford that (IMAP) users always end up on one particular > server but afaik this only works for incoming IMAP connections. > > My mail exchangers use dovecot-lda and I think indexes will be written > from these servers too or am I wrong with this? You can use LMTP and LMTP proxying. From tss at iki.fi Wed Nov 16 20:18:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:18:55 +0200 Subject: [Dovecot] Missing public folder In-Reply-To: <201111071110.20143.komodo@uvt.cz> References: <201111071110.20143.komodo@uvt.cz> Message-ID: <1321467535.21919.582.camel@hurina> On Mon, 2011-11-07 at 11:10 +0100, komodo wrote: > Hi > > i am using dovecot 2.0.13 and i can't see public folder in my mail client. But what is strange, it > was working after instalation, and i don't know when it dissapeared. Maybe after some upgrade ? Try talking IMAP protocol directly: telnet localhost 143 a login user pass b list "" * c select spam_learner > namespace { > location = maildir:/home/shared/Maildir Is this the "spam_learner" mailbox itself alone, or does it have children? ("spam_learner/spam"?) From tss at iki.fi Wed Nov 16 20:25:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:25:29 +0200 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <4EB7BF9A.9020900@webfusion.com> References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> <4EB7BF9A.9020900@webfusion.com> Message-ID: <1321467929.21919.583.camel@hurina> On Mon, 2011-11-07 at 13:23 +0200, Mark Zealey wrote: > 04-11-2011 23:56, Timo Sirainen yazm??: > From my reading of the code in 2.0.15, you pass the local & remote IP > in lmtp/commands.c:427 but the struct mail_storage_service_input only > has remote_ip and local_ip fields, not port fields (unlike the > auth_user_info struct which has both ip's and port entries as well). I > think that the mail_storage_service_input struct needs the uint > local_port, remote_port fields adding in & appropriate code changes to > pass these through? Yeah, you're right, I missed that one. Added to hg now. From tss at iki.fi Wed Nov 16 20:31:15 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:31:15 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC3FDB5.7090301@goldey.net> References: <4EC2BE26.3030807@goldey.net> <4EC3FDB5.7090301@goldey.net> Message-ID: <1321468275.21919.586.camel@hurina> Your message was discarded by mailman, probably because it was html-only. Anyway: On Wed, 2011-11-16 at 13:15 -0500, Mark Goldey wrote: > (gdb) p *auth_request->userdb->userdb->iface > $2 = {name = 0x807023e "vpopmail", preinit = 0x8065d60 > , init = 0, deinit = 0, > lookup = 0x8066010 , iterate_init = 0, iterate_next > = 0, iterate_deinit = 0} OK, so "auth_request->userdb->userdb->iface->lookup" should call vpopmail_lookup(), it's not NULL as this says: > > (gdb) bt full > > #0 0x00000000 in ?? () So something's going strangely wrong. Maybe gcc optimizations cause this and the real problem happens later. Try compiling auth process with optimizations: - Remove -O2 from src/auth/Makefile - cd src/auth ; make clean ; make ; make install Is the backtrace different? From tss at iki.fi Wed Nov 16 20:33:27 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:33:27 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: <1321468407.21919.588.camel@hurina> On Tue, 2011-11-15 at 20:50 -0500, Alex wrote: > > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging > > Yes, looks like permissions. I have the following now in syslog: > > Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: > setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): > Operation not permitted > Where is it trying to write that it can't? It's trying to change the process's UID/GID to mail/mail for some reason. Have you set "mail_uid=mail" or something? > Does alexuser need to be a member of group mail? No. From tss at iki.fi Wed Nov 16 20:34:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:34:14 +0200 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: <1321385991.21919.503.camel@hurina> Message-ID: <1321468455.21919.589.camel@hurina> On Tue, 2011-11-15 at 16:04 -0800, Mark Moseley wrote: > > The gotcha is that you have two completely independent quotas with > > independent usage/limits for the INBOX and Archive namespaces. If that > > is what you want, it should all be fine. > > Nope, that's totally fine. The idea is to put Archive on cheaper > (slower) storage and then grant more generous quotas there to make it > worth their while to use, without slowing down their Inbox. Another > application would be to put their Spam in another namespace (for > people who choose to have it put in a separate folder) with a lower > quota, again to offload it onto cheaper storage, since hardly anyone > actually looks at it. Should be fine then. > Or is this something that I could be doing more transparently in 2.1 with imapc? I don't really see how that could help. From moseleymark at gmail.com Wed Nov 16 20:40:45 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Wed, 16 Nov 2011 10:40:45 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: <1321468455.21919.589.camel@hurina> References: <1321385991.21919.503.camel@hurina> <1321468455.21919.589.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 10:34 AM, Timo Sirainen wrote: > On Tue, 2011-11-15 at 16:04 -0800, Mark Moseley wrote: >> > The gotcha is that you have two completely independent quotas with >> > independent usage/limits for the INBOX and Archive namespaces. If that >> > is what you want, it should all be fine. >> >> Nope, that's totally fine. The idea is to put Archive on cheaper >> (slower) storage and then grant more generous quotas there to make it >> worth their while to use, without slowing down their Inbox. Another >> application would be to put their Spam in another namespace (for >> people who choose to have it put in a separate folder) with a lower >> quota, again to offload it onto cheaper storage, since hardly anyone >> actually looks at it. > > Should be fine then. > >> Or is this something that I could be doing more transparently in 2.1 with imapc? > > I don't really see how that could help. Ah, bummer. I thought maybe 2.1 could proxy to a separate folder or namespace (but I've also barely had a chance to look at it), like so certain folders would be grabbed from a proxy. Haven't really thought that through though :) From pw at wk-serv.de Wed Nov 16 20:40:50 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 16 Nov 2011 19:40:50 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <1321467345.21919.580.camel@hurina> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> Message-ID: <4EC403B2.60005@wk-serv.de> Timo Sirainen schrieb: > On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: >> >> My mail exchangers use dovecot-lda and I think indexes will be written >> from these servers too or am I wrong with this? > > You can use LMTP and LMTP proxying. I already use lmtp:unix:private/dovecot-lmtp as transport but where is the link to the indexes? From bambero at gmail.com Wed Nov 16 21:25:10 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 20:25:10 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321461498.21919.566.camel@hurina> References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 5:38 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: >> Thanks a lot it almost helped. But now there is another problem: >> Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX >> ** test at apisoft.pl R=localuser T=local_delivery: Child process of >> local_delivery transport (running command >> "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f >> $sender_address -a $original_local_part@$original_domain") was >> terminated by signal 11 (Segmentation fault) >> >> ?But it anly appear if user has .sieve filter with mail redirection >> (uses submission_host) > > It's difficult to debug crashes without gdb backtrace. But since you're > running in a strange chrooted environment, it may be difficult to get a > core file.. > > Probably the easiest way would be if you apply the attached patch, then > try to deliver a mail that causes this crash (and no other mails), and > attach gdb into the process during the 10 second wait: > > gdb -p `pidof dovecot-lda` > cont > > bt full > > Can you see something here: gdb -p `pgrep dovecot-lda` GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-openbsd5.0". Attaching to process 4118 0x030e6719 in ?? () (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x0c96c825 in ?? () (gdb) bt full #0 0x0c96c825 in ?? () No symbol table info available. #1 0x7dfa61c0 in ?? () No symbol table info available. #2 0x00000001 in ?? () No symbol table info available. #3 0x00000001 in ?? () No symbol table info available. #4 0x00000002 in ?? () No symbol table info available. #5 0x2665d31c in ?? () No symbol table info available. #6 0x2665d31c in ?? () No symbol table info available. #7 0xcfbe13e8 in ?? () No symbol table info available. #8 0x0665fd79 in ?? () No symbol table info available. #9 0x00000000 in ?? () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: , process 4118 From bambero at gmail.com Wed Nov 16 21:37:32 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 20:37:32 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 8:25 PM, Bambero wrote: > On Wed, Nov 16, 2011 at 5:38 PM, Timo Sirainen wrote: >> On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: >>> Thanks a lot it almost helped. But now there is another problem: >>> Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX >>> ** test at apisoft.pl R=localuser T=local_delivery: Child process of >>> local_delivery transport (running command >>> "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f >>> $sender_address -a $original_local_part@$original_domain") was >>> terminated by signal 11 (Segmentation fault) >>> >>> ?But it anly appear if user has .sieve filter with mail redirection >>> (uses submission_host) >> >> It's difficult to debug crashes without gdb backtrace. But since you're >> running in a strange chrooted environment, it may be difficult to get a >> core file.. >> >> Probably the easiest way would be if you apply the attached patch, then >> try to deliver a mail that causes this crash (and no other mails), and >> attach gdb into the process during the 10 second wait: >> >> gdb -p `pidof dovecot-lda` >> cont >> >> bt full >> >> > > Can you see something here: > > gdb -p `pgrep dovecot-lda` > GNU gdb 6.3 > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. ?Type "show warranty" for details. > This GDB was configured as "i386-unknown-openbsd5.0". > Attaching to process 4118 > 0x030e6719 in ?? () > (gdb) cont > Continuing. > > Program received signal SIGSEGV, Segmentation fault. > 0x0c96c825 in ?? () > (gdb) bt full > #0 ?0x0c96c825 in ?? () > No symbol table info available. > #1 ?0x7dfa61c0 in ?? () > No symbol table info available. > #2 ?0x00000001 in ?? () > No symbol table info available. > #3 ?0x00000001 in ?? () > No symbol table info available. > #4 ?0x00000002 in ?? () > No symbol table info available. > #5 ?0x2665d31c in ?? () > No symbol table info available. > #6 ?0x2665d31c in ?? () > No symbol table info available. > #7 ?0xcfbe13e8 in ?? () > No symbol table info available. > #8 ?0x0665fd79 in ?? () > No symbol table info available. > #9 ?0x00000000 in ?? () > No symbol table info available. > (gdb) quit > The program is running. ?Quit anyway (and detach it)? (y or n) y > Detaching from program: , process 4118 > I forgot to say that mails are delivered using SMTP so submission_host works, mails are strored i destination mailbox too. Only this Segmentation Fault ... From tss at iki.fi Wed Nov 16 21:45:34 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 21:45:34 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: <1321472737.21919.596.camel@hurina> On Wed, 2011-11-16 at 20:25 +0100, Bambero wrote: > Can you see something here: No, the "??" isn't very helpful. > gdb -p `pgrep dovecot-lda` > This GDB was configured as "i386-unknown-openbsd5.0". Oh, this is OpenBSD. Try if this works instead: gdb /usr/local/libexec/dovecot/dovecot-lda `pgrep dovecot-lda` If it still gives only "??" output, that didn't work either.. From tss at iki.fi Wed Nov 16 21:47:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 21:47:17 +0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EC403B2.60005@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> Message-ID: <1321472837.21919.598.camel@hurina> On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: > Timo Sirainen schrieb: > > On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: > >> > >> My mail exchangers use dovecot-lda and I think indexes will be written > >> from these servers too or am I wrong with this? > > > > You can use LMTP and LMTP proxying. > > I already use lmtp:unix:private/dovecot-lmtp as transport but where is > the link to the indexes? You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot LMTP proxy, which would forward the connection to the backend server which handles that user's IMAP/POP3/LMTP connections. From guenther at rudersport.de Wed Nov 16 22:08:40 2011 From: guenther at rudersport.de (Karsten =?ISO-8859-1?Q?Br=E4ckelmann?=) Date: Wed, 16 Nov 2011 21:08:40 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: <1321474120.4854.13.camel@monkey> On Tue, 2011-11-15 at 19:00 -0500, Alex wrote: > DELIVER=/usr/libexec/dovecot/deliver > > :0 fhW > | $DELIVER -m xspamtest Do you really want the pipe to be a (f)ilter? What do you expect deliver to pass back? And you're feeding deliver the mail (h)eaders only, dropping the body. -- char *t="\10pse\0r\0dtu\0. at ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} From sb at dod.no Thu Nov 17 00:45:28 2011 From: sb at dod.no (Steinar Bang) Date: Wed, 16 Nov 2011 23:45:28 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT References: <201111091029.37963.thomas@koch.ro> <1320846789.21919.361.camel@hurina> Message-ID: <87sjlnr8jb.fsf@dod.no> >>>>> Timo Sirainen : > I'm not as much against git anymore as I was when I switched to hg, > but I don't see much benefits in switching to git either. I highly > doubt I'd get even a single patch more if I used git instead of hg. FWIW the distingushing feature of git isn't that it is a ditributed VCS. It's the ease with which it does branching and merging. What put me off git for a long time, was Linus' arrogance wrt. to existing version control systems. But after using git, I have to admit that it is... quite clever. Especially the branching and merging stuff. From tss at iki.fi Thu Nov 17 01:30:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 01:30:42 +0200 Subject: [Dovecot] v2.0.16 released Message-ID: <1321486243.21919.620.camel@hurina> http://dovecot.org/releases/2.0/dovecot-2.0.16.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.16.tar.gz.sig I only now noticed that the VSZ limits weren't being enforced with earlier v2.0.x releases (or they were set 1024 times too high). So if a Dovecot process was leaking memory, it wasn't being killed by kernel. Now that this enforcing is done, some installations will probably start seeing errors about reaching these limits in normal operation. The default_vsz_limit is 256 MB. You may want to increase it in larger installations to 1 GB just in case. Besides the changes listed below, a lot of smaller fixes were done. v2.0.16 2011-11-17 Timo Sirainen * VSZ limits weren't being enforced for any processes. On server with large mailboxes you may now see errors about it if the limits aren't high enough. To fix them, either increase individual service { vsz_limit } values or simply increase the default_vsz_limit setting. * Proxying: If using ssl=yes or starttls=yes with a hostname (not IP) as proxy destination, require that the certificate matches the given hostname. * LMTP: Changed default client_limit to 1. This should improve LMTP throughput with default settings. * dsync: Quota is no longer enforced (i.e. dsync can't fail because user is over quota). + Added "auto" mail storage driver, which can be used to autodetect mailbox location and format. This behavior is already the default for empty mail_location setting, so this change is mainly useful for shared namespace's location setting. + checkpassword: Export all auth %variables to AUTH_* environment. From mysqlstudent at gmail.com Thu Nov 17 02:29:36 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:29:36 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321468407.21919.588.camel@hurina> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> Message-ID: Hi, >> > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging >> >> Yes, looks like permissions. I have the following now in syslog: >> >> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >> Operation not permitted > >> Where is it trying to write that it can't? > > It's trying to change the process's UID/GID to mail/mail for some > reason. Have you set "mail_uid=mail" or something? mail_uid isn't set, but I have the following in dovecot.conf: mail_gid = mail mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_privileged_group = mail Is that not correct? Thanks, Alex From tss at iki.fi Thu Nov 17 02:34:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 02:34:31 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> Message-ID: <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> On 17.11.2011, at 2.29, Alex wrote: >>> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >>> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >>> Operation not permitted >> >>> Where is it trying to write that it can't? >> >> It's trying to change the process's UID/GID to mail/mail for some >> reason. Have you set "mail_uid=mail" or something? > > mail_uid isn't set, but I have the following in dovecot.conf: > > mail_gid = mail The error most likely happens because of this. Is there a reason why you want this? > mail_privileged_group = mail This should be enough for creating lock files to /var/spool/mail/. Although dovecot-lda can't use this setting, since it's not started as root.. From mysqlstudent at gmail.com Thu Nov 17 02:49:53 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:49:53 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: Hi, >>>> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >>>> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >>>> Operation not permitted >>> >>>> Where is it trying to write that it can't? >>> >>> It's trying to change the process's UID/GID to mail/mail for some >>> reason. Have you set "mail_uid=mail" or something? >> >> mail_uid isn't set, but I have the following in dovecot.conf: >> >> mail_gid = mail > > The error most likely happens because of this. Is there a reason why you want this? > >> mail_privileged_group = mail > > This should be enough for creating lock files to /var/spool/mail/. Although dovecot-lda can't use this setting, since it's not started as root.. Okay, that fixed it. It's now delivering mail properly, thanks. Just to be clear I understand, if I use the following .procmailrc, it should save all messages with the X-Spam-Status header in xspamstatus, and everything else gets delivered to /var/spool/mail/user, correct? DELIVER=/usr/libexec/dovecot/deliver :0 * ^X-Spam-Status | $DELIVER -m xspamstatus :0 | $DELIVER This is just a simple test which appears to work the way I outlined, but wanted to be sure. Thanks again for sticking with me. Best, Alex From mysqlstudent at gmail.com Thu Nov 17 02:56:37 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:56:37 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321474120.4854.13.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321474120.4854.13.camel@monkey> Message-ID: Hi, >> DELIVER=/usr/libexec/dovecot/deliver >> >> :0 fhW >> | $DELIVER -m xspamtest > > Do you really want the pipe to be a (f)ilter? What do you expect deliver > to pass back? And you're feeding deliver the mail (h)eaders only, > dropping the body. Yes, thanks. I pasted from another filter we were using, and forgot. procmail still seems to be much better than the deliver sieve language, despite how long it's been since I've used it :-) Thanks, Alex From mysqlstudent at gmail.com Thu Nov 17 02:57:12 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:57:12 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321474120.4854.13.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321474120.4854.13.camel@monkey> Message-ID: Hi, >> DELIVER=/usr/libexec/dovecot/deliver >> >> :0 fhW >> | $DELIVER -m xspamtest > > Do you really want the pipe to be a (f)ilter? What do you expect deliver > to pass back? And you're feeding deliver the mail (h)eaders only, > dropping the body. Yes, thanks. I pasted from another filter we were using, and forgot. procmail still seems to be much better than the deliver sieve language, despite how long it's been since I've used it :-) Thanks, Alex From mysqlstudent at gmail.com Thu Nov 17 03:42:04 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 20:42:04 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: Hi, My apologies if a previous message appeared twice. Not sure how that happened... > Okay, that fixed it. It's now delivering mail properly, thanks. > > Just to be clear I understand, if I use the following .procmailrc, it > should save all messages with the X-Spam-Status header in xspamstatus, > and everything else gets delivered to /var/spool/mail/user, correct? > > DELIVER=/usr/libexec/dovecot/deliver > > :0 > * ^X-Spam-Status > | $DELIVER -m xspamstatus I thought this was working, but it doesn't create or populate a folder named xspamstatus although it executes deliver: procmail: [30333] Wed Nov 16 20:39:36 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Match on "^X-Spam-Status" procmail: Assigning "LASTFOLDER=/usr/libexec/dovecot/deliver -m xspamstatus" >From smarchant at mydomain.com.au Wed Nov 16 20:39:36 2011 procmail: Executing "/usr/libexec/dovecot/deliver,-m,mymailbox" Subject: Riverside (97293) Folder: /usr/libexec/dovecot/deliver -m mymailbox 28400 I've set lda_mailbox_autocreate to yes, but it still doesn't create it. Is this folder relative to ~user/mail/ ? Is there an index file that dovecot-lda updates for imap? Can I just eliminate it entirely and just have procmail do all the delivery? Thanks again, Alex From sf.rique at gmail.com Thu Nov 17 03:53:29 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Wed, 16 Nov 2011 23:53:29 -0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EC16116.1000902@hardwarefreak.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> Message-ID: Jan-Frode Wich clustered filesytem do you have? I used to have ocfs2 but had problems with performance. So had to get back to ext4 and it solve the performance problem... My ocfs2 setup had some problems... but still.. Some numbers: OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 15 hours Ext4 1TB of maildir files. Full backup 16 Hours Incremental 1 hour Same LUN on storage. []'sf.rique On Mon, Nov 14, 2011 at 4:42 PM, Stan Hoeppner wrote: > On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > > > Agree. A non-clustered fs should give you better performance, and > > probably also be more reliable, if you can live with the SPoF and > > full downtime during patching/upgrades/maintenance. But I would expect > > xfs to be a better choice than ext*. > > Depends on the workload characteristics and how well the XFS filesystem > is tuned to the storage hardware. If setup properly, using many > allocation groups with fast spindles, a decent amount of BBWC, and a > high concurrency maildir workload (dozens to hundreds of delivery and > IMAP operations), XFS will runs circles around EXTx as it can > create/write/read to every AG in parallel. Much of EXT4's operation is > still serialized. This is why XFS outruns all other filesystems in the > highly parallel mail workload benchmarks I posted previously, EXTx by a > factor of 2-3. > > For smaller hosts that don't see parallelism, for example SOHO servers, > XFS will likely be slower than EXTx as the workload will be serialized. > > -- > Stan > From guenther at rudersport.de Thu Nov 17 04:18:38 2011 From: guenther at rudersport.de (Karsten =?ISO-8859-1?Q?Br=E4ckelmann?=) Date: Thu, 17 Nov 2011 03:18:38 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: <1321496318.4854.62.camel@monkey> On Wed, 2011-11-16 at 20:42 -0500, Alex wrote: [...] Unrelated to other dovecot specific questions... > Is there an index file that dovecot-lda updates for imap? Yes. Which would be the advantage of using dovecot deliver, directly or called from procmail. > Can I just eliminate it entirely and just have procmail do all the > delivery? Yes, you can. In fact, that's what I usually still use. Procmail can just do much more than sieve. And procmail doesn't scare me as much as sieve. But then again, I like Perl... And I've never yet encountered a problem with dovecot IMAP updating indexes on the fly -- which it does, unless deliver does it incrementally. However, wasn't your original question about converting procmail recipes to sieve? (Yes, it was.) So what would hold you back of just not converting? -- char *t="\10pse\0r\0dtu\0. at ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} From mysqlstudent at gmail.com Thu Nov 17 04:50:52 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 21:50:52 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321496318.4854.62.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> <1321496318.4854.62.camel@monkey> Message-ID: > Unrelated to other dovecot specific questions... > >> Is there an index file that dovecot-lda updates for imap? > > Yes. Which would be the advantage of using dovecot deliver, directly or > called from procmail. > >> Can I just eliminate it entirely and just have procmail do all the >> delivery? > > Yes, you can. > > In fact, that's what I usually still use. Procmail can just do much more > than sieve. And procmail doesn't scare me as much as sieve. But then > again, I like Perl... And I've never yet encountered a problem with > dovecot IMAP updating indexes on the fly -- which it does, unless > deliver does it incrementally. > > However, wasn't your original question about converting procmail recipes > to sieve? (Yes, it was.) So what would hold you back of just not > converting? Yes, because I hadn't realized it was possible to still use procmail at all. I had said previously, although possibly not clearly, that I'd prefer to use continue to use procmail -- sieve looks entirely too convoluted, and I just don't need to do mail filtering frequently enough to have the time to figure out sieve. It also didn't support a few other things I'm currently doing with procmail. I can't figure out why "deliver -m" doesn't work as expected. It just doesn't create the mailbox. Should I just use 'c' with procmail to make a copy to forward on to deliver? DELIVER=/usr/libexec/dovecot/deliver :0 { :0c * ^X-Spam-Status xspamstatus :0 | $DELIVER } There has to be a better way... Thanks, Alex From mgoldey at goldey.net Thu Nov 17 06:48:43 2011 From: mgoldey at goldey.net (Mark Goldey) Date: Wed, 16 Nov 2011 23:48:43 -0500 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 Message-ID: <4EC4922B.5050000@goldey.net> Sorry about the HTML. I removed the string "-O2" from the /auth Makefile and, for good measure, did make clean, make and make install for all of Dovecot. Still, it crashed when trying to log in via IMAP (webmail). Core dumped, here's the backtrace. [snip] Core was generated by `dovecot/auth -w'. Program terminated with signal 11, Segmentation fault. #0 0x00000000 in ?? () (gdb) bt full #0 0x00000000 in ?? () No symbol table info available. #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, args=0x82601c8) at auth-worker-client.c:405 auth_request = 0x82776e8 userdb_id = 1 #2 0x0805b630 in auth_worker_handle_line (client=0x82754d0, line=0x82757da "2\tUSER\t1\tuser=mgoldXX at goldey.net\tservice=imap\toriginal_username=mgoldXX at goldey.net\trequested_login_user\tlip=127.0.0.1\trip=127.0.0.1\tlport=143\trport=47574\tsecured=1\tsuccessful=1\tmech=PLAIN") at auth-worker-client.c:534 args = 0x82601c0 id = 2 ret = false #3 0x0805b890 in auth_worker_input (client=0x82754d0) at auth-worker-client.c:614 _data_stack_cur_id = 3 line = 0x82757da "2\tUSER\t1\tuser=mgoldXX at goldey.net\tservice=imap\toriginal_username=mgoldXX at goldey.net\trequested_login_user\tlip=127.0.0.1\trip=127.0.0.1\tlport=143\trport=47574\tsecured=1\tsuccessful=1\tmech=PLAIN" ret = 183 #4 0xb7726021 in io_loop_call_io (io=0x8275688) at ioloop.c:384 ioloop = 0x8268398 t_id = 2 #5 0xb77270d3 in io_loop_handler_run (ioloop=0x8268398) at ioloop-epoll.c:213 ctx = 0x8273f38 event = 0x8273f78 list = 0x82756b0 io = 0xbf80049f tv = {tv_sec = 2147483, tv_usec = 0} msecs = 1 ret = 1 i = 0 j = 0 call = 192 #6 0xb7725fa0 in io_loop_run (ioloop=0x8268398) at ioloop.c:405 No locals. #7 0xb7711baa in master_service_run (service=0x82682e8, callback=0x805e466 ) at master-service.c:481 No locals. #8 0x0805e7aa in main (argc=2, argv=0x82681c0) at main.c:298 c = -1 And, for good measure: (gdb) fr 1 #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, args=0x82601c8) at auth-worker-client.c:405 405 auth_request->userdb->userdb->iface-> (gdb) p *auth_request->userdb->userdb $1 = {args = 0x8268d10 "", cache_key = 0x0, blocking = true, id = 1, init_refcount = 1, iface = 0x807b320} (gdb) p *auth_request->userdb->userdb->iface $2 = {name = 0x8074f3d "vpopmail", preinit = 0x8069a27 , init = 0, deinit = 0, lookup = 0x806975f , iterate_init = 0, iterate_next = 0, iterate_deinit = 0} This is all Greek to me, but I'm happy to do more troubleshooting. --mhg From janfrode at tanso.net Thu Nov 17 11:34:49 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 17 Nov 2011 10:34:49 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> Message-ID: <20111117093449.GA18157@dibs.tanso.net> On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > Wich clustered filesytem do you have? We use IBM GPFS, with currently 7 servers working against shared LUNs from an IBM DS4800. > > My ocfs2 setup had some problems... but still.. > Some numbers: > > OCFS2 > 1TB of maildir files. > Full backup 36 Hours > Incremental 15 hours > > Ext4 > 1TB of maildir files. > Full backup 16 Hours > Incremental 1 hour Wow, local fs's are fast! We have split the backup process up to run on 6 of the servers, with each server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). The backup processing time varies quite a bit every day, but are mostly somewhere between 14-24 hours on each server. This sounds like something between 1.5x to 2x the incremental-performance you're seeing: 15 hours/incremental of 1TB ocfs2 = 15h/TB 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB 6x 20 hours for incremental of 12 TB GPFS = 10h/TB All our backups are incremental. -jf From stan at hardwarefreak.com Thu Nov 17 12:27:50 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 17 Nov 2011 04:27:50 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <1321383763.21919.484.camel@hurina> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> <1321383763.21919.484.camel@hurina> Message-ID: <4EC4E1A6.7060101@hardwarefreak.com> On 11/15/2011 1:02 PM, Timo Sirainen wrote: > On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > >> This is why I recommended mbox in the first place. If your only writes >> to these mailbox files are appends of new messages, mbox is the best >> format by far. It's faster at appending than any other format, and it's >> faster for searching than any other. > > Just as long as you're not simultaneously trying to read and write the > mbox file (or just write in 2+ sessions). Then there's a lot waiting on > locks. (mdbox has no read locks, and its write locks are very short > lived.) Of course. My understanding of Alexander's workflow is that copies of all daily new mail are written to an IMAP mailbox via some MTA bcc rule or sieve rule. A nightly script moves the daily mail to another mailbox created and named by date. These named mailboxes are then used for backup and the search function, but are never written to again. So I assume there is no simultaneous read/write of the archive mailboxes he performs searches on. It's possible I don't fully understand Alexander's work flow yet. -- Stan From Mark.Zealey at webfusion.com Thu Nov 17 12:57:09 2011 From: Mark.Zealey at webfusion.com (Mark Zealey) Date: Thu, 17 Nov 2011 10:57:09 +0000 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <1321467929.21919.583.camel@hurina> References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> <4EB7BF9A.9020900@webfusion.com>,<1321467929.21919.583.camel@hurina> Message-ID: Thanks so much for that have tested in 2.0.16 release now and is working fine. Mark ________________________________________ From: Timo Sirainen [tss at iki.fi] Sent: 16 November 2011 18:25 To: Mark Zealey Cc: Dovecot Mailing List Subject: Re: [Dovecot] Port variable in LMTP userdb lookups? On Mon, 2011-11-07 at 13:23 +0200, Mark Zealey wrote: > 04-11-2011 23:56, Timo Sirainen yazm??: > From my reading of the code in 2.0.15, you pass the local & remote IP > in lmtp/commands.c:427 but the struct mail_storage_service_input only > has remote_ip and local_ip fields, not port fields (unlike the > auth_user_info struct which has both ip's and port entries as well). I > think that the mail_storage_service_input struct needs the uint > local_port, remote_port fields adding in & appropriate code changes to > pass these through? Yeah, you're right, I missed that one. Added to hg now. From marco at carcano.ch Thu Nov 17 18:47:34 2011 From: marco at carcano.ch (Marco Carcano) Date: Thu, 17 Nov 2011 17:47:34 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <1320437339.21919.165.camel@hurina> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: Hello Timo, and thanks for your reply I waited to reply until got it another time as I already said, it does not happen very often, for example it happened on 12 november - the log is at the end of this mail > Enable mail_log plugin to make sure of this. > http://wiki2.dovecot.org/Plugins/MailLog I already did, but just for a few days: it does not happen very often that we loose mails, so I'm afraid I can damage the disks of the production server if I keep logging enabled for too much time - it will be a pain, years ago I had a server damaged because of logging enabled for too much time. I do not want to repeat such a painfull experience > >> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>> : saved mail to INBOX > > If Dovecot logs this, then the message definitely was saved to INBOX. it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - I suspect however that could be mine misconfiguration somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs (however I'm wondering why sometimes?!?) I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are cd /home/mailboxstore/theuser/Maildir grep "6000029222" */* |grep "RE:" grep "6000029222" .Drafts/* |grep "RE:" grep "6000029222" .Drafts/*/* |grep "RE:" grep "6000029222" .Junk/* |grep "RE:" grep "6000029222" .Posta\ eliminata/* |grep "RE:" grep "6000029222" .Posta\ indesiderata/* |grep "RE:" grep "6000029222" .Posta\ inviata/* |grep "RE:" grep "6000029222" .Sent/* |grep "RE:" grep "6000029222" .Templates/* |grep "RE:" grep "6000029222" .Trash/* |grep "RE:" and never got anything here is the log instead Nov 12 08:48:01 srv001 postfix/smtpd[1430]: connect from mail.tasnee.com[62.3.52.58] Nov 12 08:48:02 srv001 postfix/smtpd[1430]: 6C3874E4A9F: client=mail.tasnee.com[62.3.52.58] Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: warning: header Subject: RE: RFQ NO. 6000029222 - OUR OFFER NO. 2111221 from mail.tasnee.com[62.3.52.58]; from= to= proto=ESMTP helo= Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa > Nov 12 08:48:03 srv001 postfix/qmgr[4876]: 6C3874E4A9F: from=, size=9920, nrcpt=4 (queue active) Nov 12 08:48:06 srv001 postfix/smtpd[1442]: connect from localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 postfix/smtpd[1442]: 244774E4AA2: client=localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 postfix/cleanup[1434]: 244774E4AA2: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: from=, size=10323, nrcpt=4 (queue active) Nov 12 08:48:06 srv001 postfix/smtpd[1442]: disconnect from localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 amavis[8902]: (08902-05) Passed CLEAN, [62.3.52.58] [62.3.52.58] -> < user2 @ourdomain.local>,,,, Message-ID: <899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >, mail_id: z4aAgl2gBrfV, Hits: -0.592, size: 9919, queued_as: 244774E4AA2, 2632 ms Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 6C3874E4A9F: removed Nov 12 08:48:06 srv001 dovecot: lda(user3): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 dovecot: lda(user2): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1452]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.02/0/0.05, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 postfix/pipe[1445]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.01/0/0.06, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 dovecot: lda(theuser): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1447]: 244774E4AA2: to=, relay=dovecot, delay=0.09, delays=0.01/0.01/0/0.07, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 dovecot: lda(user4): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1450]: 244774E4AA2: to=, relay=dovecot, delay=0.26, delays=0.01/0.02/0/0.23, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: removed Nov 12 08:48:08 srv001 postfix/smtpd[1430]: disconnect from mail.tasnee.com[62.3.52.58] as you can see again, the mail seems delivered, but is not I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the whole config) thank you Marco Carcano From bind at enas.net Thu Nov 17 19:29:03 2011 From: bind at enas.net (Urban Loesch) Date: Thu, 17 Nov 2011 18:29:03 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: <4EC5445F.8@enas.net> Hi, On 17.11.2011 17:47, Marco Carcano wrote: > Hello Timo, and thanks for your reply > > I waited to reply until got it another time > > as I already said, it does not happen very often, for example it happened on 12 november - the log is at the end of this mail > > >> Enable mail_log plugin to make sure of this. >> http://wiki2.dovecot.org/Plugins/MailLog > > I already did, but just for a few days: it does not happen very often that we loose mails, so I'm afraid I can damage the disks of the production > server if I keep logging enabled for too much time - it will be a pain, years ago I had a server damaged because of logging enabled for too much time. > I do not want to repeat such a painfull experience I think logging is not a main reason for damaging disks. I have enabled mail_log plugin since march 2011 without problems and it helps me very often in such cases. > >> >>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>> : saved mail to INBOX >> >> If Dovecot logs this, then the message definitely was saved to INBOX. > > it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - I suspect however that could be mine misconfiguration > somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs > (however I'm wondering why sometimes?!?) > Could it be that some other person is downloading the mail via pop3 and then the client is deleting it from the server? This happens to me sometimes when a customer is accessing his account with a new client but forgot to disable the same account on his old PC. So it happens, that the old pc downloads all new mails, delete it then and the customer never see new mails on his new client. Regards Urban > I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are > > cd /home/mailboxstore/theuser/Maildir > > grep "6000029222" */* |grep "RE:" > grep "6000029222" .Drafts/* |grep "RE:" > grep "6000029222" .Drafts/*/* |grep "RE:" > grep "6000029222" .Junk/* |grep "RE:" > grep "6000029222" .Posta\ eliminata/* |grep "RE:" > grep "6000029222" .Posta\ indesiderata/* |grep "RE:" > grep "6000029222" .Posta\ inviata/* |grep "RE:" > grep "6000029222" .Sent/* |grep "RE:" > grep "6000029222" .Templates/* |grep "RE:" > grep "6000029222" .Trash/* |grep "RE:" > > and never got anything > > here is the log instead > > Nov 12 08:48:01 srv001 postfix/smtpd[1430]: connect from mail.tasnee.com[62.3.52.58] > Nov 12 08:48:02 srv001 postfix/smtpd[1430]: 6C3874E4A9F: client=mail.tasnee.com[62.3.52.58] > Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: warning: header Subject: RE: RFQ NO. 6000029222 - OUR OFFER NO. 2111221 from > mail.tasnee.com[62.3.52.58]; from= to= proto=ESMTP helo= > Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa> > Nov 12 08:48:03 srv001 postfix/qmgr[4876]: 6C3874E4A9F: from=, size=9920, nrcpt=4 (queue active) > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: connect from localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: 244774E4AA2: client=localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 postfix/cleanup[1434]: 244774E4AA2: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa> > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: from=, size=10323, nrcpt=4 (queue active) > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: disconnect from localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 amavis[8902]: (08902-05) Passed CLEAN, [62.3.52.58] [62.3.52.58] -> > ,,,, Message-ID: > <899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>, mail_id: z4aAgl2gBrfV, Hits: -0.592, size: 9919, queued_as: 244774E4AA2, 2632 ms > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, > relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: > queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 6C3874E4A9F: removed > Nov 12 08:48:06 srv001 dovecot: lda(user3): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 dovecot: lda(user2): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1452]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.02/0/0.05, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 postfix/pipe[1445]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.01/0/0.06, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 dovecot: lda(theuser): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1447]: 244774E4AA2: to=, relay=dovecot, delay=0.09, delays=0.01/0.01/0/0.07, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 dovecot: lda(user4): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1450]: 244774E4AA2: to=, relay=dovecot, delay=0.26, delays=0.01/0.02/0/0.23, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: removed > Nov 12 08:48:08 srv001 postfix/smtpd[1430]: disconnect from mail.tasnee.com[62.3.52.58] > > as you can see again, the mail seems delivered, but is not > > I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the > whole config) > > > thank you > > Marco Carcano > > > > > From tss at iki.fi Thu Nov 17 19:34:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 19:34:01 +0200 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> On 17.11.2011, at 18.47, Marco Carcano wrote: >>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>> : saved mail to INBOX >> >> If Dovecot logs this, then the message definitely was saved to INBOX. > > it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - Most likely reason for this is that the user's client deletes the message. Possibly an automatic client side filter or some UI issue that causes user to accidentally delete a mail. The mail_log plugin's logging would have showed if this was the case. > I suspect however that could be mine misconfiguration somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs (however I'm wondering why sometimes?!?) I can't think of any reason why it would randomly write to a wrong place. > I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are > > cd /home/mailboxstore/theuser/Maildir > > grep "6000029222" */* |grep "RE:" > grep "6000029222" .Drafts/* |grep "RE:" > grep "6000029222" .Drafts/*/* |grep "RE:" > grep "6000029222" .Junk/* |grep "RE:" > grep "6000029222" .Posta\ eliminata/* |grep "RE:" > grep "6000029222" .Posta\ indesiderata/* |grep "RE:" > grep "6000029222" .Posta\ inviata/* |grep "RE:" > grep "6000029222" .Sent/* |grep "RE:" > grep "6000029222" .Templates/* |grep "RE:" > grep "6000029222" .Trash/* |grep "RE:" Only the grep "6000029222" .Drafts/*/* |grep "RE:" was grepping from mail files. Easier would be just: grep -r "RE:.*6000029222" . > I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the whole config) I doubt this is related to configuration. But you could enable http://wiki2.dovecot.org/Plugins/Lazyexpunge so that messages won't be lost if they are expunged. The next time a message is lost, you would most likely find it from the lazy-expunge namespace. (Then you could write a script that deletes e.g. >1 week old files nightly.) From sf.rique at gmail.com Thu Nov 17 19:38:18 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Thu, 17 Nov 2011 15:38:18 -0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111117093449.GA18157@dibs.tanso.net> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> <20111117093449.GA18157@dibs.tanso.net> Message-ID: []'sf.rique How many servers have access to your maildir on ext4 so that you could > switch from ocfs2 to ext4? > I use ocfs2 in my test environment for four servers (2 MX and 2 IMAP) I used have 3 serves one mailman and 2 imaps. I experence that if take my loadbalancer and takes it all to just one server, itwould go faster becasue of cache. But once thsi server fails, the loadbalaner send it to the other one, but it simple does not work, clients experence timeouts, because the filesystem were too slow. On Thu, Nov 17, 2011 at 7:34 AM, Jan-Frode Myklebust wrote: On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > > > Wich clustered filesytem do you have? > > We use IBM GPFS, with currently 7 servers working against shared LUNs > from an IBM DS4800. > > > > > My ocfs2 setup had some problems... but still.. > > Some numbers: > > > > OCFS2 > > 1TB of maildir files. > > Full backup 36 Hours > > Incremental 15 hours > > > > Ext4 > > 1TB of maildir files. > > Full backup 16 Hours > > Incremental 1 hour > > Wow, local fs's are fast! > Much faster! I could not run a "du" before, if i did would crash the system, now i can! > We have split the backup process up to run on 6 of the servers, with each > server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). > The backup processing time varies quite a bit every day, but are mostly > somewhere between 14-24 hours on each server. This sounds like something > between 1.5x to 2x the incremental-performance you're seeing: > > 15 hours/incremental of 1TB ocfs2 = 15h/TB > 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB > 6x 20 hours for incremental of 12 TB GPFS = 10h/TB > > All our backups are incremental. > > > -jf > From e-frog at gmx.de Thu Nov 17 19:45:50 2011 From: e-frog at gmx.de (e-frog) Date: Thu, 17 Nov 2011 18:45:50 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <1321466541.21919.576.camel@hurina> References: <4EB6AFE3.3020001@gmx.de> <4EBACFC0.6060703@gmx.de> <1321466541.21919.576.camel@hurina> Message-ID: <4EC5484E.1040709@gmx.de> On 16.11.2011 19:02, wroteTimo Sirainen: > On Wed, 2011-11-09 at 20:08 +0100, e-frog wrote: >> On 06.11.2011 17:03, wrote e-frog: >>> 1 lsub "" "virtual/*" >>> * LSUB () "/" "virtual" >>> * LSUB () "/" "virtual/Lists" >>> * LSUB () "/" "virtual/unread" >>> 1 OK Lsub completed. >> >> Just tested this again with 2.1beta1 with the same result. > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/16024d03f66e > Thanks, Timo. Tested and works now as expected. From marco at carcano.ch Thu Nov 17 22:50:09 2011 From: marco at carcano.ch (Marco Carcano) Date: Thu, 17 Nov 2011 21:50:09 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> Message-ID: <126C0E4E-460B-447D-A85C-9AEC956020B5@carcano.ch> Hello Timo and Urban, and thank you for the very quick reply I have already thought about these things: I have already disabled mailbox deletion from POP3 clients, that actually just leave the messages on the server however, this time noone of the 4 recipients got the email - so it is really strange that all che clients deleted the same message and left all the others we can notice this only a few days after the issue, because customers send us a reminder forwarding the lost mail Il giorno 17/nov/11, alle ore 18:34, Timo Sirainen ha scritto: > On 17.11.2011, at 18.47, Marco Carcano wrote: > >>>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>>> : saved mail to INBOX >>> >>> If Dovecot logs this, then the message definitely was saved to >>> INBOX. >> >> it is exactly what I told to my colleagues, but belive me, sometime >> some mail get lost - > > Most likely reason for this is that the user's client deletes the > message. Possibly an automatic client side filter or some UI issue > that causes user to accidentally delete a mail. The mail_log > plugin's logging would have showed if this was the case. > >> I suspect however that could be mine misconfiguration somewhere, so >> that lda sometimes write the email not in the right place, but >> elsewhere, and just write the phrase " saved mail to INBOX in the >> logs (however I'm wondering why sometimes?!?) > > I can't think of any reason why it would randomly write to a wrong > place. maybe something related to LDAP lookups? what can happen if for some reason the LDAP server does not reply to the lookup? Timo, belive to me, I use dovecot since 3 years, and I am one of the thousands of happy user, I really can't explain what is happening on this server I'll do this trial: I'll enable lda debug logging ont a file on a USB hard disk - in order to know where exactly messages are stored, and enable also maillog too I'll keep you informed on what I'll find - maybe it will take some days, as I told it happens quite seldom thank you > >> I tried to find the missed email in the Maildir, but have not been >> able to get it - the commands used are >> >> cd /home/mailboxstore/theuser/Maildir >> >> grep "6000029222" */* |grep "RE:" >> grep "6000029222" .Drafts/* |grep "RE:" >> grep "6000029222" .Drafts/*/* |grep "RE:" >> grep "6000029222" .Junk/* |grep "RE:" >> grep "6000029222" .Posta\ eliminata/* |grep "RE:" >> grep "6000029222" .Posta\ indesiderata/* |grep "RE:" >> grep "6000029222" .Posta\ inviata/* |grep "RE:" >> grep "6000029222" .Sent/* |grep "RE:" >> grep "6000029222" .Templates/* |grep "RE:" >> grep "6000029222" .Trash/* |grep "RE:" > > Only the grep "6000029222" .Drafts/*/* |grep "RE:" was grepping from > mail files. Easier would be just: > > grep -r "RE:.*6000029222" . > >> I really think is some kind of misconfiguration of mine, may you >> help me, please? Just tell me what pieces of config to show (just >> not to flood the whole config) > > I doubt this is related to configuration. > > But you could enable http://wiki2.dovecot.org/Plugins/Lazyexpunge so > that messages won't be lost if they are expunged. The next time a > message is lost, you would most likely find it from the lazy-expunge > namespace. (Then you could write a script that deletes e.g. >1 week > old files nightly.) From andy.robbins at firespring.com Thu Nov 17 23:30:07 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 15:30:07 -0600 Subject: [Dovecot] Dovecot performance issues with many writes Message-ID: We are currently experiencing performance issue with our Dovecot system which we believe is caused by excessive writes to the dovecot files. The confusing thing is that we are seeing more writes than reads on our Dovecot volume when you would assume that most of the IO should be reads from customers checking their mail. We're seeing reads vs. writes similar to the following: # iostat -d 5 -x Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 8.00 28.69 19.63 0.70 94.00 drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 8.00 111.42 94.44 0.74 99.60 We can't figure out why we would be seeing twice as many writes as we are reads. Could this have something to do with the index files? Appreciate any assistance on this issue. From micah at riseup.net Thu Nov 17 23:33:37 2011 From: micah at riseup.net (micah anderson) Date: Thu, 17 Nov 2011 16:33:37 -0500 Subject: [Dovecot] dsync problems In-Reply-To: <1321389804.21919.530.camel@hurina> References: <871ut9i3z1.fsf@algae.riseup.net> <1321389804.21919.530.camel@hurina> Message-ID: <87d3cqe8ni.fsf@algae.riseup.net> On Tue, 15 Nov 2011 22:43:24 +0200, Timo Sirainen wrote: > On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > > When a user renames their username, I am using dsync to copy their mail > > over to the new username's mail location[0]. > > > > Some of the dsyncs are failing with errors that I dont know how to work > > with, for example: > > > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > > dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > > These shouldn't really happen. Something's going internally wrong with > dsync. Can you reproduce this reliably somehow? Well, I dont know if I can do it reliably, but its been happening a lot. One point of information that might be useful is that these users were converted from courier maildir to mdbox, and their courier bits are still around in the source mailbox (I haven't become brave enough to remove them yet). > > > The errors cause a non-zero exit code from dsync, which causes my rename > > script to bail out. What are these errors, and how can I fix them? > > Does a second dsync on error succeed? :) Before I tried it again, I looked at their mailboxes: # doveadm mailbox list -u ): Info: INBOX: only in dest (guid=14bf0409fa08c04e682900007ef44c7a) dsync(): Error: Trying to open a non-listed mailbox with guid=14bf0409fa08c04e682900007ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 14bf0409fa08c04e682900007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=14bf0409fa08c04e682900007ef44c7a and the mailboxes: # doveadm mailbox list -u Trash_084ed82bc59ca54eb53700007ef44c7a Sent Drafts INBOX_094ed82bc59ca54eb53700007ef44c7a I tried it a third time, and... it worked, no error, and now their mailbox list: # doveadm mailbox list -u Trash Sent Drafts INBOX This particular user only has one test email > > > 0. Why use dsync instead of a simple mv operation? This seems to be > > necessary for two corner cases: > > > > 1. dovecot creates the new mailbox automatically when the user logs in > > or receives a mail, so if the user changes their mail and logs in or > > receives an email before the move has been done, then the mailbox is > > created and then a move command will fail. > > > > 2. If there has been new mail created under the new name, we can't just > > simply remove the stuff that is automatically created there and replace > > it with the old things because we could potentially be removing mail > > that has been delivered in the mean time. > > You could temporarily change the permissions for the home directory so > that no new mailboxes/mails could be created during the move (e.g. 0700 > root). The problem is there are a number of users on the system and all the mail is stored under /srv/mailstorage//. So if "foo" wants to change their username to "bar" -- I dont have a deterministic way of determining that "bar" exists yet because mail could be delivered or they could login and dovecot would create it and I can't set /srv/mailstorage/ 0700 root or nobody would be able to receive mail. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Thu Nov 17 23:51:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 23:51:06 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> On 17.11.2011, at 23.30, Andy Robbins wrote: > We can't figure out why we would be seeing twice as many writes as we are > reads. Could this have something to do with the index files? dovecot -n output would be helpful. From andy.robbins at firespring.com Fri Nov 18 00:20:15 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 16:20:15 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> Message-ID: # dovecot -n # 1.2.9: //etc/dovecot.conf # OS: Linux 2.6.18-164.9.1.el5 x86_64 CentOS release 5.4 (Final) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log protocols: imap pop3 ssl_cert_file: /etc/postfix/ssl/smtp1.domain.com.cert ssl_key_file: /etc/postfix/ssl/smtp1.domain.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): //libexec/dovecot/imap-login login_executable(imap): //libexec/dovecot/imap-login login_executable(pop3): //libexec/dovecot/pop3-login login_user: postfix login_greeting: Welcome login_process_per_connection: no login_processes_count: 2 mail_max_userip_connections: 50 mail_uid: 500 mail_gid: 500 mail_location: maildir:~/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /etc/dovecot/last-login-imap.sh mail_executable(imap): /etc/dovecot/last-login-imap.sh mail_executable(pop3): /etc/dovecot/last-login-pop3.sh mail_plugins: quota mail_plugin_dir(default): //lib/dovecot/imap mail_plugin_dir(imap): //lib/dovecot/imap mail_plugin_dir(pop3): //lib/dovecot/pop3 pop3_uidl_format: %f lda: postmaster_address: postmaster at domain.com auth_socket_path: /var/run/dovecot/auth-master log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver.log fsync_disable: no auth default: mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP user: vmail username_translation: %@ verbose: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: maildir:User quota quota_rule: *:storage=0 From tss at iki.fi Fri Nov 18 00:34:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 00:34:33 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> Message-ID: <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> On 18.11.2011, at 0.20, Andy Robbins wrote: > # iostat -d 5 -x > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz > avgqu-sz await svctm %util > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > 8.00 28.69 19.63 0.70 94.00 > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > 8.00 111.42 94.44 0.74 99.60 These devices don't look like NFS, but you have: > mmap_disable: yes > mail_nfs_storage: yes > mail_nfs_index: yes So what filesystem is that? Are they storing only mails, or other stuff too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing things up at least somewhat.. > lda: > fsync_disable: no This is explicitly set for LDA, but it's the default already elsewhere. Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? From janfrode at tanso.net Fri Nov 18 00:46:30 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 17 Nov 2011 23:46:30 +0100 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <20111117224630.GA1315@dibs.tanso.net> On Thu, Nov 17, 2011 at 03:30:07PM -0600, Andy Robbins wrote: > We are currently experiencing performance issue with our Dovecot system > which we believe is caused by excessive writes to the dovecot files. The > confusing thing is that we are seeing more writes than reads on our Dovecot > volume when you would assume that most of the IO should be reads from > customers checking their mail. Have you mounted the fs without noatime options? -jf From andy.robbins at firespring.com Fri Nov 18 00:38:46 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 16:38:46 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Well, the iostat command was run from the NFS server and dovecot was run from the mail server where it is mounted, hence the discrepancy there. On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > # iostat -d 5 -x > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > avgrq-sz > > avgqu-sz await svctm %util > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 28.69 19.63 0.70 94.00 > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 111.42 94.44 0.74 99.60 > > These devices don't look like NFS, but you have: > > > mmap_disable: yes > > mail_nfs_storage: yes > > mail_nfs_index: yes > > So what filesystem is that? Are they storing only mails, or other stuff > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > things up at least somewhat.. > > > lda: > > fsync_disable: no > > This is explicitly set for LDA, but it's the default already elsewhere. > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > From tss at iki.fi Fri Nov 18 00:42:12 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 00:42:12 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Do you have more than one Dovecot server? If only one, you can still disable the mail_nfs_* settings. Also it would be interesting to see nfsstat numbers from the Dovecot server, compared to those iostat numbers.. On 18.11.2011, at 0.38, Andy Robbins wrote: > Well, the iostat command was run from the NFS server and dovecot was run from the mail server where it is mounted, hence the discrepancy there. > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > # iostat -d 5 -x > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz > > avgqu-sz await svctm %util > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 28.69 19.63 0.70 94.00 > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 111.42 94.44 0.74 99.60 > > These devices don't look like NFS, but you have: > > > mmap_disable: yes > > mail_nfs_storage: yes > > mail_nfs_index: yes > > So what filesystem is that? Are they storing only mails, or other stuff too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing things up at least somewhat.. > > > lda: > > fsync_disable: no > > This is explicitly set for LDA, but it's the default already elsewhere. Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > From p.heinlein at heinlein-support.de Fri Nov 18 01:43:01 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Fri, 18 Nov 2011 00:43:01 +0100 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <201111180043.01419.p.heinlein@heinlein-support.de> Am Donnerstag, 17. November 2011, 22:30:07 schrieb Andy Robbins: > We are currently experiencing performance issue with our Dovecot > system which we believe is caused by excessive writes to the dovecot > files. The confusing thing is that we are seeing more writes than > reads on our Dovecot volume when you would assume that most of the > IO should be reads from customers checking their mail. Yes, but your read-requests are answered from your local kernel-buffer / cache and aren't really read from your hard-disk. (At least if you have enough free memory for a cache, that's large enought.) Incoming mails ARE written on the disk, but read later on just from your cache. Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From victormanuelo at gmail.com Fri Nov 18 05:24:53 2011 From: victormanuelo at gmail.com (Victor) Date: Thu, 17 Nov 2011 22:54:53 -0430 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 Message-ID: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> Hi, I'm new in this List, but I have 6 years using Dovecot on my debian from etch,lenny and now squeeze Package: dovecot-imapd Version: 1:1.2.15-4 Tags: squeeze -- System Information: Debian Release: 6.0 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/24 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages dovecot-imapd depends on: ii dovecot-common 1:1.2.15-4 secure mail server that supports m ii libc6 2.11.2-10 Embedded GNU C Library: Shared lib ii libldap-2.4-2 2.4.23-7 OpenLDAP libraries ii libssl0.9.8 0.9.8o-4 SSL shared libraries the quota_imap plugin not working when I use against MailquotaSize Attribute of LDAP users. this setup works perfectly on lenny and dovecot-imapd 1:1.0.15-2.3 but on squeeze dovecot-imad 1:1.2.15-7 doesn't work, the service is up, but The quota does not display. In logs nothing happens the configuration in dovecot.conf below protocol imap { mail_plugins = quota imap_quota } protocol lda { mail_plugins = quota } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } the configuration in dovecot-ldap.conf below uris = ldap://server-ldap.com.ve auth_bind = yes ldap_version = 3 base = dc=dominio,dc=com,dc=uk deref = never scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota=dirsize:storage user_filter = (&(objectClass=posixAccount)(uid=%u)) pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid pass_filter = (&(objectClass=posixAccount)(uid=%u)) This is the same configuration in the 6 last years. So, I would really appreciate if give me a hand which are more than 15000 Users who administered easily taking the quota in the LDAP attribute. and I want to keep the scheme. Thz Victor Onate Usuario Linux From oni-neko at gmx.net Fri Nov 18 15:34:28 2011 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Fri, 18 Nov 2011 14:34:28 +0100 Subject: [Dovecot] seting acl for master user? Message-ID: <20111118133428.63140@gmx.net> Hello! I'm sorry, but i don't really understand it the way it is written in the wiki, so i'm trying my luck here =) i followed the wiki for creating a master user (the non-sql-version) and i can login as another user via this master user. my dovecot version is 1.2.9 on an ubuntu server 10.4.3 lts. what i don't get is the acl-setting, so that the master-user can access all the other users mails and folders. i made the changes in the dovecot.conf as written in the wiki: protocol imap { mail_plugins = acl imap_acl } protocol lda { mail_plugins = sieve acl } plugin { acl = vfile:/etc/dovecot/acls } in my case I only want the master user (called cyrus) to be able to do everything with/in all mailboxes, so I made a file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". i'm pretty sure i'm missing something rather obvious here, but somehow i'm just stuck. any help would be greatly appreciated. thanks in advance =) silvana -- NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zur?ck-Garantie! Jetzt informieren: http://www.gmx.net/de/go/freephone From robert at schetterer.org Fri Nov 18 15:47:29 2011 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 18 Nov 2011 14:47:29 +0100 Subject: [Dovecot] seting acl for master user? In-Reply-To: <20111118133428.63140@gmx.net> References: <20111118133428.63140@gmx.net> Message-ID: <4EC661F1.4000201@schetterer.org> Am 18.11.2011 14:34, schrieb oni-neko at gmx.net: > Hello! > > I'm sorry, but i don't really understand it the way it is written in the wiki, so i'm trying my luck here =) > i followed the wiki for creating a master user (the non-sql-version) and i can login as another user via this master user. > my dovecot version is 1.2.9 on an ubuntu server 10.4.3 lts. > > what i don't get is the acl-setting, so that the master-user can access all the other users mails and folders. > i made the changes in the dovecot.conf as written in the wiki: > protocol imap { > mail_plugins = acl imap_acl > } > protocol lda { > mail_plugins = sieve acl > } > plugin { > acl = vfile:/etc/dovecot/acls > } > > in my case I only want the master user (called cyrus) to be able to do everything with/in all mailboxes, so I made a file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". > > i'm pretty sure i'm missing something rather obvious here, but somehow i'm just stuck. any help would be greatly appreciated. thanks in advance =) > > silvana are you sure that you dont mixed up master users with acl normally you use master user like described here http://wiki.dovecot.org/Authentication/MasterUsers -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dave.mehler at gmail.com Fri Nov 18 16:27:42 2011 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 18 Nov 2011 09:27:42 -0500 Subject: [Dovecot] setting up multiple listeners Message-ID: Hello, I'm using Dovecot v2 and am atempting to get webmail going. I'm trying to set up imap so that port 993 only is listening on the external interface, for both ipv4 and ipv6, but that on localhost only again for ipv4 and ipv6 unencrypted port 143 is active. I've tried various combinations but keep getting an error invalid number. I've googled and not come up with the missing point. Thanks. Dave. From tss at iki.fi Fri Nov 18 19:22:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:22:02 +0200 Subject: [Dovecot] seting acl for master user? In-Reply-To: <20111118133428.63140@gmx.net> References: <20111118133428.63140@gmx.net> Message-ID: <1321636922.11489.46.camel@hurina> On Fri, 2011-11-18 at 14:34 +0100, oni-neko at gmx.net wrote: > what i don't get is the acl-setting, so that the master-user can > access all the other users mails and folders. .. > in my case I only want the master user (called cyrus) to be able to do > everything with/in all mailboxes, so I made a > file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". Unfortunately currently there is no default ACL file. The above would create an ACL file for a mailbox called "default". I'll need to get the default ACLs implemented some day.. But if you don't need ACLs for anything else, you can simply disable the ACL plugin and the master user will have access to everything. If you do need ACLs, there are some kludges you can still do. From tss at iki.fi Fri Nov 18 19:24:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:24:31 +0200 Subject: [Dovecot] setting up multiple listeners In-Reply-To: References: Message-ID: <1321637071.11489.48.camel@hurina> On Fri, 2011-11-18 at 09:27 -0500, David Mehler wrote: > Hello, > > I'm using Dovecot v2 and am atempting to get webmail going. I'm trying > to set up imap so that port 993 only is listening on the external > interface, for both ipv4 and ipv6, but that on localhost only again > for ipv4 and ipv6 unencrypted port 143 is active. I've tried various > combinations but keep getting an error invalid number. I've googled > and not come up with the missing point. A full error message would have been nice, but basically you can do: service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = 1.2.3.4 f00::baa # your external ip addresses } } From geek at uniserve.com Fri Nov 18 19:25:35 2011 From: geek at uniserve.com (Dave Stevens) Date: Fri, 18 Nov 2011 09:25:35 -0800 Subject: [Dovecot] CCing outbound IMAP mail? Message-ID: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> For some users on my site I'd like to be able to CC a particular user on all outgoing IMAP mails. Incoming I can do but don't see how to do outgoing. Ideas? Dave -- It is told that such are the aerodynamics and wing loading of the bumblebee that, in principle, it cannot fly...if all this be true...life among bumblebees must bear a remarkable resemblance to life in the United States. -- John Kenneth Galbraith, in American Capitalism: The Concept of Countervailing Power From tss at iki.fi Fri Nov 18 19:27:34 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:27:34 +0200 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 In-Reply-To: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> References: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> Message-ID: <1321637255.11489.50.camel@hurina> On Thu, 2011-11-17 at 22:54 -0430, Victor wrote: > the quota_imap plugin not working when I use against MailquotaSize Attribute of LDAP users. > this setup works perfectly on lenny and dovecot-imapd 1:1.0.15-2.3 but on squeeze dovecot-imad > 1:1.2.15-7 doesn't work, the service is up, but The quota does not display. In logs nothing happens The quota configuration changed between v1.0 and v1.1. > user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota=dirsize:storage > user_filter = (&(objectClass=posixAccount)(uid=%u)) maildirQuotaSize=quota_rule=*:storage=%$ and to dovecot.conf add: plugin { quota = dirsize } From tss at iki.fi Fri Nov 18 19:30:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:30:08 +0200 Subject: [Dovecot] CCing outbound IMAP mail? In-Reply-To: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> References: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> Message-ID: <1321637408.11489.53.camel@hurina> On Fri, 2011-11-18 at 09:25 -0800, Dave Stevens wrote: > For some users on my site I'd like to be able to CC a particular user > on all outgoing IMAP mails. Incoming I can do but don't see how to do > outgoing. Ideas? Typically the users would be sending the outgoing mails via your SMTP server and you should be able to do it the same way in MTA. If your users aren't sending mails via your SMTP server, then it depends on if they are saving the mails to the "Sent" mailbox on IMAP server. That "Sent" mailbox could be named different things, and in any case there's really no simple way to trigger sending a mail when a mail is being saved via IMAP.. From robert at schetterer.org Fri Nov 18 19:33:03 2011 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 18 Nov 2011 18:33:03 +0100 Subject: [Dovecot] CCing outbound IMAP mail? In-Reply-To: <1321637408.11489.53.camel@hurina> References: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> <1321637408.11489.53.camel@hurina> Message-ID: <4EC696CF.3090807@schetterer.org> Am 18.11.2011 18:30, schrieb Timo Sirainen: > On Fri, 2011-11-18 at 09:25 -0800, Dave Stevens wrote: >> For some users on my site I'd like to be able to CC a particular user >> on all outgoing IMAP mails. Incoming I can do but don't see how to do >> outgoing. Ideas? > > Typically the users would be sending the outgoing mails via your SMTP > server and you should be able to do it the same way in MTA. > > If your users aren't sending mails via your SMTP server, then it depends > on if they are saving the mails to the "Sent" mailbox on IMAP server. > That "Sent" mailbox could be named different things, and in any case > there's really no simple way to trigger sending a mail when a mail is > being saved via IMAP.. > > perhaps not the right thing in postfix there is sender_bcc_maps (default: empty) Optional BCC (blind carbon-copy) address lookup tables, indexed by sender address. The BCC address (multiple results are not supported) is added when mail enters from outside of Postfix. This feature is available in Postfix 2.1 and later. The table search order is as follows: Look up the "user+extension at domain.tld" address including the optional address extension. Look up the "user at domain.tld" address without the optional address extension. Look up the "user+extension" address local part when the sender domain equals $myorigin, $mydestination, $inet_interfaces or $proxy_interfaces. Look up the "user" address local part when the sender domain equals $myorigin, $mydestination, $inet_interfaces or $proxy_interfaces. Look up the "@domain.tld" part. Specify the types and names of databases to use. After change, run "postmap /etc/postfix/sender_bcc". Note: if mail to the BCC address bounces it will be returned to the sender. -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stephan at rename-it.nl Sat Nov 19 18:34:38 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 19 Nov 2011 17:34:38 +0100 Subject: [Dovecot] Released Pigeonhole v0.2.5 for Dovecot v2.0.16 Message-ID: <4EC7DA9E.6010601@rename-it.nl> Hello Dovecot users, Before I move active development of Pigeonhole to Dovecot v2.1, I first release all the pending bug fixes and small improvements. Changelog v0.2.5: + Sieve vacation extension: made discard message for implicit delivery more verbose - The sieve-test tool: mixed up original and final envelope recipient in implementation of command line arguments. - Sieve vacation extension: resolved FIXME regarding the use of variables in the :handle argument. Variables are now handled correctly. - Sieve body extension: fixed handling of :content "message/rfc822". This now yields the headers of the embedded message as required by the specification. Handling of :content "multipart" remains to be fixed. - LDA Sieve plugin: fixed problem with recipient_delimiter configuration. Now it falls back to global recipient_delimiter setting if plugin/recipient_delimiter is not set. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.5.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.5.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From metro_domain_admin at fastmail.fm Sat Nov 19 21:05:13 2011 From: metro_domain_admin at fastmail.fm (metro_domain_admin at fastmail.fm) Date: Sat, 19 Nov 2011 14:05:13 -0500 Subject: [Dovecot] issues with 2.1b1 on OS X Message-ID: <1321729513.18027.140661001080053@webmail.messagingengine.com> Two issues came up for me when building 2.1beta1 on OS X (10.5.8 on PPC). The first was already mentioned in an earlier thread by a BSD user (http://dovecot.org/list/dovecot/2011-September/061201.html). I just wanted to point out that the error is still occurring in spite of a new test for GNU ld in the configure script. I needed to remove "-no-undefined" specifically: #NOPLUGIN_LDFLAGS="-no-undefined" if test "$with_gnu_ld" = yes; then NOPLUGIN_LDFLAGS="$NOPLUGIN_LDFLAGS -Wl,--as-needed" fi More important, Dovecot aborts on launch with the following error: Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument I configured using the same args as I had used successfully on 2.0.15: --with-gssapi --with-ldap --with-sql --prefix=/opt/dovecot-2.X.X. I also tried an exact copy of my known-good config from 2.0.15, to no avail. Thanks, Tony From tss at iki.fi Sat Nov 19 22:34:36 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 19 Nov 2011 22:34:36 +0200 Subject: [Dovecot] issues with 2.1b1 on OS X In-Reply-To: <1321729513.18027.140661001080053@webmail.messagingengine.com> References: <1321729513.18027.140661001080053@webmail.messagingengine.com> Message-ID: On 19.11.2011, at 21.05, metro_domain_admin at fastmail.fm wrote: > Two issues came up for me when building 2.1beta1 on OS X (10.5.8 on > PPC). The first was already mentioned in an earlier thread by a BSD user > (http://dovecot.org/list/dovecot/2011-September/061201.html). I just > wanted to point out that the error is still occurring in spite of a new > test for GNU ld in the configure script. I needed to remove > "-no-undefined" specifically: This error? Undefined symbols: "_environ", referenced from: _env_clean in env-util.o http://hg.dovecot.org/dovecot-2.1/rev/b2c9298e981 and http://hg.dovecot.org/dovecot-2.1/rev/9d022d3fba42 fixes it. > More important, Dovecot aborts on launch with the following error: > > Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, > READ, 16) failed: Invalid argument > I configured using the same args as I had used successfully on 2.0.15: > --with-gssapi --with-ldap --with-sql --prefix=/opt/dovecot-2.X.X. I also > tried an exact copy of my known-good config from 2.0.15, to no avail. Works in 10.6. What is the gdb backtrace of the crash with the attached patch? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 577 bytes Desc: not available URL: From davide at cavion.eu Sun Nov 20 11:08:20 2011 From: davide at cavion.eu (Davide Cavion) Date: Sun, 20 Nov 2011 10:08:20 +0100 Subject: [Dovecot] Using google 2 step authentication Message-ID: Google authenticator has a PAM interface: http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions Have someone tried implementing it onto Dovecot? It would be really great if you consider your emails important! From dovecot.user at seibercom.net Sun Nov 20 18:09:22 2011 From: dovecot.user at seibercom.net (Jerry) Date: Sun, 20 Nov 2011 11:09:22 -0500 Subject: [Dovecot] Using google 2 step authentication In-Reply-To: References: Message-ID: <20111120110922.3488d46d@scorpio> On Sun, 20 Nov 2011 10:08:20 +0100 Davide Cavion articulated: > Google authenticator has a PAM interface: > > http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions > > Have someone tried implementing it onto Dovecot? It would be really > great if you consider your emails important! Isn't this an oxymoron; ie "Security" & "Google"? -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From davide at cavion.eu Sun Nov 20 18:18:01 2011 From: davide at cavion.eu (Davide Cavion) Date: Sun, 20 Nov 2011 17:18:01 +0100 Subject: [Dovecot] Using google 2 step authentication In-Reply-To: <20111120110922.3488d46d@scorpio> References: <20111120110922.3488d46d@scorpio> Message-ID: <924E1EE7-6487-443E-AE88-FBD3AF7AA62E@cavion.eu> Haha, you are right =) However this is open source and don't speak with google's servers ;) Il giorno 20/nov/2011, alle ore 17:09, Jerry ha scritto: > On Sun, 20 Nov 2011 10:08:20 +0100 > Davide Cavion articulated: > >> Google authenticator has a PAM interface: >> >> http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions >> >> Have someone tried implementing it onto Dovecot? It would be really >> great if you consider your emails important! > > Isn't this an oxymoron; ie "Security" & "Google"? > > -- > Jerry ? > Dovecot.user at seibercom.net > > Disclaimer: off-list followups get on-list replies or get ignored. > Please do not ignore the Reply-To header. > __________________________________________________________________ > From victormanuelo at gmail.com Mon Nov 21 02:05:27 2011 From: victormanuelo at gmail.com (Victor) Date: Sun, 20 Nov 2011 19:35:27 -0430 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 Message-ID: <9BD55AF0-B054-4AD3-AC94-E6490E1B495D@gmail.com> Thank a lot, your were completely in the truth Victor Onate From tss at iki.fi Mon Nov 21 04:06:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 04:06:18 +0200 Subject: [Dovecot] issues with 2.1b1 on OS X In-Reply-To: <1321729513.18027.140661001080053@webmail.messagingengine.com> References: <1321729513.18027.140661001080053@webmail.messagingengine.com> Message-ID: <1321841181.24607.23.camel@hurina> On Sat, 2011-11-19 at 14:05 -0500, metro_domain_admin at fastmail.fm wrote: > More important, Dovecot aborts on launch with the following error: > > Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, > READ, 16) failed: Invalid argument This probably helps: service stats { fifo_listener stats-mail { mode = 0 } } Since this fails only in OSX 10.5 (not 10.6), I don't know if I'll bother adding any special code to work around it. In any case it means that kevent() can't be used on a FIFO, and that means you can't use v2.1's stats tracking feature. Or possibly you could try if it works with configure --with-ioloop=poll or select instead of kqueue. From preen at informatik.uni-freiburg.de Mon Nov 21 11:18:02 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Mon, 21 Nov 2011 10:18:02 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4DACA6E3.5020301@bio.umass.edu> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> Message-ID: <4ECA174A.1020909@informatik.uni-freiburg.de> Hello, after serveral tests (and reading a lot of howto's) I'm now fairly convinced that the Solaris HW-crypto engine is not automatically used. Even when dovecot ist compiled with the OpenSSL version provided by Solaris. Currently I have only a T1-CPU available for testing (Sun Fire T2000) and after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with ENGINE *e; ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); ENGINE_set_default_ciphers(e); in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() the crypto device gets used. I'm sure that this is not the whole story since this only seems to affect the IMAP login. One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in the various articles). Is there any chance that Dovecot gets updated/patched for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. Regards. Martin Chris Hoogendyk wrote: > > On 4/18/11 6:30 AM, Martin Preen wrote: >> Hello, >> I tried to find out how about to use the hardware crypto engines under >> Solaris (Sun Fire T2000). It seems, that its not just a compilation >> issue: >> >>> For operations that are to be offloaded, it is necessary to restrict >>> use to subset >> > of OpenSSL functions (the EVP_ functions) and explicitly indicate >> the use of the PKCS11 >> > engine; something like the following works for bulk ciphers (the >> process for RSA is similar): >>> >>> ENGINE *e; >>> ENGINE_load_builtin_engines(); >>> e = ENGINE_by_id("pkcs11"); >>> ENGINE_set_default_ciphers(e); >>> EVP_CIPHER_CTX_init (&ctx); >>> EVP_EncryptInit (&ctx, EVP_des_cbc (), key, iv); >>> EVP_EncryptUpdate (.....); >> >> Since I'm not familiar with SSL programming, I wonder if it's possible to >> modify the dovecot 1.2.x source code. Maybe someone has already tried >> this >> or can give me hint. > > You don't want to do that. > > The objective is for the ssl engine to be low level and basically > invisible to layers of application above that use it. So, build openssl > properly and it just works for everything else. > > I'm not at work today, so I don't have reference to my notes. Ah, just > found an email I sent to my colleagues -- copied it below. --------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/~preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6337 bytes Desc: S/MIME Cryptographic Signature URL: From hummel at pasteur.fr Mon Nov 21 12:19:42 2011 From: hummel at pasteur.fr (Thomas Hummel) Date: Mon, 21 Nov 2011 11:19:42 +0100 Subject: [Dovecot] Dovecot Log analysis Message-ID: <20111121101941.GA25724@parmesan.sis.pasteur.fr> Hello, Before writing them myself, I'd like to figure out if some tools exist (anvil ? ...) and how to use them to extract from dovecot logs things like who is doing what and how hard or how fast in order to do some rate limiting. For instance, I'd like to be able to identify, slow or block users who . read or write to much / slice of time . login/logout too often / slice of time Note : I'm still using the 1.2.x branch. Thanks -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From pareshchalke at yahoo.com Mon Nov 21 08:52:44 2011 From: pareshchalke at yahoo.com (Paresh Chalke) Date: Mon, 21 Nov 2011 06:52:44 +0000 (UTC) Subject: [Dovecot] uid=error Message-ID: Hi Timo, We are facing this issue on our mail server 1. Two email id's are mapped to a same account 2. If I send a mail to both the account simultaneously i get below error "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" 3. Due to this the mail is delivered only to 1 email id. 4. I am loosing all my mails. Kindly help in the same ASAP. Regards, Paresh Chalke From wgrcunha at gmail.com Mon Nov 21 16:06:16 2011 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 21 Nov 2011 12:06:16 -0200 Subject: [Dovecot] Iterate over domain Message-ID: Hello, We need to use doveadm commands on all accounts of a single domain, for now i need to query the database using scripts and for each one user, run doveadm. I can use iterate but, only for all server accounts, have any way to use doveadm iterate over a single domain? Thanks From duihi77 at gmail.com Mon Nov 21 16:20:54 2011 From: duihi77 at gmail.com (Duane Hill) Date: Mon, 21 Nov 2011 14:20:54 +0000 Subject: [Dovecot] Iterate over domain In-Reply-To: References: Message-ID: <1178881680.20111121142054@gmail.com> On Monday, November 21, 2011 at 14:06:16 UTC, wgrcunha at gmail.com confabulated: > Hello, > We need to use doveadm commands on all accounts of a single domain, for now > i need to query the database using scripts and for each one user, run > doveadm. I can use iterate but, only for all server accounts, have any way > to use doveadm iterate over a single domain? Instead of using the -A option for all, use: -u '*@thedomain.tld' -- There are 10 kinds of people in the world... Those who understand binary, and those who don't. From jgiles at cybermesa.com Mon Nov 21 17:44:13 2011 From: jgiles at cybermesa.com (jgiles at cybermesa.com) Date: Mon, 21 Nov 2011 08:44:13 -0700 (MST) Subject: [Dovecot] Question about imaps Message-ID: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> Hi List! Had a questions about imap vs. imaps. I read the ssl article at http://wiki2.dovecot.org/SSL and from what I am reading, imaps is no longer needed as imap with initiate SSL/TLS on its own if STARTTLS is started and I have disable_plaintext_auth=yes and ssl=required settings. I do have SSL up and working, and can connect using imap (143) to my server without issue, but was not able to connect using imaps (993) using Kmail as my client. I Was getting a client error stating that the authentication failed using my password. However, there was nothing in the mail.err or mail.log file even when I enable debugging. If I specify imaps as my only protocol, then dovecot fails to even start without any error loging at all. As I stated, imap works just fine. So, is imaps deprecated as mentioned in the article and imaps no longer needed? I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. This server is simply an imap serer and not a MTA (No postfix or sendmail installed.) Thanks! Joe From jgiles at cybermesa.com Mon Nov 21 18:40:02 2011 From: jgiles at cybermesa.com (jgiles at cybermesa.com) Date: Mon, 21 Nov 2011 09:40:02 -0700 (MST) Subject: [Dovecot] Question about imaps In-Reply-To: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> Message-ID: <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > Hi List! > > Had a questions about imap vs. imaps. > > I read the ssl article at http://wiki2.dovecot.org/SSL and from what I am > reading, imaps is no longer needed as imap with initiate SSL/TLS on its > own if STARTTLS is started and I have disable_plaintext_auth=yes and > ssl=required settings. > > I do have SSL up and working, and can connect using imap (143) to my > server without issue, but was not able to connect using imaps (993) using > Kmail as my client. > > I Was getting a client error stating that the authentication failed using > my password. However, there was nothing in the mail.err or mail.log file > even when I enable debugging. > > If I specify imaps as my only protocol, then dovecot fails to even start > without any error loging at all. > > As I stated, imap works just fine. > > So, is imaps deprecated as mentioned in the article and imaps no longer > needed? > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > This server is simply an imap serer and not a MTA (No postfix or sendmail > installed.) > > Thanks! > Joe > > > > LOL, replying to my own e-mail... I just discovered that the problem is most likely in my Kmail 2 client. Tried to connect using the following: openssl s_client -connect mymailserver.com:993 I was able to connect, log in, and get information about my mail folders. Will try Thunderbird when I get home from work. Kmail 2 is still a WIP I guess. That said, I am still reading that imap with starttls is the way to go, so might be a moot issue. Thanks! Joe From tss at iki.fi Mon Nov 21 20:06:19 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 20:06:19 +0200 Subject: [Dovecot] uid=error In-Reply-To: References: Message-ID: On 21.11.2011, at 8.52, Paresh Chalke wrote: > We are facing this issue on our mail server > 1. Two email id's are mapped to a same account > 2. If I send a mail to both the account simultaneously i get below error > > "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, > uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. From dovecot.user at seibercom.net Mon Nov 21 20:20:55 2011 From: dovecot.user at seibercom.net (Jerry) Date: Mon, 21 Nov 2011 13:20:55 -0500 Subject: [Dovecot] Question about imaps In-Reply-To: <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> Message-ID: <20111121132055.15af04cf@scorpio> On Mon, 21 Nov 2011 09:40:02 -0700 (MST) jgiles at cybermesa.com articulated: > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > > Hi List! > > > > Had a questions about imap vs. imaps. > > > > I read the ssl article at http://wiki2.dovecot.org/SSL and from > > what I am reading, imaps is no longer needed as imap with initiate > > SSL/TLS on its own if STARTTLS is started and I have > > disable_plaintext_auth=yes and ssl=required settings. > > > > I do have SSL up and working, and can connect using imap (143) to my > > server without issue, but was not able to connect using imaps (993) > > using Kmail as my client. > > > > I Was getting a client error stating that the authentication failed > > using my password. However, there was nothing in the mail.err or > > mail.log file even when I enable debugging. > > > > If I specify imaps as my only protocol, then dovecot fails to even > > start without any error loging at all. > > > > As I stated, imap works just fine. > > > > So, is imaps deprecated as mentioned in the article and imaps no > > longer needed? > > > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > > This server is simply an imap serer and not a MTA (No postfix or > > sendmail installed.) > > > > Thanks! > > Joe > > LOL, replying to my own e-mail... > > I just discovered that the problem is most likely in my Kmail 2 > client. > > Tried to connect using the following: > > openssl s_client -connect mymailserver.com:993 > > I was able to connect, log in, and get information about my mail > folders. > > Will try Thunderbird when I get home from work. Kmail 2 is still a > WIP I guess. > > That said, I am still reading that imap with starttls is the way to > go, so might be a moot issue. > > Thanks! > Joe Try this: (substitute your actual host for "remote.host") openssl s_client -connect remote.host:143 -starttls imap See if you get a connection. If so, then the server offers STARTTLS. -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From andy.robbins at firespring.com Mon Nov 21 20:34:45 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Mon, 21 Nov 2011 12:34:45 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Actually, some of the information I provided previously was incorrect. We have four mail servers in our cluster which each have the NFS share mounted. These four servers handle all SMTP and sending of mail. However, it was thought that we could boost performance by simply running POP3 and IMAP directly from the NFS server, since nothing besides Dovecot is currently running there. So, here is the dovecot -n output from our IMAP/POP3 server where the file system is local storage and is where we believe the performance issues are originating. # dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 2304). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.32-24-server x86_64 Ubuntu 10.04.1 LTS protocols: imap pop3 ssl_cert_file: /etc/dovecot/smtp1.domain.com.cert ssl_key_file: /etc/dovecot/smtp1.domain.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Welcome login_process_per_connection: no login_processes_count: 2 max_mail_processes: 2048 mail_max_userip_connections: 50 mail_uid: 500 mail_gid: 500 mail_location: maildir:~/Maildir fsync_disable: yes maildir_very_dirty_syncs: yes mbox_write_locks: fcntl dotlock mail_executable(default): /etc/dovecot/last-login-imap.sh mail_executable(imap): /etc/dovecot/last-login-imap.sh mail_executable(pop3): /etc/dovecot/last-login-pop3.sh mail_plugins: quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format: %f auth default: mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP username_translation: %@ verbose: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf plugin: quota: maildir:User quota quota_rule: *:storage=0 So you can see that we have already tried a number of performance modifications such as setting fsync_disable: yes with little or no effect. If I run nfsstat on the email servers where the volumes are mounted and only running SMTP I get: # nfsstat Client rpc stats: calls retrans authrefrsh 51762532 72668 0 Client nfs v3: null getattr setattr lookup access readlink 0 0% 9639844 18% 3846547 7% 10495207 20% 11930633 23% 8 0% read write create mkdir symlink mknod 7011062 13% 5372278 10% 1593131 3% 95 0% 0 0% 0 0% remove rmdir rename link readdir readdirplus 830662 1% 0 0% 865484 1% 2576 0% 110845 0% 24963 0% fsstat fsinfo pathconf commit 2074 0% 8 0% 0 0% 37113 0% My apologies for the somewhat garbled output, there, but I it seems to me that we're not witnessing the same higher IO writes as compared to reads. Thanks for your help so far. Hopefully, this sheds more light on your previous questions. On Thu, Nov 17, 2011 at 4:42 PM, Timo Sirainen wrote: > Do you have more than one Dovecot server? If only one, you can still > disable the mail_nfs_* settings. Also it would be interesting to see > nfsstat numbers from the Dovecot server, compared to those iostat numbers.. > > On 18.11.2011, at 0.38, Andy Robbins wrote: > > > Well, the iostat command was run from the NFS server and dovecot was run > from the mail server where it is mounted, hence the discrepancy there. > > > > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > > > # iostat -d 5 -x > > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > avgrq-sz > > > avgqu-sz await svctm %util > > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > 8.00 28.69 19.63 0.70 94.00 > > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > 8.00 111.42 94.44 0.74 99.60 > > > > These devices don't look like NFS, but you have: > > > > > mmap_disable: yes > > > mail_nfs_storage: yes > > > mail_nfs_index: yes > > > > So what filesystem is that? Are they storing only mails, or other stuff > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > things up at least somewhat.. > > > > > lda: > > > fsync_disable: no > > > > This is explicitly set for LDA, but it's the default already elsewhere. > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > > > > > > From jgiles at cybermesa.com Mon Nov 21 21:17:57 2011 From: jgiles at cybermesa.com (Joe Giles) Date: Mon, 21 Nov 2011 12:17:57 -0700 Subject: [Dovecot] Question about imaps In-Reply-To: <20111121132055.15af04cf@scorpio> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> <20111121132055.15af04cf@scorpio> Message-ID: <21698362.YkgfMOa32X@linuxbox> On Monday, November 21, 2011 01:20:55 PM Jerry wrote: > On Mon, 21 Nov 2011 09:40:02 -0700 (MST) > jgiles at cybermesa.com articulated: > > > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > > > Hi List! > > > > > > Had a questions about imap vs. imaps. > > > > > > I read the ssl article at http://wiki2.dovecot.org/SSL and from > > > what I am reading, imaps is no longer needed as imap with initiate > > > SSL/TLS on its own if STARTTLS is started and I have > > > disable_plaintext_auth=yes and ssl=required settings. > > > > > > I do have SSL up and working, and can connect using imap (143) to my > > > server without issue, but was not able to connect using imaps (993) > > > using Kmail as my client. > > > > > > I Was getting a client error stating that the authentication failed > > > using my password. However, there was nothing in the mail.err or > > > mail.log file even when I enable debugging. > > > > > > If I specify imaps as my only protocol, then dovecot fails to even > > > start without any error loging at all. > > > > > > As I stated, imap works just fine. > > > > > > So, is imaps deprecated as mentioned in the article and imaps no > > > longer needed? > > > > > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > > > This server is simply an imap serer and not a MTA (No postfix or > > > sendmail installed.) > > > > > > Thanks! > > > Joe > > > > LOL, replying to my own e-mail... > > > > I just discovered that the problem is most likely in my Kmail 2 > > client. > > > > Tried to connect using the following: > > > > openssl s_client -connect mymailserver.com:993 > > > > I was able to connect, log in, and get information about my mail > > folders. > > > > Will try Thunderbird when I get home from work. Kmail 2 is still a > > WIP I guess. > > > > That said, I am still reading that imap with starttls is the way to > > go, so might be a moot issue. > > > > Thanks! > > Joe > > Try this: (substitute your actual host for "remote.host") > > openssl s_client -connect remote.host:143 -starttls imap > > See if you get a connection. If so, then the server offers STARTTLS. > > Hi, Jerry, Thanks for the information. Yes, I ran that command and got my cert and other information proving to me that TLS is working. I was also able to log in and get mailbox information. Thanks again! Joe From victormanuelo at gmail.com Mon Nov 21 21:42:56 2011 From: victormanuelo at gmail.com (=?UTF-8?Q?Victor_O=C3=B1ate?=) Date: Mon, 21 Nov 2011 15:12:56 -0430 Subject: [Dovecot] Question about imaps In-Reply-To: <21698362.YkgfMOa32X@linuxbox> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> <20111121132055.15af04cf@scorpio> <21698362.YkgfMOa32X@linuxbox> Message-ID: Try to conect from shell # telnet server 993 2011/11/21 Joe Giles : > On Monday, November 21, 2011 01:20:55 PM Jerry wrote: >> On Mon, 21 Nov 2011 09:40:02 -0700 (MST) >> jgiles at cybermesa.com articulated: >> >> > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: >> > > Hi List! >> > > >> > > Had a questions about imap vs. imaps. >> > > >> > > I read the ssl article at http://wiki2.dovecot.org/SSL and from >> > > what I am reading, imaps is no longer needed as imap with initiate >> > > SSL/TLS on its own if STARTTLS is started and I have >> > > disable_plaintext_auth=yes and ssl=required settings. >> > > >> > > I do have SSL up and working, and can connect using imap (143) to my >> > > server without issue, but was not able to connect using imaps (993) >> > > using Kmail as my client. >> > > >> > > I Was getting a client error stating that the authentication failed >> > > using my password. However, there was nothing in the mail.err or >> > > mail.log file even when I enable debugging. >> > > >> > > If I specify imaps as my only protocol, then dovecot fails to even >> > > start without any error loging at all. >> > > >> > > As I stated, imap works just fine. >> > > >> > > So, is imaps deprecated as mentioned in the article and imaps no >> > > longer needed? >> > > >> > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. >> > > This server is simply an imap serer and not a MTA (No postfix or >> > > sendmail installed.) >> > > >> > > Thanks! >> > > Joe >> > >> > LOL, replying to my own e-mail... >> > >> > I just discovered that the problem is most likely in my Kmail 2 >> > client. >> > >> > Tried to connect using the following: >> > >> > openssl s_client -connect mymailserver.com:993 >> > >> > I was able to connect, log in, and get information about my mail >> > folders. >> > >> > Will try Thunderbird when I get home from work. Kmail 2 is still a >> > WIP I guess. >> > >> > That said, I am still reading that imap with starttls is the way to >> > go, so might be a moot issue. >> > >> > Thanks! >> > Joe >> >> Try this: (substitute your actual host for "remote.host") >> >> openssl s_client -connect remote.host:143 -starttls imap >> >> See if you get a connection. If so, then the server offers STARTTLS. >> >> > > Hi, Jerry, > > Thanks for the information. > > Yes, I ran that command and got my cert and other information proving to me that TLS is working. I was also able to log in and get mailbox information. > > Thanks again! > Joe > From stan at hardwarefreak.com Mon Nov 21 21:45:54 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 21 Nov 2011 13:45:54 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: <4ECAAA72.4040802@hardwarefreak.com> On 11/21/2011 12:34 PM, Andy Robbins wrote: > Warning: fd limit 1024 is lower than what Dovecot can use under full load > (more than 2304). Either grow the limit or change login_max_processes_count > and max_mail_processes settings Why haven't you increased the file descriptor limit? -- Stan From achekalin at lazurit.com Mon Nov 21 22:02:39 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 21 Nov 2011 23:02:39 +0300 Subject: [Dovecot] Unique message IDs? Message-ID: <4ECAAE5F.1040507@lazurit.com> Hello, I'm working with indexing my mail box. What I need is to have index in which mailing specified header string is located. I work with the messages via IMAP. I know that sounds like FTS can help me but no, I don't want to index whole message nor I want to patch FTS source to make it to index headers only. I need a way to identify message across my mail box. The idea is that I can move messages between IMAP folders and the index still should be able to identify it. What I want to know, will UID strings be unique across all my messages no matter which IMAP folder it belongs now to, or the UID can change if I move message? If I remove dovecot-uidlist from the folder will UIDs of messages in the folder be changed? What happened if I put a message to the given IMAP folder (and dovecot-uidlist will record UID for it), then shut down dovecot, and (given maildir storage model) move the message's file info another folder in the same mail box - after I start dovecot, will the UID for the message be the same or not? Thank you in advance, Alexander Chekalin From robert at schetterer.org Mon Nov 21 22:14:53 2011 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 21 Nov 2011 21:14:53 +0100 Subject: [Dovecot] uid=error In-Reply-To: References: Message-ID: <4ECAB13D.3050501@schetterer.org> Am 21.11.2011 19:06, schrieb Timo Sirainen: > On 21.11.2011, at 8.52, Paresh Chalke wrote: > >> We are facing this issue on our mail server >> 1. Two email id's are mapped to a same account >> 2. If I send a mail to both the account simultaneously i get below error >> >> "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, >> uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" > > This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. > Hi Timo, just for info i also have sporadic error like this Nov 21 20:36:10 mail01 dovecot: lmtp(10398, user at user.de): save: box=INBOX, uid=error, msgid=<20111121193610.69B4D200A2F at server.de>, size=6356 as mails get delivered i dont care, i see this happen sometimes during rsync backuptimes but i have not really an idea what exact cause this -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Mon Nov 21 22:22:58 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 22:22:58 +0200 Subject: [Dovecot] uid=error In-Reply-To: <4ECAB13D.3050501@schetterer.org> References: <4ECAB13D.3050501@schetterer.org> Message-ID: On 21.11.2011, at 22.14, Robert Schetterer wrote: >> This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. >> > Hi Timo, > just for info > i also have sporadic error like this > Nov 21 20:36:10 mail01 dovecot: lmtp(10398, user at user.de): save: > box=INBOX, uid=error, msgid=<20111121193610.69B4D200A2F at server.de>, > size=6356 > > as mails get delivered i dont care, i see > this happen sometimes during rsync backuptimes > but i have not really an idea what exact cause this Oh, I was wrong. The uid=error simply means that the UID is unknown for some reason. Possibly an internal bug.. The mail got saved just fine. Some backends like saving to a virtual mailbox will also give uid=error. So anyway, back to the original question: This log message doesn't indicate any error. The problem of some mail not being saved is somewhere else, possibly in MTA configuration? From tss at iki.fi Mon Nov 21 22:27:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 22:27:59 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECAAE5F.1040507@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> Message-ID: On 21.11.2011, at 22.02, Alexander Chekalin wrote: > I need a way to identify message across my mail box. The idea is that I can move messages between IMAP folders and the index still should be able to identify it. Message GUIDs are pretty good for that. > What I want to know, will UID strings be unique across all my messages no matter which IMAP folder it belongs now to, or the UID can change if I move message? If I remove dovecot-uidlist from the folder will UIDs of messages in the folder be changed? What happened if I put a message to the given IMAP folder (and dovecot-uidlist will record UID for it), then shut down dovecot, and (given maildir storage model) move the message's file info another folder in the same mail box - after I start dovecot, will the UID for the message be the same or not? With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). Assuming you're using Dovecot v2.x, when mail is copied to another mailbox its filename is preserved. So deleting dovecot* files won't lose the GUID. The only problem is that if you copy the same mail twice to another mailbox, it can't of course have the same filename twice, so Dovecot will assign it a new filename. But in a new enough version (probably v2.0.something) it still preserves the GUID by writing it to dovecot-uidlist file. In this situation if you delete the uidlist, the GUID changes to its filename. From tss at iki.fi Mon Nov 21 23:41:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:41:57 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: <1321911718.24607.51.camel@hurina> I don't see anything obviously bad in your setup. And since you said you're running Dovecot in the IMAP server, perhaps it's not even Dovecot at all that is causing the writes?.. Here are anyway some thoughts: * Log files? Make sure syslog isn't fsyncing each Dovecot log line to disk (or just set log_path = /var/log/dovecot.log and it definitely won't fsync) * Try if disabling index files lowers writes (I don't see why it would make a big difference): mail_location = maildir:~/Maildir:INDEX=MEMORY * Try if temporarily disabling Maildir++ quota makes any difference. There aren't really any other potential writers in Dovecot.. On Mon, 2011-11-21 at 12:34 -0600, Andy Robbins wrote: > Actually, some of the information I provided previously was incorrect. We > have four mail servers in our cluster which each have the NFS share > mounted. These four servers handle all SMTP and sending of mail. However, > it was thought that we could boost performance by simply running POP3 and > IMAP directly from the NFS server, since nothing besides Dovecot is > currently running there. So, here is the dovecot -n output from our > IMAP/POP3 server where the file system is local storage and is where we > believe the performance issues are originating. > > # dovecot -n > # 1.2.9: /etc/dovecot/dovecot.conf > Warning: fd limit 1024 is lower than what Dovecot can use under full load > (more than 2304). Either grow the limit or change login_max_processes_count > and max_mail_processes settings > # OS: Linux 2.6.32-24-server x86_64 Ubuntu 10.04.1 LTS > protocols: imap pop3 > ssl_cert_file: /etc/dovecot/smtp1.domain.com.cert > ssl_key_file: /etc/dovecot/smtp1.domain.com.key > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > login_greeting: Welcome > login_process_per_connection: no > login_processes_count: 2 > max_mail_processes: 2048 > mail_max_userip_connections: 50 > mail_uid: 500 > mail_gid: 500 > mail_location: maildir:~/Maildir > fsync_disable: yes > maildir_very_dirty_syncs: yes > mbox_write_locks: fcntl dotlock > mail_executable(default): /etc/dovecot/last-login-imap.sh > mail_executable(imap): /etc/dovecot/last-login-imap.sh > mail_executable(pop3): /etc/dovecot/last-login-pop3.sh > mail_plugins: quota > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > pop3_uidl_format: %f > auth default: > mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP > username_translation: %@ > verbose: yes > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > plugin: > quota: maildir:User quota > quota_rule: *:storage=0 > > So you can see that we have already tried a number of performance > modifications such as setting fsync_disable: yes with little or no effect. > If I run nfsstat on the email servers where the volumes are mounted and > only running SMTP I get: > > # nfsstat > Client rpc stats: > calls retrans authrefrsh > 51762532 72668 0 > > Client nfs v3: > null getattr setattr lookup access > readlink > 0 0% 9639844 18% 3846547 7% 10495207 20% 11930633 23% 8 > 0% > read write create mkdir symlink > mknod > 7011062 13% 5372278 10% 1593131 3% 95 0% 0 0% 0 > 0% > remove rmdir rename link readdir > readdirplus > 830662 1% 0 0% 865484 1% 2576 0% 110845 0% 24963 > 0% > fsstat fsinfo pathconf commit > 2074 0% 8 0% 0 0% 37113 0% > > My apologies for the somewhat garbled output, there, but I it seems to me > that we're not witnessing the same higher IO writes as compared to reads. > > Thanks for your help so far. Hopefully, this sheds more light on your > previous questions. > > > > On Thu, Nov 17, 2011 at 4:42 PM, Timo Sirainen wrote: > > > Do you have more than one Dovecot server? If only one, you can still > > disable the mail_nfs_* settings. Also it would be interesting to see > > nfsstat numbers from the Dovecot server, compared to those iostat numbers.. > > > > On 18.11.2011, at 0.38, Andy Robbins wrote: > > > > > Well, the iostat command was run from the NFS server and dovecot was run > > from the mail server where it is mounted, hence the discrepancy there. > > > > > > > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > > > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > > > > > # iostat -d 5 -x > > > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > > avgrq-sz > > > > avgqu-sz await svctm %util > > > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > > 8.00 28.69 19.63 0.70 94.00 > > > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > > 8.00 111.42 94.44 0.74 99.60 > > > > > > These devices don't look like NFS, but you have: > > > > > > > mmap_disable: yes > > > > mail_nfs_storage: yes > > > > mail_nfs_index: yes > > > > > > So what filesystem is that? Are they storing only mails, or other stuff > > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > > things up at least somewhat.. > > > > > > > lda: > > > > fsync_disable: no > > > > > > This is explicitly set for LDA, but it's the default already elsewhere. > > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > > > > > > > > > > > From tss at iki.fi Mon Nov 21 23:45:39 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:45:39 +0200 Subject: [Dovecot] Dovecot Log analysis In-Reply-To: <20111121101941.GA25724@parmesan.sis.pasteur.fr> References: <20111121101941.GA25724@parmesan.sis.pasteur.fr> Message-ID: <1321911939.24607.55.camel@hurina> On Mon, 2011-11-21 at 11:19 +0100, Thomas Hummel wrote: > Hello, > > Before writing them myself, I'd like to figure out if some tools exist (anvil ? > ...) and how to use them to extract from dovecot logs things like who is doing > what and how hard or how fast in order to do some rate limiting. For instance, > I'd like to be able to identify, slow or block users who > > . read or write to much / slice of time > > . login/logout too often / slice of time > > Note : I'm still using the 1.2.x branch. The reading/writing rate limiting could be a bit tricky. I guess with Linux you could look into /proc//io files for all imap/pop3 processes and determine from there which processes are doing too much. For pop3 there's also http://dovecot.org/patches/1.2/pop3-throttle-plugin.c Other than that, I don't know of any existing stuff for v1.2. For v2.1 there's the new "stats" service and you can ask all kinds of statistics from it, including what users/processes have used the most disk IO. From pw at wk-serv.de Mon Nov 21 23:45:49 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 21 Nov 2011 22:45:49 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <1321472837.21919.598.camel@hurina> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> Message-ID: <4ECAC68D.3060007@wk-serv.de> Timo Sirainen schrieb: > On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: >> I already use lmtp:unix:private/dovecot-lmtp as transport but where is >> the link to the indexes? > > You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot > LMTP proxy, which would forward the connection to the backend server > which handles that user's IMAP/POP3/LMTP connections. I don't know if we're talking about the same :) From achekalin at lazurit.com Mon Nov 21 23:47:15 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 00:47:15 +0300 Subject: [Dovecot] Unique message IDs? In-Reply-To: References: <4ECAAE5F.1040507@lazurit.com> Message-ID: <4ECAC6E3.9090307@lazurit.com> > Message GUIDs are pretty good for that. Oh, thank you! Nice news! > With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). But what if I one day decide to convert my maildir's to mbox'es? I really plan to do such conversion in a while (as soon as I finish the indexing system). Yours, Alexander From tss at iki.fi Mon Nov 21 23:52:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:52:28 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4ECA174A.1020909@informatik.uni-freiburg.de> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> Message-ID: <1321912348.24607.58.camel@hurina> On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: > Hello, > after serveral tests (and reading a lot of howto's) I'm now fairly convinced > that the Solaris HW-crypto engine is not automatically used. Even when dovecot > ist compiled with the OpenSSL version provided by Solaris. > > Currently I have only a T1-CPU available for testing (Sun Fire T2000) and > after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with > > ENGINE *e; > ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); Does "openssl engine" return this "pkcs11" string? > ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); > ENGINE_set_default_ciphers(e); > > in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() > the crypto device gets used. I'm sure that this is not the whole story since > this only seems to affect the IMAP login. It should work for POP3 as well, all of the SSL code is shared. > One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in > the various articles). Is there any chance that Dovecot gets updated/patched > for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. I guess I could add ssl_crypto_device setting for this. But I'll need to figure out proper ifdefs to avoid compile failures with older OpenSSL versions. From tss at iki.fi Mon Nov 21 23:56:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:56:46 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECAC6E3.9090307@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> Message-ID: <1321912606.24607.61.camel@hurina> On Tue, 2011-11-22 at 00:47 +0300, Alexander Chekalin wrote: > > With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). > > But what if I one day decide to convert my maildir's to mbox'es? > I really plan to do such conversion in a while (as soon as I finish the > indexing system). mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of fakes it by returning MD5 of specific headers as GUIDs (so not 100% reliable). With mdbox GUIDs work even better than with Maildir, the GUID is always stored in the message's metadata. From hoogendyk at bio.umass.edu Tue Nov 22 00:26:58 2011 From: hoogendyk at bio.umass.edu (Chris Hoogendyk) Date: Mon, 21 Nov 2011 17:26:58 -0500 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1321912348.24607.58.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> Message-ID: <4ECAD032.6060902@bio.umass.edu> On 11/21/11 4:52 PM, Timo Sirainen wrote: > On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: >> Hello, >> after serveral tests (and reading a lot of howto's) I'm now fairly convinced >> that the Solaris HW-crypto engine is not automatically used. Even when dovecot >> ist compiled with the OpenSSL version provided by Solaris. >> >> Currently I have only a T1-CPU available for testing (Sun Fire T2000) and >> after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with >> >> ENGINE *e; >> ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); > Does "openssl engine" return this "pkcs11" string? > >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >> ENGINE_set_default_ciphers(e); >> >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >> the crypto device gets used. I'm sure that this is not the whole story since >> this only seems to affect the IMAP login. > It should work for POP3 as well, all of the SSL code is shared. > >> One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in >> the various articles). Is there any chance that Dovecot gets updated/patched >> for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. > I guess I could add ssl_crypto_device setting for this. But I'll need to > figure out proper ifdefs to avoid compile failures with older OpenSSL > versions. Just a quick comment -- the Sun instructions for setting up Apache specifically noted the importance of specifying "SSLCryptoDevice pkcs11" in httpd.conf. So, the OpenSSL provided by Solaris will do what it should, IFF pkcs11 is specified by the program that is calling it. I had an email exchange with a Solaris developer at some point, who said that GnuPG simply could not use the crypto device, because it was built with libcrypt, did not use the pkcs11 engine, and it would take a significant rewrite to make it do so. -- --------------- Chris Hoogendyk - O__ ---- Systems Administrator c/ /'_ --- Biology& Geology Departments (*) \(*) -- 140 Morrill Science Center ~~~~~~~~~~ - University of Massachusetts, Amherst --------------- Erd?s 4 From preen at informatik.uni-freiburg.de Tue Nov 22 00:49:28 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Mon, 21 Nov 2011 23:49:28 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1321912348.24607.58.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> Message-ID: <4ECAD578.4060304@informatik.uni-freiburg.de> Timo Sirainen wrote: > On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: >> Hello, >> after serveral tests (and reading a lot of howto's) I'm now fairly convinced >> that the Solaris HW-crypto engine is not automatically used. Even when dovecot >> ist compiled with the OpenSSL version provided by Solaris. >> >> Currently I have only a T1-CPU available for testing (Sun Fire T2000) and >> after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with >> >> ENGINE *e; >> ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); > > Does "openssl engine" return this "pkcs11" string? This requires the Solaris OpenSSL version (or another version using the pkcs11 patch). # /usr/sfw/bin/openssl engine (pkcs11) PKCS #11 engine support >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >> ENGINE_set_default_ciphers(e); >> >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >> the crypto device gets used. I'm sure that this is not the whole story since >> this only seems to affect the IMAP login. > > It should work for POP3 as well, all of the SSL code is shared. I couldn't find the EncryptUpdate call which has to be changed too (due to the howto documents). Maybe some other call needs e patch. But I don't know which. Martin >> One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in >> the various articles). Is there any chance that Dovecot gets updated/patched >> for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. > > I guess I could add ssl_crypto_device setting for this. But I'll need to > figure out proper ifdefs to avoid compile failures with older OpenSSL > versions. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6350 bytes Desc: S/MIME Cryptographic Signature URL: From Damon_Atkins at yahoo.com.au Tue Nov 22 04:17:39 2011 From: Damon_Atkins at yahoo.com.au (Damon Atkins) Date: Tue, 22 Nov 2011 13:17:39 +1100 Subject: [Dovecot] Solaris hardware crypto engines Message-ID: <4ECB0643.7070308@yahoo.com.au> Here are some blogs on the topic. http://wikis.sun.com/display/CryptoPerf/Using+the+UltraSPARC+cryptographic+accelerators Solaris 10 # /usr/sfw/bin/openssl engine -c -t # cc -fast*-I /usr/sfw/include -L /usr/sfw/lib -lcrypto* aes_test.c -o aes_test.out http://blogs.oracle.com/DanX/entry/sparc_t4_openssl_engine http://blogs.oracle.com/DanX/entry/where_s_the_crypto_libraries http://blogs.oracle.com/DanX/entry/solaris_x86_aesni_openssl_engine http://blogs.oracle.com/chichang1/entry/rsa_performance_of_sun_fire Here is some info from my intel box Solaris 11 # /usr/bin/openssl engine -c -t (aesni) Intel AES-NI engine (no-aesni) % no-aesni means no aes H/W acceleration [ available ] (dynamic) Dynamic engine loading support [ unavailable ] (pkcs11) PKCS #11 engine support [RSA, DSA, DH, RAND, DSA] [ available ] $ isainfo -v # My cpu does not have 'aes' support 64-bit amd64 applications cx16 sse3 sse2 sse fxsr mmx cmov amd_sysc cx8 tsc fpu 32-bit i386 applications ahf cx16 sse3 sse2 sse fxsr mmx cmov sep cx8 tsc fpu # ldd /opt/dovecot/libexec/dovecot/ssl-build-param **** libssl.so.1.0.0 => /lib/libssl.so.1.0.0 *** *** libcrypto.so.1.0.0 => /lib/libcrypto.so.1.0.0* *** libc.so.1 => /lib/libc.so.1 libnsl.so.1 => /lib/libnsl.so.1 libsocket.so.1 => /lib/libsocket.so.1 librt.so.1 => /lib/librt.so.1 libsendfile.so.1 => /lib/libsendfile.so.1 libmp.so.2 => /lib/libmp.so.2 libmd.so.1 => /lib/libmd.so.1 libm.so.2 => /lib/libm.so.2 ./configure --prefix=/opt/dovecot --with-ldap=yes --with-gssapi --with-ssldir=/etc/openssl Install prefix . : /opt/dovecot File offsets ... : 64bit I/O polling .... : poll I/O notifys .... : none SSL ............ : yes (OpenSSL) GSSAPI ......... : yes passdbs ........ : passwd passwd-file shadow pam checkpassword ldap : -bsdauth -sia -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword ldap : -sql -vpopmail -nss SQL drivers .... : : -pgsql -mysql -sqlite Note Under OpenSolaris I did the following: CPPFLAGS=-I/usr/sfw/include LDFLAGS=-R/usr/sfw/lib ./configure --prefix=/opt/dovecot --with-ldap=yes --with-gssapi --with-ssldir=/etc/openssl (most likely Solaris 10 is the same as above, openssl may look old but I believe it is patched with compatible *fixes* from current openssl so application do not break. Apparently it took 5mths to update Solaris 11 to OpenSSL 1.0 and test everything) Cheers Damon. From achekalin at lazurit.com Tue Nov 22 07:39:30 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 08:39:30 +0300 Subject: [Dovecot] Unique message IDs? In-Reply-To: <1321912606.24607.61.camel@hurina> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> Message-ID: <4ECB3592.1010500@lazurit.com> >>> With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). >> >> But what if I one day decide to convert my maildir's to mbox'es? >> I really plan to do such conversion in a while (as soon as I finish the >> indexing system). > > mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of > fakes it by returning MD5 of specific headers as GUIDs (so not 100% > reliable). With mdbox GUIDs work even better than with Maildir, the GUID > is always stored in the message's metadata. I'd like to use the best optimized one (mdbox), but there is a reason not to do that is when I use mbox or maildir I can see where given IMAP folder mails are stored, so, say, if I want to copy only one IMAP folder to some remote site, I just copy know dir or file. With mdbox this is different, I simply can not guess where my messages exactly are. If it be possible to have per-IMAP-folders mdboxes, I love to use it. But from what I know and tried this is not way mdbox used to work, right? Yours, Alexander Chekalin From janfrode at tanso.net Tue Nov 22 09:16:24 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 22 Nov 2011 08:16:24 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4ECAC68D.3060007@wk-serv.de> References: <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> Message-ID: <20111122071624.GA6110@dibs.tanso.net> On Mon, Nov 21, 2011 at 10:45:49PM +0100, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: > >>I already use lmtp:unix:private/dovecot-lmtp as transport but where is > >>the link to the indexes? > > > >You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot > >LMTP proxy, which would forward the connection to the backend server > >which handles that user's IMAP/POP3/LMTP connections. > > I don't know if we're talking about the same :) > I wondered that too. It looked to me like you tried to ask where the lmtp-service picks up the path to indexes, right? AFAIU it picks that up from the /var/run/dovecot/auth-master socket. -jf From achekalin at lazurit.com Tue Nov 22 11:10:12 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 12:10:12 +0300 Subject: [Dovecot] No duplicates on imap copy? Message-ID: <4ECB66F4.7060204@lazurit.com> Hello, when from my script I try to copy messages from one imap folder into another own (within the same imap mailbox). That works perfectly, but... If I copy the same message N times from one folder into another one, in the destination folder I see N copies of the same message, each with its own UID. I wonder if there any way to deduplicate folder content (so in the destination folder I'll get only one copy of each message)? I'd implement something like that myself in my script but I just don't know which part of the messages should I compare - probably md5 of headers? Thank you in advance! Yours, Alexander Chekalin From tss at iki.fi Tue Nov 22 11:58:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 11:58:46 +0200 Subject: [Dovecot] No duplicates on imap copy? In-Reply-To: <4ECB66F4.7060204@lazurit.com> References: <4ECB66F4.7060204@lazurit.com> Message-ID: <74E4AA7F-089C-4FF4-BE74-F80E8D1848D3@iki.fi> On 22.11.2011, at 11.10, Alexander Chekalin wrote: > when from my script I try to copy messages from one imap folder into another own (within the same imap mailbox). That works perfectly, but... If I copy the same message N times from one folder into another one, in the destination folder I see N copies of the same message, each with its own UID. Right. > I wonder if there any way to deduplicate folder content (so in the destination folder I'll get only one copy of each message)? Not any easy way currently. > I'd implement something like that myself in my script but I just don't know which part of the messages should I compare - probably md5 of headers? What Dovecot version and what mailbox format? From tss at iki.fi Tue Nov 22 12:01:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 12:01:28 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECB3592.1010500@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> <4ECB3592.1010500@lazurit.com> Message-ID: <275436D2-0816-47E2-877C-FC55AC43B67D@iki.fi> On 22.11.2011, at 7.39, Alexander Chekalin wrote: > I'd like to use the best optimized one (mdbox), but there is a reason not to do that is when I use mbox or maildir I can see where given IMAP folder mails are stored, so, say, if I want to copy only one IMAP folder to some remote site, I just copy know dir or file. With mdbox this is different, I simply can not guess where my messages exactly are. > > If it be possible to have per-IMAP-folders mdboxes, I love to use it. But from what I know and tried this is not way mdbox used to work, right? sdbox would work like that. The reason mdbox doesn't work like that is because copying messages would be rather slow then. The idea with mdbox is anyway that you'd use Dovecot's tools to manage the mailboxes rather than access them directly through filesystem. So if you want to copy one IMAP folder, you'd use either dsync or doveadm import to do it. From pw at wk-serv.de Tue Nov 22 12:17:12 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 22 Nov 2011 11:17:12 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <20111122071624.GA6110@dibs.tanso.net> References: <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> Message-ID: <4ECB76A8.80208@wk-serv.de> Jan-Frode Myklebust schrieb: > I wondered that too. It looked to me like you tried to ask where the > lmtp-service picks up the path to indexes, right? AFAIU it picks that up > from the /var/run/dovecot/auth-master socket. No. I want to know if dovecot writes to the indexes if it receives a mail via lmtp. Someone proposed to store the index files on a locally installed SSD on a frontend (imap) machine and stick the users to that machine but if the lmtp-service writes to the indexes (and I think he does), that machine needs access to the indexes too which will bring us back to shared storage. From janfrode at tanso.net Tue Nov 22 12:45:47 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 22 Nov 2011 11:45:47 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4ECB76A8.80208@wk-serv.de> References: <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> Message-ID: <20111122104547.GA10680@dibs.tanso.net> On Tue, Nov 22, 2011 at 11:17:12AM +0100, Patrick Westenberg wrote: > > No. I want to know if dovecot writes to the indexes if it receives a > mail via lmtp. > > Someone proposed to store the index files on a locally installed SSD > on a frontend (imap) machine and stick the users to that machine but > if the lmtp-service writes to the indexes (and I think he does), > that machine needs access to the indexes too which will bring us > back to shared storage. Ah, then Timo's reply was right. He suggested you do the lmtp-deliveries to the same server that you would send you imap-user to. You can do this trough dovecot director and lmtp-proxying. So instead of: lmtp:unix:private/dovecot-lmtp you should use: lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be the Dovecot LMTP proxy that proxies to the same machine as you would use for imap for this particular recipient. -jf From stan at hardwarefreak.com Tue Nov 22 19:00:08 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 22 Nov 2011 11:00:08 -0600 Subject: [Dovecot] 1.2.15 can't get zlib working Message-ID: <4ECBD518.9020206@hardwarefreak.com> Following these instructions: http://wiki.dovecot.org/Plugins/Zlib I enabled zlib and restarted Dovecot. I made a backup copy of a 68MB mbox file and moved it to a non mail directory. The mbox file is an existing TBird IMAP folder in my UNIX user mail directory. It is an archive of a defunct mailing list. It was a regular IMAP folder prior to attempting this. I was able to access all emails in the folder. It worked fine. I gzipped the original mbox file down to 14MB and removed the write flag with chmod. I launched TBird and received the following error in Activity Manager when accessing this IMAP folder: The current operation on '1-Spam-l' did not succeed. The mail server for account stan at hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l I deleted the Dovecot index files and the TBird 1-Spam-l.msf file which didn't help. Curiously, there are no entries in either dovecot.err or dovecot.log, giving not even the slightest hint as to where I should troubleshoot. Kinda sucks when one follows instructions exactly, simple instructions at that, and it doesn't work, and generates zero log errors. A bit frustrating... Any ideas? /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_plugins: fts fts_squat zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster at hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10 -- Stan From mrvjtod at gmail.com Tue Nov 22 20:24:15 2011 From: mrvjtod at gmail.com (Chris Young) Date: Tue, 22 Nov 2011 13:24:15 -0500 Subject: [Dovecot] disable imap info logging Message-ID: Is there a way to disable the imap info logging? My logs are filling with tons of login/logout messages Nov 22 18:23:25 imap-login: Info: Login: user=, method={method}, rip=10.86.189.79, lip=10.86.189.82, mpid=22488 Nov 22 18:23:25 imap(USER1): Info: Disconnected: Logged out bytes=54/726 From tss at iki.fi Tue Nov 22 20:28:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 20:28:56 +0200 Subject: [Dovecot] disable imap info logging In-Reply-To: References: Message-ID: <4A94BBF7-5704-4542-9796-942B50940367@iki.fi> On 22.11.2011, at 20.24, Chris Young wrote: > Is there a way to disable the imap info logging? > > My logs are filling with tons of login/logout messages > > Nov 22 18:23:25 imap-login: Info: Login: user=, method={method}, > rip=10.86.189.79, lip=10.86.189.82, mpid=22488 > Nov 22 18:23:25 imap(USER1): Info: Disconnected: Logged out bytes=54/726 Rotate the logs and have it delete old ones? Or simply: info_log_path = /dev/null From tss at iki.fi Tue Nov 22 20:30:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 20:30:33 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECBD518.9020206@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> Message-ID: On 22.11.2011, at 19.00, Stan Hoeppner wrote: > The current operation on '1-Spam-l' did not succeed. The mail server > for account stan at hardwarefreak.com responded: Mailbox doesn't exist: > 1-Spam-l .. > /$ la /home/stan/mail/1-Spam-l.gz > -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) From Juergen.Obermann at hrz.uni-giessen.de Tue Nov 22 22:11:02 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Tue, 22 Nov 2011 21:11:02 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC Message-ID: Hello, compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC with Sun Studio 11 stops with the following error: Making all in lib-imap-client gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-imap-client' source='imapc-client.c' object='imapc-client.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-client.lo imapc-client.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-client.c -KPIC -DPIC -o .libs/imapc-client.o "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-client.c -o imapc-client.o >/dev/null 2>&1 source='imapc-connection.c' object='imapc-connection.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-connection.lo imapc-connection.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-connection.c -KPIC -DPIC -o .libs/imapc-connection.o "../../src/lib-imap/imap-util.h", line 6: warning: useless declaration "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) "imapc-connection.c", line 1446: operand cannot have void type: op "==" "imapc-connection.c", line 1446: operands have incompatible types: const void "==" pointer to struct imapc_command {pointer to struct pool {..} pool, pointer to struct buffer {..} data, unsigned int send_pos, unsigned int tag, enum imapc_command_flags {IMAPC_COMMAND_FLAG_RETRIABLE(4), IMAPC_COMMAND_FLAG_PRELOGIN(2), IMAPC_COMMAND_FLAG_SELECT(1)} flags, pointer to struct imapc_connection {..} conn, pointer to struct imapc_client_mailbox {..} box, union {..} streams, pointer to function(pointer to const struct imapc_command_reply {..}, pointer to void) returning void callback, pointer to void context, unsigned int idle :1, unsigned int wait_for_literal :1} cc: acomp failed for imapc-connection.c gmake[3]: *** [imapc-connection.lo] Error 1 gmake[3]: Leaving directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-imap-client' Viele Gr??e, J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From ramiblanco at gmail.com Tue Nov 22 22:52:38 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Tue, 22 Nov 2011 17:52:38 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search Message-ID: Hi, I found that doing a wildcard search with doveadm doesn't expand the %d global variable when doing a wildcard search like: doveadm search *@domain ALL LARGER 10000k doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users In /var/log/dovecot.log: Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 22 20:48:07 auth-worker: Error: ldap(?): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object Is this a bug? Configs: #file /usr/local/etc/dovecot/dovecot-ldap.conf.ext: hosts = x.x.x.x dn = cn=dovecot,dc=domain dnpass = mypass auth_bind = yes auth_bind_userdn = mail=%u,vd=%d,dc=domain ldap_version = 3 base = vd=%d,dc=domain deref = never scope = subtree user_attrs = mailbox=mail=maildir:/home/vmail/domains/%$,quota=quota_rule=*:storage=%$,vdHome=home=%$/%d/%1u/%1.1u/%n,=uid=504,=gid=12 user_filter = (&(&(objectClass=VirtualMailAccount)(mail=%u))(accountActive=TRUE)) pass_attrs = userPassword=password,mail=user pass_filter = (&(&(objectClass=VirtualMailAccount)(mail=%u))(accountActive=TRUE)) iterate_attrs = mail=user iterate_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)) default_pass_scheme = CRYPT # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.3.1.el5 x86_64 CentOS release 5.5 (Final) ext3 auth_mechanisms = plain login dict { expire = pgsql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } hostname = localhost lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot.log mail_gid = 12 mail_location = maildir:/home/vmail/domains/%d/%1u/%1.1u/%n mail_plugins = quota autocreate expire mail_privileged_group = mail mail_uid = 504 managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Enviados autocreate2 = Papelera autocreate3 = Borrador autocreate4 = Spam autosubscribe = Enviados autosubscribe2 = Papelera autosubscribe3 = Borrador autosubscribe4 = Spam expire = Papelera expire2 = Papelera/* expire3 = Spam expire4 = Spam/* expire_dict = proxy::expire quota = maildir:User quota quota_rule = *:storage=100M quota_rule2 = Papelera:storage=+100M sieve = ~/sieve/dovecot.sieve sieve_before = /usr/local/etc/sieve/spam.sieve sieve_dir = ~/sieve sieve_global_dir = /usr/local/etc/sieve/ } postmaster_address = postmaster at d omain protocols = imap sieve pop3 service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } vsz_limit = 64 M } ssl_cert = References: <4ECBD518.9020206@hardwarefreak.com> Message-ID: <4ECC2149.4070403@hardwarefreak.com> On 11/22/2011 12:30 PM, Timo Sirainen wrote: > On 22.11.2011, at 19.00, Stan Hoeppner wrote: > >> The current operation on '1-Spam-l' did not succeed. The mail server >> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >> 1-Spam-l > .. >> /$ la /home/stan/mail/1-Spam-l.gz >> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz > > The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) Aha. That was it. Thanks Timo. For some reason my read of the wiki page made me think this was handled transparently--just zip the file and everything works as it did before. Apparently it's not as simple as the wiki leads one (me anyway) to believe. This bit of the wiki caused me some confusion as well: "Compressed mbox files can be accessed only as read-only" Thus I chmod'ed the .gz file to read-only. This creates a problem. It appears that when Dovecot creates the .imap folder of the same name it inherits the permissions of the zipped mbox file. Thus it can't create the indexes: 2011-11-22 14:59:23 IMAP(stan): Error: file_dotlock_open(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log) failed: Permission denied (euid=1000(stan) egid=1000(stan) access(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log, 4) failed: No such file or directory) Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki could be reworded in a way that doesn't prompt some folks to manually make the zipped files read-only. It took a while for Dovecot to index the 15K+ messages. With that finished, accessing the folder is similar to before, but there's a small lag when opening messages. As this is an archive folder the contents won't change, so Squat FTS should be very fast after the first search, just as before. Interestingly, it appears my squat indexes aren't updating--for any folder. I've searched 4 folders via Tbird body search with xyzzyx (took forever) and I see no changes to the dates or sizes of existing indexes. I deleted the squat indexes for one folder and ran the search again. No new squat indexes were created. No errors in the logs. Any ideas why the squat indexes aren't updating? IIRC this happened once before and I was able to fix it. Don't recall how I did it though.... Once again: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_process_size: 320 mail_plugins: zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster at hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10 Thanks. -- Stan From dovecot at knutejohnson.com Wed Nov 23 00:53:30 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Tue, 22 Nov 2011 14:53:30 -0800 Subject: [Dovecot] How to disable pop3 in version 2? Message-ID: <4ECC27EA.2000200@knutejohnson.com> I'm about to bring up a new mail server running Ubuntu 11.10 with Dovecot 2. I've been using 1.2 all along and there are a bunch more config files in 2. In 1.2 I just edited the protocols statement but I can't find one in version 2. Is there a way to disable pop3 or do I just firewall the ports? And a related question, I've been using imaps on port 993 but I hear that's not the best solution anymore, I should be using imap with STARTTLS? How do I disable imaps? Thanks, -- Knute Johnson From tss at iki.fi Wed Nov 23 00:56:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 00:56:10 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: Message-ID: On 22.11.2011, at 22.52, Ramiro Blanco wrote: > Hi, I found that doing a wildcard search with doveadm doesn't expand the %d > global variable when doing a wildcard search like: > > Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain > scope=subtree Iteration is supposed to iterate through all users.. > Is this a bug? More like a missing feature I guess. I guess it could make it expand the %d if it's known.. From tss at iki.fi Wed Nov 23 01:01:44 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:01:44 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC In-Reply-To: References: Message-ID: On 22.11.2011, at 22.11, J?rgen Obermann wrote: > "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) What? No it doesn't.. Does changing the 0x400000000 in line 20 to 0x200000000 help? > "imapc-connection.c", line 1446: operand cannot have void type: op "==" > "imapc-connection.c", line 1446: operands have incompatible types: Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1cdd39d11ce4 From tss at iki.fi Wed Nov 23 01:03:35 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:03:35 +0200 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <4ECC27EA.2000200@knutejohnson.com> References: <4ECC27EA.2000200@knutejohnson.com> Message-ID: <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> On 23.11.2011, at 0.53, Knute Johnson wrote: > I'm about to bring up a new mail server running Ubuntu 11.10 with Dovecot 2. I've been using 1.2 all along and there are a bunch more config files in 2. In 1.2 I just edited the protocols statement but I can't find one in version 2. Is there a way to disable pop3 or do I just firewall the ports? In the example dovecot.conf there's a protocols setting. If you don't see any, you can just add your own wherever. "protocols=imap" should work. > And a related question, I've been using imaps on port 993 but I hear that's not the best solution anymore, I should be using imap with STARTTLS? How do I disable imaps? Well, http://wiki2.dovecot.org/SSL explains this more. I don't think you really need to disable imaps, but if you want to, you can do it with: service imap-login { inet_listener imaps { port = 0 } } From tss at iki.fi Wed Nov 23 01:07:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:07:28 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECC2149.4070403@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> Message-ID: <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> On 23.11.2011, at 0.25, Stan Hoeppner wrote: > On 11/22/2011 12:30 PM, Timo Sirainen wrote: >> On 22.11.2011, at 19.00, Stan Hoeppner wrote: >> >>> The current operation on '1-Spam-l' did not succeed. The mail server >>> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >>> 1-Spam-l >> .. >>> /$ la /home/stan/mail/1-Spam-l.gz >>> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz >> >> The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) > > Aha. That was it. Thanks Timo. For some reason my read of the wiki > page made me think this was handled transparently--just zip the file and > everything works as it did before. Apparently it's not as simple as the > wiki leads one (me anyway) to believe. I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :) > This bit of the wiki caused me some confusion as well: > "Compressed mbox files can be accessed only as read-only" > > Thus I chmod'ed the .gz file to read-only. This creates a problem. It > appears that when Dovecot creates the .imap folder of the same name it > inherits the permissions of the zipped mbox file. Thus it can't create > the indexes: I've fixed this in some version. I guess in v2.0. > Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki > could be reworded in a way that doesn't prompt some folks to manually > make the zipped files read-only. Well, I don't really care that much about v1.x anymore. > It took a while for Dovecot to index the 15K+ messages. With that > finished, accessing the folder is similar to before, but there's a small > lag when opening messages. Yeah, it's uncompressing the entire file until it finds the message you're opening. > As this is an archive folder the contents > won't change, so Squat FTS should be very fast after the first search, > just as before. Interestingly, it appears my squat indexes aren't > updating--for any folder. I've searched 4 folders via Tbird body search > with xyzzyx (took forever) and I see no changes to the dates or sizes of > existing indexes. I deleted the squat indexes for one folder and ran > the search again. No new squat indexes were created. No errors in the > logs. > > Any ideas why the squat indexes aren't updating? IIRC this happened > once before and I was able to fix it. Don't recall how I did it though.... > mail_plugins: zlib Doesn't look like fts, fts_squat is enabled? From dovecot at knutejohnson.com Wed Nov 23 02:40:31 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Tue, 22 Nov 2011 16:40:31 -0800 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> Message-ID: <4ECC40FF.8020103@knutejohnson.com> On 11/22/2011 3:03 PM, Timo Sirainen wrote: > On 23.11.2011, at 0.53, Knute Johnson wrote: > >> I'm about to bring up a new mail server running Ubuntu 11.10 with >> Dovecot 2. I've been using 1.2 all along and there are a bunch >> more config files in 2. In 1.2 I just edited the protocols >> statement but I can't find one in version 2. Is there a way to >> disable pop3 or do I just firewall the ports? > > In the example dovecot.conf there's a protocols setting. If you don't > see any, you can just add your own wherever. "protocols=imap" should > work. > >> And a related question, I've been using imaps on port 993 but I >> hear that's not the best solution anymore, I should be using imap >> with STARTTLS? How do I disable imaps? > > Well, http://wiki2.dovecot.org/SSL explains this more. I don't think > you really need to disable imaps, but if you want to, you can do it > with: > > service imap-login { inet_listener imaps { port = 0 } } > > Thanks for that. The service imap-login line above is in the file /etc/dovecot/conf.d/10-master.conf. I don't know if that is standard for version 2 or something that Ubuntu created. The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; !include_try /usr/share/dovecot/protocols.d/*.protocol That directory contains one file; dovecot-imapd.protocol and it contains one line; protocols = $protocols imap So I am assuming that it is adding imap to the protocols in the environment variable $protocols somehow? dovecot -n yields; protocols = " imap" I did get TBird to retrieve mail from port 993 using SSL however. Will it work without a protocols = imaps? Thanks, -- Knute Johnson From stan at hardwarefreak.com Wed Nov 23 02:56:29 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 22 Nov 2011 18:56:29 -0600 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> Message-ID: <4ECC44BD.3080806@hardwarefreak.com> On 11/22/2011 5:07 PM, Timo Sirainen wrote: > On 23.11.2011, at 0.25, Stan Hoeppner wrote: > >> On 11/22/2011 12:30 PM, Timo Sirainen wrote: >>> On 22.11.2011, at 19.00, Stan Hoeppner wrote: >>> >>>> The current operation on '1-Spam-l' did not succeed. The mail server >>>> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >>>> 1-Spam-l >>> .. >>>> /$ la /home/stan/mail/1-Spam-l.gz >>>> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz >>> >>> The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) >> >> Aha. That was it. Thanks Timo. For some reason my read of the wiki >> page made me think this was handled transparently--just zip the file and >> everything works as it did before. Apparently it's not as simple as the >> wiki leads one (me anyway) to believe. > > I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :) Out of curiosity, what (or who) prompted the development of the compressed mbox feature? Or was it that you wanted to do it for maildir, and then figured you should for mbox as well? The implementation seems to work ok. The instructions just seem a bit...thin. :) >> This bit of the wiki caused me some confusion as well: >> "Compressed mbox files can be accessed only as read-only" >> >> Thus I chmod'ed the .gz file to read-only. This creates a problem. It >> appears that when Dovecot creates the .imap folder of the same name it >> inherits the permissions of the zipped mbox file. Thus it can't create >> the indexes: > > I've fixed this in some version. I guess in v2.0. Yeah, I'm waiting for Debian to get a backport of 2.0.x. For many reasons the only thing I'm comfortable installing from source is the Linux kernel. >> Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki >> could be reworded in a way that doesn't prompt some folks to manually >> make the zipped files read-only. > > Well, I don't really care that much about v1.x anymore. Understandable. >> It took a while for Dovecot to index the 15K+ messages. With that >> finished, accessing the folder is similar to before, but there's a small >> lag when opening messages. > > Yeah, it's uncompressing the entire file until it finds the message you're opening. It's pretty damn fast at it. I haven't seen anything more than a couple of seconds lag while randomly accessing mail all over the folder. The original gzip of the file took >45 seconds. >> As this is an archive folder the contents >> won't change, so Squat FTS should be very fast after the first search, >> just as before. Interestingly, it appears my squat indexes aren't >> updating--for any folder. I've searched 4 folders via Tbird body search >> with xyzzyx (took forever) and I see no changes to the dates or sizes of >> existing indexes. I deleted the squat indexes for one folder and ran >> the search again. No new squat indexes were created. No errors in the >> logs. >> >> Any ideas why the squat indexes aren't updating? IIRC this happened >> once before and I was able to fix it. Don't recall how I did it though.... > >> mail_plugins: zlib > > Doesn't look like fts, fts_squat is enabled? Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins had to be declared in a single line directive. I did this as part of my troubleshooting when zlib wasn't working, thinking putting it on it's own line may help--not. -- Stan From Juergen.Obermann at hrz.uni-giessen.de Wed Nov 23 12:01:25 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Wed, 23 Nov 2011 11:01:25 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10on SPARC In-Reply-To: References: Message-ID: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> ----- Nachricht von tss at iki.fi --------- Datum: Wed, 23 Nov 2011 01:01:44 +0200 Von: Timo Sirainen Betreff: Re: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC An: J?rgen Obermann Cc: dovecot at dovecot.org > On 22.11.2011, at 22.11, J?rgen Obermann wrote: > >> "imapc-client.h", line 21: warning: enumerator value overflows >> INT_MAX (2147483647) > > What? No it doesn't.. Does changing the 0x400000000 in line 20 to > 0x200000000 help? No, but stripping off the last zero helps, because INT_MAX is 0x7FFFFFFF. >> "imapc-connection.c", line 1446: operand cannot have void type: op "==" >> "imapc-connection.c", line 1446: operands have incompatible types: > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1cdd39d11ce4 > > > OK now. ----- Ende der Nachricht von tss at iki.fi ----- One more compilation problem occurs: gmake[5]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-storage/index/imapc' source='imapc-storage.c' object='imapc-storage.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../../../depcomp \ /bin/bash ../../../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-storage.lo imapc-storage.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-storage.c -KPIC -DPIC -o .libs/imapc-storage.o "../../../../src/lib-storage/mailbox-list.h", line 13: warning: useless declaration "../../../../src/lib-storage/mailbox-list-private.h", line 14: warning: useless declaration "imapc-sync.h", line 4: warning: useless declaration "imapc-storage.c", line 500: void function cannot return value cc: acomp failed for imapc-storage.c gmake[5]: *** [imapc-storage.lo] Error 1 Greetings, J?rgen Obermann -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From patrickdk at patrickdk.com Wed Nov 23 14:24:39 2011 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 23 Nov 2011 07:24:39 -0500 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECB3592.1010500@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> <4ECB3592.1010500@lazurit.com> Message-ID: <20111123072439.Horde.FvMFLZLnE6FOzOYHNPlAn_A@kishi.patrickdk.com> Quoting Alexander Chekalin : >>>> With Maildir the message GUID is typically the same as the >>>> Maildir base filename (i.e. everything before ':' character). >>> >>> But what if I one day decide to convert my maildir's to mbox'es? >>> I really plan to do such conversion in a while (as soon as I finish the >>> indexing system). >> >> mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of >> fakes it by returning MD5 of specific headers as GUIDs (so not 100% >> reliable). With mdbox GUIDs work even better than with Maildir, the GUID >> is always stored in the message's metadata. > > I'd like to use the best optimized one (mdbox), but there is a > reason not to do that is when I use mbox or maildir I can see where > given IMAP folder mails are stored, so, say, if I want to copy only > one IMAP folder to some remote site, I just copy know dir or file. > With mdbox this is different, I simply can not guess where my > messages exactly are. > > If it be possible to have per-IMAP-folders mdboxes, I love to use > it. But from what I know and tried this is not way mdbox used to > work, right? > You can always use the info from the wiki, I took some code from it to create this little script, that dumps my spam folder and uses it to learn. Basically dumps a mailfolder back into maildir format. doveadm search -u $useraccount mailbox Spam | while read guid uid; do doveadm fetch -u $useraccount text mailbox-guid $guid uid $uid > msg.$uid; done If your using mdbox on the other end, you could reimport them I suppose, I haven't looked into doing that, since I haven't needed that yet. From elhijo at 0lim.net Wed Nov 23 16:19:34 2011 From: elhijo at 0lim.net (David LEROUX) Date: Wed, 23 Nov 2011 15:19:34 +0100 Subject: [Dovecot] dovecot as an imap "client" Message-ID: <4ECD00F6.4060908@0lim.net> Hi, I need to get emails from one of my users imap account on a server which is not managed by us to his local imap mailbox. I'm almost sure that I've read some times that dovecot would be able to do that, kind of imap relay, but I can't find it again. I'm not sure to be very clear. Thanks, -- David From frank at moltke28.B.Shuttle.DE Wed Nov 23 16:53:18 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 23 Nov 2011 15:53:18 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: <4ECD00F6.4060908@0lim.net> References: <4ECD00F6.4060908@0lim.net> Message-ID: (auto-added) On Wed, 23 Nov 2011 15:19:34 +0100 David LEROUX wrote: > Hi, > I need to get emails from one of my users imap account on a server which > is not managed by us to his local imap mailbox. imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner From micah at riseup.net Wed Nov 23 16:54:21 2011 From: micah at riseup.net (Micah Anderson) Date: Wed, 23 Nov 2011 09:54:21 -0500 Subject: [Dovecot] doveadm import assertion failed Message-ID: <87vcqax52a.fsf@algae.riseup.net> Restoring a user's mailbox yesterday resulted in 'doveadm import' panic'ing with an assertion failure and giving a backtrace: /usr/bin/doveadm import -u mdbox:/maildir/riseup.net/a//.daily.1/mdbox restored_from_backups/daily1 all doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac tion log sequence (3) doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac tion log sequence (3) doveadm(): Panic: file mail-transaction-log-file.c: line 187 (mail_transaction_log_file_add_to_list): assertion failed: ((*p)->hdr.file_se q < file->hdr.file_seq) doveadm(): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd1a) [0x7f05152bbd1a] -> /usr/lib/dovecot/libdovecot.so.0(default_f atal_handler+0x32) [0x7f05152bbe02] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f051529519f] -> /usr/lib/dovecot/libdovecot-storage.so .0(+0xa596a) [0x7f05159e696a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_file_open+0x1f8) [0x7f05159e7bc8] -> /usr/lib/ dovecot/libdovecot-storage.so.0(mail_transaction_log_find_file+0xd0) [0x7f05159e46e0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transa ction_log_view_set+0xcb) [0x7f05159e8bdb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0xbe) [0x7f05159dff6e] -> /usr/lib/ dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7f05159d1816] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a6da) [0x7f05159db6da] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x56) [0x7f05159db956] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_ index_sync_begin+0x1e) [0x7f05159dc21e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x121) [0x7f05159bbb41] -> /usr/lib/dove cot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7f05159bc1f6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x43) [0x7f 05159bc293] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7f0515974031] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7f05159748b7] -> /usr/bin/doveadm(doveadm_mail_iter_init+0x8b) [0x40c9db] -> /usr/bin/doveadm() [0x40c284] -> /usr/bin/doveadm() [0x40a673] -> /usr/bin/doveadm(doveadm_mail_single_user+0x61) [0x40a731] -> /usr/bin/doveadm() [0x40a95d] -> /usr/bin/doveadm(doveadm_mail_try_run+0x141) [0x40ad71] -> /usr/bin/doveadm(main+0x381) [0x4107e1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f0514b2dc4d] -> /usr/bin/doveadm() [0x409e59] This is with 2.0.15. Micah -- From tss at iki.fi Wed Nov 23 16:58:41 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 16:58:41 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10on SPARC In-Reply-To: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> Message-ID: <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> On 23.11.2011, at 12.01, J?rgen Obermann wrote: >> On 22.11.2011, at 22.11, J?rgen Obermann wrote: >> >>> "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) >> >> What? No it doesn't.. Does changing the 0x400000000 in line 20 to 0x200000000 help? > > No, but stripping off the last zero helps, because INT_MAX is 0x7FFFFFFF. Oops :) Looks like gcc automatically increases the enum's size to 64bit in that case. > "imapc-storage.c", line 500: void function cannot return value http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e From elhijo at 0lim.net Wed Nov 23 17:07:25 2011 From: elhijo at 0lim.net (David LEROUX) Date: Wed, 23 Nov 2011 16:07:25 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: (auto-added) References: <4ECD00F6.4060908@0lim.net> (auto-added) Message-ID: <4ECD0C2D.8080901@0lim.net> On 11/23/2011 03:53 PM, Frank Elsner wrote: > imapsync comes to my mind. - http://imapsync.lamiral.info/ - > http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync > --Frank Elsner Thanks, Thought that dovecot would be able to do that. David. From info at fduerr.de Wed Nov 23 10:16:30 2011 From: info at fduerr.de (Frank) Date: Wed, 23 Nov 2011 08:16:30 +0000 (UTC) Subject: [Dovecot] dsync strange duplicate directories Message-ID: Hello, i'm testing whether dsync will help me in setting up a redundant master - master imap setup and experience this: 1. Master 1 dovecot is running, master 2 dovecot is down 2. I use rsync to create an exact copy of the master 1 maildir on master 2 3. I start up master 2 dovecot 4. I do a dsync on master 2: dsync -D -u test at example.com mirror ssh root at master1 dsync -D -u test at example.com Now what i see is this: Each mail directory is duplicated on master 1 and master 2 to a directory name with some MD5 hash value (as it seems) e.g. before dsync: ls -a /home/vmail/example.com/test/Maildir cur new .Drafts .Junk .Sent ... after dsync: ls -a /home/vmail/example.com/test/Maildir cur new .Drafts .Drafts_af61060f6f9fcb4efc0a00002691362e .Junk .Junk_b261060f6f9fcb4efc0a00002691362e .Sent .Sent_b261060f6f9fcb4efc0a00002691362e ... The new directories contain the same files as the original directories. What am i doing wrong? Thanks for the help Frank From tss at iki.fi Wed Nov 23 17:29:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 17:29:16 +0200 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: <4ECD0C2D.8080901@0lim.net> References: <4ECD00F6.4060908@0lim.net> (auto-added) <4ECD0C2D.8080901@0lim.net> Message-ID: On 23.11.2011, at 17.07, David LEROUX wrote: > On 11/23/2011 03:53 PM, Frank Elsner wrote: >> imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner > Thanks, > Thought that dovecot would be able to do that. v2.1 has "imapc" backend, which you could use with dsync to do a migration, but for now it's probably easier to just use imapsync. From Juergen.Obermann at hrz.uni-giessen.de Wed Nov 23 19:07:09 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Wed, 23 Nov 2011 18:07:09 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris10on SPARC In-Reply-To: <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> Message-ID: Am 23.11.2011 15:58, schrieb Timo Sirainen: > On 23.11.2011, at 12.01, J?rgen Obermann wrote: > >>> On 22.11.2011, at 22.11, J?rgen Obermann wrote: >>> >>>> "imapc-client.h", line 21: warning: enumerator value overflows >>>> INT_MAX (2147483647) >>> >>> What? No it doesn't.. Does changing the 0x400000000 in line 20 to >>> 0x200000000 help? >> >> No, but stripping off the last zero helps, because INT_MAX is >> 0x7FFFFFFF. > > Oops :) Looks like gcc automatically increases the enum's size to > 64bit in that case. > >> "imapc-storage.c", line 500: void function cannot return value > > http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e There seems to be one more similar problem: gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/anvil' source='anvil-connection.c' object='anvil-connection.o' libtool=no \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-master -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c anvil-connection.c "anvil-connection.c", line 156: void function cannot return value cc: acomp failed for anvil-connection.c gmake[3]: *** [anvil-connection.o] Error 2 -- J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From tss at iki.fi Wed Nov 23 19:11:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 19:11:55 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris10on SPARC In-Reply-To: References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> Message-ID: <1322068318.24607.64.camel@hurina> On Wed, 2011-11-23 at 18:07 +0100, J?rgen Obermann wrote: > > http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e > > There seems to be one more similar problem: > > "anvil-connection.c", line 156: void function cannot return value Fixed: http://hg.dovecot.org/dovecot-2.1/rev/4f1d20b57c04 From pch0317 at gmail.com Wed Nov 23 21:55:58 2011 From: pch0317 at gmail.com (pch0317) Date: Wed, 23 Nov 2011 19:55:58 +0000 Subject: [Dovecot] Problem with Outlook 2010 Message-ID: <4ECD4FCE.4000400@gmail.com> Hi, Maybe this time somebody help. Thanks -------- Original Message -------- Subject: Problem with Outlook 2010 Date: Tue, 07 Jun 2011 20:20:56 +0100 From: pch0317 To: Dovecot Mailing List Hi, I use dovecot 2.0. I have problem with Outlook 2010. This application freezes for about 2 minutes when I move or delete messages. Other applications such as Outlook Express and Thunderbird work correctly. Has anyone had similar problem? Thanks From tss at iki.fi Wed Nov 23 22:09:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 22:09:18 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: Message-ID: <1322078958.24607.76.camel@hurina> On Wed, 2011-11-23 at 00:56 +0200, Timo Sirainen wrote: > > Hi, I found that doing a wildcard search with doveadm doesn't expand the %d > > global variable when doing a wildcard search like: > > > > Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain > > scope=subtree > > Iteration is supposed to iterate through all users.. > > > Is this a bug? > > > More like a missing feature I guess. I guess it could make it expand the %d if it's known.. hg version of v2.1 now supports this. From dovecot.user at seibercom.net Wed Nov 23 22:28:00 2011 From: dovecot.user at seibercom.net (Jerry) Date: Wed, 23 Nov 2011 15:28:00 -0500 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <20111123152800.6504a2ba@scorpio> On Wed, 23 Nov 2011 19:55:58 +0000 pch0317 articulated: > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work > correctly. > > Has anyone had similar problem? Seriously, do you have any log messages, etcetera to diagnose this problem with? -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From dovecot.user at seibercom.net Wed Nov 23 22:31:56 2011 From: dovecot.user at seibercom.net (Jerry) Date: Wed, 23 Nov 2011 15:31:56 -0500 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <20111123153156.0c02bb74@scorpio> On Wed, 23 Nov 2011 19:55:58 +0000 pch0317 articulated: > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work > correctly. You could start here to get some useful information as to what Outlook 2010 is doing. http://www.theemailadmin.com/2010/08/troubleshooting-outlook-2010-connections/ -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Wed Nov 23 22:57:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 22:57:50 +0200 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <4E754D10.5010404@babelmonkeys.de> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> Message-ID: <1322081870.24607.78.camel@hurina> On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: > Am 16.09.2011 14:40, schrieb Timo Sirainen: > > On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: > > > > Looks pretty good. Below are a few things I noticed. I could fix these > > myself next week also, or you can do them during weekend if you want > > to. :) > > > I decided to do it myself, hope this fixes all issues. Committed finally to http://hg.dovecot.org/dovecot-2.1 with some cleanups. From tss at iki.fi Wed Nov 23 23:02:30 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:02:30 +0200 Subject: [Dovecot] Can't delete sub-folder In-Reply-To: <4EBAFFD7.8020201@gmail.com> References: <4EBAFFD7.8020201@gmail.com> Message-ID: <1322082151.24607.79.camel@hurina> On Wed, 2011-11-09 at 22:33 +0000, pch0317 wrote: > I use thunderbird 8.0 as a client and dovecot 2.0.9 with imap protocol .. > When I try to delete 'parent' folder which contain 'messages only' > folder, error message appear: [ALREADYEXISTS] Mailbox has children, > delete them first. Ok, so I delete 'message only' folder first and try > to delete parent folder, but the same error appear: [ALREADYEXISTS] > Mailbox has children, delete them first. > When I list mailboxes from console I see inside 'parent' folder ".imap" > folder. When I delete .imap folder from console and then delete parent > folder from thunderbird, parent folder is deleted correctly. You can reproduce this? I tried with v2.0.16 and it works ok, so I have probably fixed this since v2.0.9. From stephan at rename-it.nl Wed Nov 23 23:07:14 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 23 Nov 2011 22:07:14 +0100 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <1322081870.24607.78.camel@hurina> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> <1322081870.24607.78.camel@hurina> Message-ID: <4ECD6082.4070406@rename-it.nl> On 11/23/2011 9:57 PM, Timo Sirainen wrote: > On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: >> Am 16.09.2011 14:40, schrieb Timo Sirainen: >>> On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: >>> >>> Looks pretty good. Below are a few things I noticed. I could fix these >>> myself next week also, or you can do them during weekend if you want >>> to. :) >>> >> I decided to do it myself, hope this fixes all issues. > Committed finally to http://hg.dovecot.org/dovecot-2.1 with some > cleanups. Great, the Pigeonhole ManageSieve implementation is now suddenly fully RFC compliant, without any effort on my part! :) Regards, Stephan. From tss at iki.fi Wed Nov 23 23:11:53 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:11:53 +0200 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <4ECD6082.4070406@rename-it.nl> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> <1322081870.24607.78.camel@hurina> <4ECD6082.4070406@rename-it.nl> Message-ID: <1322082713.24607.81.camel@hurina> On Wed, 2011-11-23 at 22:07 +0100, Stephan Bosch wrote: > On 11/23/2011 9:57 PM, Timo Sirainen wrote: > > On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: > >> Am 16.09.2011 14:40, schrieb Timo Sirainen: > >>> On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: > >>> > >>> Looks pretty good. Below are a few things I noticed. I could fix these > >>> myself next week also, or you can do them during weekend if you want > >>> to. :) > >>> > >> I decided to do it myself, hope this fixes all issues. > > Committed finally to http://hg.dovecot.org/dovecot-2.1 with some > > cleanups. > > Great, the Pigeonhole ManageSieve implementation is now suddenly fully > RFC compliant, without any effort on my part! :) I'm not actually sure about that :) The final replying is probably wrong, since ManageSieve supports sending it to client, unlike IMAP/POP3.. From elhijo at 0lim.net Wed Nov 23 23:13:20 2011 From: elhijo at 0lim.net (elhijo) Date: Wed, 23 Nov 2011 22:13:20 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: References: <4ECD00F6.4060908@0lim.net> (auto-added) <4ECD0C2D.8080901@0lim.net> Message-ID: <4ECD61F0.1010806@0lim.net> On 23/11/2011 16:29, Timo Sirainen wrote: > On 23.11.2011, at 17.07, David LEROUX wrote: > >> On 11/23/2011 03:53 PM, Frank Elsner wrote: >>> imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner >> Thanks, >> Thought that dovecot would be able to do that. > v2.1 has "imapc" backend, which you could use with dsync to do a migration, but for now it's probably easier to just use imapsync. > I'll stick to imapsync which seems to prefectly fit to my needs. Thanks for your help, David From stephan at rename-it.nl Wed Nov 23 23:15:00 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 23 Nov 2011 22:15:00 +0100 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <4EB9BD21.6060501@rename-it.nl> References: <1320791738.21919.350.camel@hurina> <4EB9BD21.6060501@rename-it.nl> Message-ID: <4ECD6254.2020606@rename-it.nl> On 11/9/2011 12:37 AM, Stephan Bosch wrote: > On 11/8/2011 11:35 PM, Timo Sirainen wrote: >> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz >> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig >> >> Here's the first beta release of Dovecot v2.1. This version has already >> been tested quite a lot, so I'm not expecting any major bugs. So please >> upgrade and see if you can find any problems. I'm optimistic about >> getting rc1 released this year and perhaps even v2.1.0. >> > > Ok, I guess it is time for a Pigeonhole release for Dovecot v2.1. I'm > a bit swamped this week, but next weekend I should be able to compose > one. For now you can use the dovecot-2.0-pigeonhole hg and patch it > with > http://hg.rename-it.nl/pigeonhole-0.2-dovecot-2.1-patches/file/tip/pigeonhole-0.2-dovecot-2.1.patch > to make it compile against v2.1. I have an update on this. Although there is no tarball release so far, I did start a separate repository with Pigeonhole (v0.3.x) for Dovecot v2.1: http://hg.rename-it.nl/dovecot-2.1-pigeonhole This means that the patch procedure mentioned above is no longer necessary. From this point onwards, new features are only implemented for Pigeonhole v0.3.x, unless a back-port can be achieved with little risk and effort. First in line is support for the editheader extension, which I expect to finish next week. Regards, Stephan. From ramiblanco at gmail.com Wed Nov 23 23:19:35 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 18:19:35 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322078958.24607.76.camel@hurina> References: <1322078958.24607.76.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > > hg version of v2.1 now supports this. > Does it need some extra configuration? I've upgraded to hg ver. of 2.1 and still the same error... Thanks for the quick response! -- Ramiro Blanco From tss at iki.fi Wed Nov 23 23:27:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:27:47 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> Message-ID: <1322083667.24607.82.camel@hurina> On Wed, 2011-11-23 at 18:19 -0300, Ramiro Blanco wrote: > 2011/11/23 Timo Sirainen : > > > > hg version of v2.1 now supports this. > > > > Does it need some extra configuration? No. > I've upgraded to hg ver. of 2.1 and still the same error... Does it still show "%d" in the error messages, or is it "" now? From ramiblanco at gmail.com Thu Nov 24 00:08:20 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 19:08:20 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322083667.24607.82.camel@hurina> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > > Does it still show "%d" in the error messages, or is it "" now? > It still shows "%d" Nov 23 18:42:59 auth-worker(729): Debug: ldap: iterate: base=vd=%d,dc=domain scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 23 18:42:59 auth-worker(729): Error: ldap(): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object -- Ramiro Blanco From tss at iki.fi Thu Nov 24 01:13:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:13:06 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> Message-ID: <1322089987.24607.84.camel@hurina> On Wed, 2011-11-23 at 19:08 -0300, Ramiro Blanco wrote: > 2011/11/23 Timo Sirainen : > > > > Does it still show "%d" in the error messages, or is it "" now? > > > It still shows "%d" > > Nov 23 18:42:59 auth-worker(729): Debug: ldap: iterate: > base=vd=%d,dc=domain scope=subtree Actually only the debug message was wrong. The expansion should have worked. Anyway, fixed the debug message: http://hg.dovecot.org/dovecot-2.1/rev/1fed3c1c166e From tss at iki.fi Thu Nov 24 01:21:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:21:04 +0200 Subject: [Dovecot] doveadm import assertion failed In-Reply-To: <87vcqax52a.fsf@algae.riseup.net> References: <87vcqax52a.fsf@algae.riseup.net> Message-ID: <1322090464.24607.87.camel@hurina> On Wed, 2011-11-23 at 09:54 -0500, Micah Anderson wrote: > Restoring a user's mailbox yesterday resulted in 'doveadm import' > panic'ing with an assertion failure and giving a backtrace: > > /usr/bin/doveadm import -u mdbox:/maildir/riseup.net/a//.daily.1/mdbox restored_from_backups/daily1 all > doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac > tion log sequence (3) So this path points to the backed up mailbox you're importing, not the destination mailbox? And you can reproduce this crash reliably? Could you send me these files: mailboxes/INBOX/dbox-Mails/dovecot.index mailboxes/INBOX/dbox-Mails/dovecot.index.log files? storage/dovecot.map.index storage/dovecot.map.index.log None of them contain anything sensitive about the user's mailbox. From tss at iki.fi Thu Nov 24 01:23:26 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:23:26 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECC44BD.3080806@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> <4ECC44BD.3080806@hardwarefreak.com> Message-ID: <1322090606.24607.89.camel@hurina> On Tue, 2011-11-22 at 18:56 -0600, Stan Hoeppner wrote: > Out of curiosity, what (or who) prompted the development of the > compressed mbox feature? Or was it that you wanted to do it for > maildir, and then figured you should for mbox as well? The > implementation seems to work ok. The instructions just seem a > bit...thin. :) I think it was one of my friends saying he can't start using Dovecot until it supports compressed mboxes. > >> mail_plugins: zlib > > > > Doesn't look like fts, fts_squat is enabled? > > Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So > 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins > had to be declared in a single line directive. I did this as part of my > troubleshooting when zlib wasn't working, thinking putting it on it's > own line may help--not. With v2.0 you can do: mail_plugins = $mail_plugins zlib mail_plugins = $mail_plugins fts fts_squat From tss at iki.fi Thu Nov 24 01:25:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:25:03 +0200 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <4ECC40FF.8020103@knutejohnson.com> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> <4ECC40FF.8020103@knutejohnson.com> Message-ID: <1322090703.24607.91.camel@hurina> On Tue, 2011-11-22 at 16:40 -0800, Knute Johnson wrote: > The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; > > !include_try /usr/share/dovecot/protocols.d/*.protocol > > That directory contains one file; > > dovecot-imapd.protocol > > and it contains one line; > > protocols = $protocols imap > > So I am assuming that it is adding imap to the protocols in the > environment variable $protocols somehow? Not environment variable $protocols, but the previous "protocols" setting's value in dovecot.conf. > I did get TBird to retrieve mail from port 993 using SSL however. Will > it work without a protocols = imaps? Yeah, there's no such thing as "imaps protocol". I should never have included it as a protocol in previous Dovecot versions either.. From tss at iki.fi Thu Nov 24 01:55:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:55:08 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4ECAD578.4060304@informatik.uni-freiburg.de> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> Message-ID: <1322092508.24607.93.camel@hurina> On Mon, 2011-11-21 at 23:49 +0100, Martin Preen wrote: > # /usr/sfw/bin/openssl engine > (pkcs11) PKCS #11 engine support > > >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); > >> ENGINE_set_default_ciphers(e); > >> > >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() > >> the crypto device gets used. I'm sure that this is not the whole story since > >> this only seems to affect the IMAP login. > > > > It should work for POP3 as well, all of the SSL code is shared. > > I couldn't find the EncryptUpdate call which has to be changed too > (due to the howto documents). Maybe some other call needs e patch. > But I don't know which. What EncryptUpdate?.. I've anyway added the engine init/deinit calls in your email to v2.1 hg. Lets hope it works :) At least it didn't break when I tried it with "dynamic" value (which is the only engine my OpenSSL supports). From tss at iki.fi Thu Nov 24 02:08:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:08:23 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1322092508.24607.93.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> <1322092508.24607.93.camel@hurina> Message-ID: <1322093304.24607.94.camel@hurina> On Thu, 2011-11-24 at 01:55 +0200, Timo Sirainen wrote: > What EncryptUpdate?.. I've anyway added the engine init/deinit calls in > your email to v2.1 hg. Lets hope it works :) At least it didn't break > when I tried it with "dynamic" value (which is the only engine my > OpenSSL supports). Oh, and in Solaris "pkcs11" value also works. But I don't know if it's actually using hardware crypto. From tss at iki.fi Thu Nov 24 02:25:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:25:17 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC4922B.5050000@goldey.net> References: <4EC4922B.5050000@goldey.net> Message-ID: <1322094317.24607.96.camel@hurina> On Wed, 2011-11-16 at 23:48 -0500, Mark Goldey wrote: > (gdb) bt full > #0 0x00000000 in ?? () > No symbol table info available. > #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, > args=0x82601c8) at auth-worker-client.c:405 Still doesn't make any sense.. Try what it logs with attached patch. And perhaps also try running via valgrind: service auth-worker { executable = /usr/bin/valgrind -q /usr/local/libexec/dovecot/auth -w } -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 800 bytes Desc: not available URL: From nanitous at xs4all.nl Thu Nov 24 02:44:44 2011 From: nanitous at xs4all.nl (Nanitous) Date: Thu, 24 Nov 2011 01:44:44 +0100 Subject: [Dovecot] Excluding some system users after configuration Message-ID: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> Dear readers, Consider an installed Postfix configuration (2.3.3) on CentOS. Now I want to install Dovecot (alas only version 1.0.7.7 is available) to provide IMAP services for a few system users, but not all. Most importantly the mail delivery by postfix for some of these system user /must/ not change after the installation of Dovecot. How can I prevent the installation of Dovecot to touch some specific system user accounts? Thanks in advance, /Twan From dovecot at knutejohnson.com Thu Nov 24 02:45:10 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Wed, 23 Nov 2011 16:45:10 -0800 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <1322090703.24607.91.camel@hurina> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> <4ECC40FF.8020103@knutejohnson.com> <1322090703.24607.91.camel@hurina> Message-ID: <4ECD9396.1050703@knutejohnson.com> On 11/23/2011 3:25 PM, Timo Sirainen wrote: > On Tue, 2011-11-22 at 16:40 -0800, Knute Johnson wrote: > >> The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; >> >> !include_try /usr/share/dovecot/protocols.d/*.protocol >> >> That directory contains one file; >> >> dovecot-imapd.protocol >> >> and it contains one line; >> >> protocols = $protocols imap >> >> So I am assuming that it is adding imap to the protocols in the >> environment variable $protocols somehow? > > Not environment variable $protocols, but the previous "protocols" > setting's value in dovecot.conf. > >> I did get TBird to retrieve mail from port 993 using SSL however. Will >> it work without a protocols = imaps? > > Yeah, there's no such thing as "imaps protocol". I should never have > included it as a protocol in previous Dovecot versions either.. > Thanks very much Timo! -- Knute Johnson From tss at iki.fi Thu Nov 24 02:46:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:46:56 +0200 Subject: [Dovecot] v2.1.rc1 released Message-ID: <1322095616.24607.108.camel@hurina> http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz.sig I'll probably release v2.1.0 pretty soon, unless people report some new bugs. I know v2.1 is already being used to serve mails to tens (or hundreds?) of thousands of users, so it should be pretty stable. Most of the changes since v2.1.beta1 have been for rather small issues. Many of the fixed bugs exist also in v2.0, but since they are so minor I didn't bother backporting the fixes. As new features since v2.1.beta1 there are the things I did also for v2.0.16, and: * ssl_client_cert/key: Proxying can now send SSL certificate to backend server * ssl_crypto_device for enabling hardware encryption * SCRAM-SHA-1 authentication mechanism support by Florian Zeitz * passdb/userdb checkpassword: Export all auth %variables to AUTH_* environment. * maildir_broken_filename_sizes setting for people upgrading from other maildir servers that wrote tons of broken S= values And as a reminder, here are the biggest new things since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now From jayw at interoceansystems.com Thu Nov 24 02:54:16 2011 From: jayw at interoceansystems.com (Jay Welch) Date: Wed, 23 Nov 2011 16:54:16 -0800 Subject: [Dovecot] Random Loss of Connection with Thunderbird and Dovecot Message-ID: <4ECD95B8.8070807@interoceansystems.com> Hi, Dovecot Version: 1.2.16 OS: CENTOS 5.5 I am having issues with Thunderbird connecting to Dovecot at random times. Usually an error comes up and states that Thunderbird cannot connect to the server. The only way I have been able to fix this is reset/repair the network connection or reboot the computer. Is there any parameters I can change in Dovecot to fix this issue? Does anyone know what is likely going on? I have been reading forums and I cannot find anything. Thanks From tss at iki.fi Thu Nov 24 02:51:48 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:51:48 +0200 Subject: [Dovecot] Excluding some system users after configuration In-Reply-To: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> References: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> Message-ID: <1322095908.24607.110.camel@hurina> On Thu, 2011-11-24 at 01:44 +0100, Nanitous wrote: > Dear readers, > > Consider an installed Postfix configuration (2.3.3) on CentOS. > Now I want to install Dovecot (alas only version 1.0.7.7 is available) to provide IMAP services for a few system users, but not all. Most importantly the mail delivery by postfix for some of these system user /must/ not change after the installation of Dovecot. > > How can I prevent the installation of Dovecot to touch some specific system user accounts? As long as you don't configure Postfix to deliver mails via Dovecot's "deliver", there is no change to how mails are delivered. If you want to restrict IMAP access to only some users, you can do it with pam_listfile: http://wiki.dovecot.org/Authentication/RestrictAccess From tss at iki.fi Thu Nov 24 02:56:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:56:07 +0200 Subject: [Dovecot] Random Loss of Connection with Thunderbird and Dovecot In-Reply-To: <4ECD95B8.8070807@interoceansystems.com> References: <4ECD95B8.8070807@interoceansystems.com> Message-ID: <1322096168.24607.114.camel@hurina> On Wed, 2011-11-23 at 16:54 -0800, Jay Welch wrote: > Dovecot Version: 1.2.16 > > > I am having issues with Thunderbird connecting to Dovecot at random > times. Usually an error comes up and states that Thunderbird cannot > connect to the server. The only way I have been able to fix this is > reset/repair the network connection or reboot the computer. Is there any > parameters I can change in Dovecot to fix this issue? Does anyone know > what is likely going on? I have been reading forums and I cannot find > anything. Dovecot always logs the reason for logout, e.g.: > Nov 24 02:53:24 imap(tss): Info: Connection closed in=0 out=291 This means that the connection got disconnected (i.e. Dovecot didn't do the disconnection) > Nov 24 02:52:47 imap(tss): Info: Disconnected: Logged out in=8 out=334 And this means that the client used a LOGOUT command. and so on. Find the log entries about the times when you're having problems. Dovecot logs about every single connection it sees. If you don't see a log message about some connection, it means that Dovecot never saw it, and the problem is outside Dovecot. I'd first look into any antivirus/software firewalls you've installed. They're usually the reason for connection problems with Windows. From ramiblanco at gmail.com Thu Nov 24 04:00:28 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 23:00:28 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322089987.24607.84.camel@hurina> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > Actually only the debug message was wrong. The expansion should have > worked. Anyway, fixed the debug message: > http://hg.dovecot.org/dovecot-2.1/rev/1fed3c1c166e Still failing: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users Now "%d" it's empty, it shows "vd=" instead of "vd=%d": Nov 23 22:50:46 auth-worker(26556): Debug: ldap: iterate: base=vd=,dc=uvq,dc=edu,dc=ar scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 23 22:50:46 auth-worker(26556): Error: ldap(): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object It is supposed to parse domain part from input in "doveadm search *@domain ..", right? -- Ramiro Blanco From tss at iki.fi Thu Nov 24 04:20:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 04:20:25 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> Message-ID: <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> On 24.11.2011, at 4.00, Ramiro Blanco wrote: > Now "%d" it's empty, it shows "vd=" instead of "vd=%d": > > It is supposed to parse domain part from input in "doveadm search > *@domain ..", right? Too tired to test search myself now, but this is what I used to test it myself: doveadm user '*@domain' If that works, then the problem is with doveadm and it needs more patching.. From ramiblanco at gmail.com Thu Nov 24 04:47:24 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 23:47:24 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> Message-ID: 2011/11/23 Timo Sirainen : > Too tired to test search myself now, but this is what I used to test it myself: > You've done too much already! > doveadm user '*@domain' > > If that works, then the problem is with doveadm and it needs more patching.. That works just fine. I guess it needs some patching then. -- Ramiro Blanco From robert at schetterer.org Thu Nov 24 08:05:47 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 24 Nov 2011 07:05:47 +0100 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <4ECDDEBB.6000608@schetterer.org> Am 23.11.2011 20:55, schrieb pch0317: > Hi, > > Maybe this time somebody help. > > Thanks > > -------- Original Message -------- > Subject: Problem with Outlook 2010 > Date: Tue, 07 Jun 2011 20:20:56 +0100 > From: pch0317 > To: Dovecot Mailing List > > > > Hi, > > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work correctly. > > Has anyone had similar problem? > > > Thanks > > no problem with any outlook version what are your logs about the problem? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From oni-neko at gmx.net Thu Nov 24 09:10:27 2011 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Thu, 24 Nov 2011 08:10:27 +0100 Subject: [Dovecot] seting acl for master user? In-Reply-To: <1321636922.11489.46.camel@hurina> References: <20111118133428.63140@gmx.net> <1321636922.11489.46.camel@hurina> Message-ID: <20111124071027.28190@gmx.net> thank you very much for the reply, it works now, after i removed all the acl-stuff. =) i don't really know why it didn't work directly after i set up the master user (but before trying around with acls), but hey, I'm not complaning that it's working now =) thanks again and sorry for the late re =) bye sil -------- Original-Nachricht -------- > Datum: Fri, 18 Nov 2011 19:22:02 +0200 > Von: Timo Sirainen > An: oni-neko at gmx.net > CC: dovecot at dovecot.org > Betreff: Re: [Dovecot] seting acl for master user? > On Fri, 2011-11-18 at 14:34 +0100, oni-neko at gmx.net wrote: > > > what i don't get is the acl-setting, so that the master-user can > > access all the other users mails and folders. > .. > > in my case I only want the master user (called cyrus) to be able to do > > everything with/in all mailboxes, so I made a > > file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". > > Unfortunately currently there is no default ACL file. The above would > create an ACL file for a mailbox called "default". I'll need to get the > default ACLs implemented some day.. > > But if you don't need ACLs for anything else, you can simply disable the > ACL plugin and the master user will have access to everything. If you do > need ACLs, there are some kludges you can still do. > > -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de From preen at informatik.uni-freiburg.de Thu Nov 24 10:45:26 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Thu, 24 Nov 2011 09:45:26 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1322092508.24607.93.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> <1322092508.24607.93.camel@hurina> Message-ID: <4ECE0426.7020805@informatik.uni-freiburg.de> Timo Sirainen wrote: > On Mon, 2011-11-21 at 23:49 +0100, Martin Preen wrote: > >> # /usr/sfw/bin/openssl engine >> (pkcs11) PKCS #11 engine support >> >>>> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >>>> ENGINE_set_default_ciphers(e); >>>> >>>> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >>>> the crypto device gets used. I'm sure that this is not the whole story since >>>> this only seems to affect the IMAP login. >>> It should work for POP3 as well, all of the SSL code is shared. >> I couldn't find the EncryptUpdate call which has to be changed too >> (due to the howto documents). Maybe some other call needs e patch. >> But I don't know which. > > What EncryptUpdate?.. I've anyway added the engine init/deinit calls in > your email to v2.1 hg. Lets hope it works :) At least it didn't break > when I tried it with "dynamic" value (which is the only engine my > OpenSSL supports). Probably I'm wrong (I have no experience with SSL programming). I thught the EncryptUpdate was necessary for the encoding of the SSL data stream. But maybe there has to be a link between engine initialization and the SSL contexts ? Martin --------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/~preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6337 bytes Desc: S/MIME Cryptographic Signature URL: From olli.raisanen at telemail.fi Thu Nov 24 11:22:20 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-1?q?R=E4is=E4nen?=) Date: Thu, 24 Nov 2011 11:22:20 +0200 Subject: [Dovecot] How to create home directories for virtual users? Message-ID: <201111241122.21131.olli.raisanen@telemail.fi> Hello, I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home directory should not be the same as mail directory. When I create new user accounts with PostfixAdmin the mail directory is not a problem but is there some smart way to create the home directory as well (locating it for instance in '/srv/vmail/%d/%n/home')? Regards, Olli From pw at wk-serv.de Thu Nov 24 12:04:25 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 11:04:25 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <20111122104547.GA10680@dibs.tanso.net> References: <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> Message-ID: <1d5f057161f05918ab010a979c455d9c@wk-serv.de> On Tue, 22 Nov 2011 11:45:47 +0100, Jan-Frode Myklebust wrote: > Ah, then Timo's reply was right. He suggested you do the > lmtp-deliveries > to the same server that you would send you imap-user to. You can do > this > trough dovecot director and lmtp-proxying. > > So instead of: > > lmtp:unix:private/dovecot-lmtp > > you should use: > > lmtp:tcp:1.2.3.4:24 > > where 1.2.3.4 would be the Dovecot LMTP proxy that proxies to the > same > machine as you would use for imap for this particular recipient. I see. So as far as I understood: - I set up a new server as LMTP proxy for my two MX 10 to connect to - the proxy redirects to my backend imap servers which will then store the mails on my shared storage and the index files to a local disk (so I have to enable LMTP additionally to enable this servers to store the mails) - I set up a frontend imap server for my users to connect to which will redirect them to the backend servers Am I right so far? Patrick From johannes at sipsolutions.net Thu Nov 24 15:11:06 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 14:11:06 +0100 Subject: [Dovecot] modules directory Message-ID: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Hi! I'm a little confused with how plugin loading works in dovecot 2, I can't seem to make it load from $(moduledir)/imap/. Is that intended to work? Just a configuration issue? Reason I ask is that I ported my antispam plugin (I know there's a fork, but I still like mine better) to dovecot 2 and the default "make install" no longer loads properly with just "mail_plugins = antispam". So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, but I still see subdirectories for at least auth. Should I install into just $(moduledir)/ now? johannes From pw at wk-serv.de Thu Nov 24 15:35:55 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 14:35:55 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <20111124104709.GB24297@dibs.tanso.net> References: <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> Message-ID: <4ECE483B.5090208@wk-serv.de> Jan-Frode Myklebust schrieb: > Yes, and in ascii-art this becomes :-) > > MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > \---(lmtp)---> backend-server2 > > > IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > \--(imap) --> backend-server2 I failed creating a nice ascii view ;-) However, I have some more questions: - What happens if the MTA accepted a message but the backend-server fails and is unable to store it? - What about load-balancing and/or failover? Maybe all users on backend-server1 are fetching their mails causing high load while backend-server2 is idle. Is it possible to specify more than one backend-server IPs as I can do for the database hosts for userdb queries? From pw at wk-serv.de Thu Nov 24 15:38:43 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 14:38:43 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <20111124104709.GB24297@dibs.tanso.net> References: <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> Message-ID: <4ECE48E3.1070806@wk-serv.de> Jan-Frode Myklebust schrieb: > MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > \---(lmtp)---> backend-server2 > > > IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > \--(imap) --> backend-server2 In this scenario it should be possible to use a non cluster filesystem to store my mails, isn't it? Patrick From tss at iki.fi Thu Nov 24 16:02:24 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 16:02:24 +0200 Subject: [Dovecot] modules directory In-Reply-To: <1322140266.5366.8.camel@jlt3.sipsolutions.net> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Message-ID: On 24.11.2011, at 15.11, Johannes Berg wrote: > I'm a little confused with how plugin loading works in dovecot 2, I > can't seem to make it load from $(moduledir)/imap/. Is that intended to > work? Just a configuration issue? > > Reason I ask is that I ported my antispam plugin (I know there's a fork, > but I still like mine better) to dovecot 2 and the default "make > install" no longer loads properly with just "mail_plugins = antispam". > > So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, > but I still see subdirectories for at least auth. There's no /imap/ directory anymore, because it was annoying having to create symlinks to most of the plugins. Then with v2.0 there came new binaries and it wouldn't have made much sense adding lots of more directories with lots of more symlinks.. > Should I install into just $(moduledir)/ now? Yep. From janfrode at tanso.net Thu Nov 24 16:21:55 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 24 Nov 2011 15:21:55 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <4ECE48E3.1070806@wk-serv.de> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE48E3.1070806@wk-serv.de> Message-ID: <20111124142155.GA28600@dibs.tanso.net> On Thu, Nov 24, 2011 at 02:38:43PM +0100, Patrick Westenberg wrote: > Jan-Frode Myklebust schrieb: > > >MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > > \---(lmtp)---> backend-server2 > > > > > >IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > > \--(imap) --> backend-server2 > > In this scenario it should be possible to use a non cluster filesystem > to store my mails, isn't it? Yes, that should work fine -- assuming you have acceptable uptime on the backend server that is hosting the messages. You also need to configure a user<->backend-server mapping trough proxy extrafield to make sure that the user lands on the correct server: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy -jf From johannes at sipsolutions.net Thu Nov 24 16:13:40 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 15:13:40 +0100 Subject: [Dovecot] modules directory In-Reply-To: References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Message-ID: <1322144020.5366.20.camel@jlt3.sipsolutions.net> On Thu, 2011-11-24 at 16:02 +0200, Timo Sirainen wrote: > On 24.11.2011, at 15.11, Johannes Berg wrote: > > > I'm a little confused with how plugin loading works in dovecot 2, I > > can't seem to make it load from $(moduledir)/imap/. Is that intended to > > work? Just a configuration issue? > > > > Reason I ask is that I ported my antispam plugin (I know there's a fork, > > but I still like mine better) to dovecot 2 and the default "make > > install" no longer loads properly with just "mail_plugins = antispam". > > > > So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, > > but I still see subdirectories for at least auth. > > There's no /imap/ directory anymore, because it was annoying having to > create symlinks to most of the plugins. Then with v2.0 there came new > binaries and it wouldn't have made much sense adding lots of more > directories with lots of more symlinks.. > > > Should I install into just $(moduledir)/ now? > > Yep. Alright, thanks! Do you remember if that would be compatible with older versions as well, or do those require it being in /imap/? johannes From tss at iki.fi Thu Nov 24 16:15:58 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 16:15:58 +0200 Subject: [Dovecot] modules directory In-Reply-To: <1322144020.5366.20.camel@jlt3.sipsolutions.net> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> <1322144020.5366.20.camel@jlt3.sipsolutions.net> Message-ID: <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> On 24.11.2011, at 16.13, Johannes Berg wrote: >>> Should I install into just $(moduledir)/ now? >> >> Yep. > > Alright, thanks! Do you remember if that would be compatible with older > versions as well, or do those require it being in /imap/? What changed was that the default mail_plugin_dir changed from being $moduledir/imap to just plain $moduledir. So older versions require it being in imap/ by default. From johannes at sipsolutions.net Thu Nov 24 16:19:41 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 15:19:41 +0100 Subject: [Dovecot] modules directory In-Reply-To: <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> <1322144020.5366.20.camel@jlt3.sipsolutions.net> <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> Message-ID: <1322144381.5366.21.camel@jlt3.sipsolutions.net> On Thu, 2011-11-24 at 16:15 +0200, Timo Sirainen wrote: > On 24.11.2011, at 16.13, Johannes Berg wrote: > > >>> Should I install into just $(moduledir)/ now? > >> > >> Yep. > > > > Alright, thanks! Do you remember if that would be compatible with older > > versions as well, or do those require it being in /imap/? > > What changed was that the default mail_plugin_dir changed from being > $moduledir/imap to just plain $moduledir. So older versions require it > being in imap/ by default. Ok, thanks. I'll find a way to make it depend on the version. johannes From janfrode at tanso.net Thu Nov 24 16:33:21 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 24 Nov 2011 15:33:21 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <4ECE483B.5090208@wk-serv.de> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> Message-ID: <20111124143321.GB28600@dibs.tanso.net> On Thu, Nov 24, 2011 at 02:35:55PM +0100, Patrick Westenberg wrote: > > - What happens if the MTA accepted a message but the backend-server > fails and is unable to store it? Your MTA will be notified about the failure trough SMTP/LMTP error codes, and retry if it was a temp-failure. > - What about load-balancing and/or failover? > Maybe all users on backend-server1 are fetching their mails causing > high load while backend-server2 is idle. > Is it possible to specify more than one backend-server IPs as I can > do for the database hosts for userdb queries? We use a clusterfs (so it's not very important which backend server the user is directed to), and have configured the dovecot director with: director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 passdb { args = proxy=y nopassword=y driver = static } which gives us loadbalancing of new logins, and also failover should a backend server fail. If you want to work with at local index and maildirs on NFS, you'll need to configure something smarter for selecting backend server. Maybe use proxy extrafield to specify a host for the user, and point this at a failover IP that will hang on the preferred server by default, but fail over to another working server if the preferred server fails.. http://wiki2.dovecot.org/PasswordDatabase/ExtraField -jf From simon.brereton at buongiorno.com Thu Nov 24 16:42:49 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 24 Nov 2011 09:42:49 -0500 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111241122.21131.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: On Nov 24, 2011 4:22 AM, "Olli R?is?nen" wrote: > > Hello, > > I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. > Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > directory should not be the same as mail directory. When I create new user > accounts with PostfixAdmin the mail directory is not a problem but is there > some smart way to create the home directory as well (locating it for instance > in '/srv/vmail/%d/%n/home')? Why do you need a home directory for virtual users? Simon From JTRUTWIN at CSBSJU.EDU Thu Nov 24 16:46:57 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Thu, 24 Nov 2011 14:46:57 +0000 Subject: [Dovecot] POP - preventing re-download with server move Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> Hi, Curious if anyone has any good suggestions to handle this. I try to get my hosting users to use IMAP but most use Outhouse and the IMAP support is somewhat annoying for many so I typically have them change their settings when using POP to leave copies on the server. That way when they get the inevitable virus or wanna use webmail they have a backup copy and webmail sees their saved mail. So now I have to move my sites to a new server and I'm trying to avoid having all these messages re-downloaded by POP clients. Some of the sites I've moved with low email usage have just dealt with the re-download and cleaned up afterwards. I have a few other accounts though that I'm more concerned about that have massive amounts of stored email. The mail is stored in Maildirs - a sample file: 1280409166.23580.foo:2,S If I move this to server "bar" would renaming the file to 1280409166.23580.bar:2,S prevent this somehow? Old server uses qmail-pop, new server uses dovecot 2.0. Thanks for any tips, Josh From robert at schetterer.org Thu Nov 24 17:07:44 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 24 Nov 2011 16:07:44 +0100 Subject: [Dovecot] POP - preventing re-download with server move In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> Message-ID: <4ECE5DC0.4000503@schetterer.org> Am 24.11.2011 15:46, schrieb Trutwin, Joshua: > Hi, > > Curious if anyone has any good suggestions to handle this. > > I try to get my hosting users to use IMAP but most use Outhouse and > the IMAP support is somewhat annoying for many so I typically have > them change their settings when using POP to leave copies on the > server. That way when they get the inevitable virus or wanna use > webmail they have a backup copy and webmail sees their saved mail. > > So now I have to move my sites to a new server and I'm trying to > avoid having all these messages re-downloaded by POP clients. Some > of the sites I've moved with low email usage have just dealt with the > re-download and cleaned up afterwards. I have a few other accounts > though that I'm more concerned about that have massive amounts of > stored email. > > The mail is stored in Maildirs - a sample file: 1280409166.23580.foo:2,S > > If I move this to server "bar" would renaming the file to > 1280409166.23580.bar:2,S prevent this somehow? > > Old server uses qmail-pop, new server uses dovecot 2.0. > > Thanks for any tips, > > Josh perhaps you can do it i.e like this use imapsync with diff between the two servers then block pop3 with firewall for public on the old one, run a last sync session, change dns and/or ip etc , users use new server i think this is the most secured way, i did it like this the problem by redownload may stay cause the pop3 clients have their own list of mail downloaded etc build by i.e with some one hashes i.e. http://kb.mozillazine.org/Popstate.dat and i dont know some way to press them to download anything -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Thu Nov 24 17:37:53 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 17:37:53 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: On 24.11.2011, at 16.42, Simon Brereton wrote: >> Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > > Why do you need a home directory for virtual users? It says right there in the wiki. From joseba.torre at ehu.es Thu Nov 24 17:38:55 2011 From: joseba.torre at ehu.es (Joseba Torre) Date: Thu, 24 Nov 2011 16:38:55 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: <4ECE650F.2070700@ehu.es> El 24/11/11 15:42, Simon Brereton escribi?: > On Nov 24, 2011 4:22 AM, "Olli R?is?nen" wrote: >> >> Hello, >> >> I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. >> Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home >> directory should not be the same as mail directory. When I create new user >> accounts with PostfixAdmin the mail directory is not a problem but is > there >> some smart way to create the home directory as well (locating it for > instance >> in '/srv/vmail/%d/%n/home')? > > Why do you need a home directory for virtual users? For storing files wich are not mails. Sieve rules, subscription files and so on. Aaaaaaaaaaaagur. From mlists at edicom.eu Thu Nov 24 18:11:11 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 17:11:11 +0100 Subject: [Dovecot] Problem with lmtp proxy Message-ID: <201111241711.11504.mlists@edicom.eu> Hello everyone, I have set up postfix to deliver mails to dovecot (2.0.16) using LMTP. On the other hand, I've successfully configured the IMAP proxy setting in dovecot in order to be able to distribute mailboxes among different servers. I wanted to do the same proxy at LMTP level, but it's not working. If I put lmtp_proxy = no, then everything works ok (assuming the mailbox is local), but when I set lmtp_proxy = yes then the user is not found when deliverying the message via LMTP, so the mail remains in the postfix queue. Users are validated through active directory. However, this AD hasn't the SFU installed, thus its LDAP schema doesn't provide me with the required uid, gid, etc. To solve this, I have winbind configured in the system, so I'm doing this: - To validate users *and* be able to set the proxy extra fields, I use ldap as "passdb" (I believe it's not possible to use proxies with PAM). I'm using some LDAP field to store the host that has the user's mailbox. - In order to get the user account data that is not available in the AD, I use passwd as "userdb". I know that when using LMTP with proxy, a passdb needs to be configured. I assume this is needed for the service to lookup for the appropriate host to send the message. So, initially I set ldap as the passdb for LMTP, but just because it didn't work I put both ldap and passwd, and even a userdb (passwd), but the problems remain the same. Anyway the ldap should be the right one as it is there where I have the host information for each user. I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! This is my current config: ************* dovecot -n ************** # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37-gentoo-edicom-1104 x86_64 Gentoo Base System release 1.12.14 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_username_format = %n auth_verbose = yes base_dir = /var/run/dovecot/ listen = * lmtp_proxy = yes login_trusted_networks = 127.0.0.1 mail_debug = yes mail_location = maildir:~/maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at domain.com protocols = imap pop3 sieve lmtp quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = root mode = 0600 user = root } } service imap { vsz_limit = 512 M } service lmtp { inet_listener lmtp { address = 192.168.0.90 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } ssl_cert = , res_matched: <> dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_free_request (origid 1, msgid 1) dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_parse_result dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_msgfree dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_result ld 0x16518d0 msgid -1 dovecot [2011-11-24 15:53:50] [err] auth: Error: wait4msg ld 0x16518d0 msgid -1 (timeout 0 usec) dovecot [2011-11-24 15:53:50] [err] auth: Error: wait4msg continue ld 0x16518d0 msgid -1 all 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Connections: dovecot [2011-11-24 15:53:50] [err] auth: Error: * host: domain.com port: 389 (default) dovecot [2011-11-24 15:53:50] [err] auth: Error: refcnt: 1 status: Connected dovecot [2011-11-24 15:53:50] [err] auth: Error: last used: Thu Nov 24 15:53:50 2011 dovecot [2011-11-24 15:53:50] [err] auth: Error: dovecot [2011-11-24 15:53:50] [err] auth: Error: dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Outstanding Requests: dovecot [2011-11-24 15:53:50] [err] auth: Error: Empty dovecot [2011-11-24 15:53:50] [err] auth: Error: ld 0x16518d0 request count 0 (abandoned 0) dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Response Queue: dovecot [2011-11-24 15:53:50] [err] auth: Error: Empty dovecot [2011-11-24 15:53:50] [err] auth: Error: ld 0x16518d0 response count 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_chkResponseList ld 0x16518d0 msgid -1 all 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_chkResponseList returns ld 0x16518d0 NULL dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_int_select dovecot [2011-11-24 15:53:50] [info] lmtp(10506): Disconnect from local: Client quit (in reset) **************************************************** From tom at whyscream.net Thu Nov 24 18:13:47 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 24 Nov 2011 17:13:47 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111241122.21131.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: <4ECE6D3B.6020208@whyscream.net> On 24-11-11 10:22, Olli R?is?nen wrote: > Hello, > > I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. > Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > directory should not be the same as mail directory. When I create new user > accounts with PostfixAdmin the mail directory is not a problem but is there > some smart way to create the home directory as well (locating it for instance > in '/srv/vmail/%d/%n/home')? > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail as mailstore? Otherwise, postfixadmin has a post-create hook that is able to execute arbitrary scripts, creating homedirs should be among the included examples. -- Tom From mlists at edicom.eu Thu Nov 24 18:27:38 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 17:27:38 +0100 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241711.11504.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> Message-ID: <201111241727.38458.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 17:11:11 Miguel Tormo escribi?: > dn = cn=binduser,cn=Users,dc=edicom,dc=es Obviously this part should be "dn = cn=binduser,cn=Users,dc=domain,dc=com" to be coherent with the rest. I replace the real domain and user names used, but forgot this part :S. From ecasarero at gmail.com Thu Nov 24 18:30:03 2011 From: ecasarero at gmail.com (Eduardo Casarero) Date: Thu, 24 Nov 2011 13:30:03 -0300 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241727.38458.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> <201111241727.38458.mlists@edicom.eu> Message-ID: 2011/11/24 Miguel Tormo > El Jueves, 24 de Noviembre de 2011 17:11:11 Miguel Tormo escribi?: > > dn = cn=binduser,cn=Users,dc=edicom,dc=es > > Obviously this part should be "dn = cn=binduser,cn=Users,dc=domain,dc=com" > to be coherent with the rest. I replace the real domain and user names > used, but forgot this part :S. > > From tss at iki.fi Thu Nov 24 18:58:40 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 18:58:40 +0200 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241711.11504.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> Message-ID: <1322153921.24607.124.camel@hurina> On Thu, 2011-11-24 at 17:11 +0100, Miguel Tormo wrote: > I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! The problem is: > ************** cat dovecot-ldap.conf.ext ************** > auth_bind = yes auth_bind=yes requires a user authentication, but LMTP of course can't authenticate a user. So it fails: > dovecot [2011-11-24 15:53:50] [debug] auth: Debug: password(myuser): passdb doesn't support credential lookups Either try to get auth_bind=no working, or I think you can also set up a separate passdb for lmtp: protocol lmtp { passdb { driver = ldap args = some-other-ldap.conf } } And in this other ldap.conf have auth_bind=no, and possibly return password field always as something like "foo". From mlists at edicom.eu Thu Nov 24 19:26:07 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 18:26:07 +0100 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <1322153921.24607.124.camel@hurina> References: <201111241711.11504.mlists@edicom.eu> <1322153921.24607.124.camel@hurina> Message-ID: <201111241826.07847.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 17:58:40 Timo Sirainen escribi?: > On Thu, 2011-11-24 at 17:11 +0100, Miguel Tormo wrote: > > I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! > > The problem is: > > > ************** cat dovecot-ldap.conf.ext ************** > > auth_bind = yes > > auth_bind=yes requires a user authentication, but LMTP of course can't > authenticate a user. So it fails: > > > dovecot [2011-11-24 15:53:50] [debug] auth: Debug: password(myuser): passdb doesn't support credential lookups > > Either try to get auth_bind=no working, or I think you can also set up a > separate passdb for lmtp: > > protocol lmtp { > passdb { > driver = ldap > args = some-other-ldap.conf > } > } > > And in this other ldap.conf have auth_bind=no, and possibly return > password field always as something like "foo". > I just configured your suggestion (set a separate passdb for lmtp with ldap and auth_bind = no ) and it works perfectly. Thank you very much! From lists at luigirosa.com Thu Nov 24 20:04:12 2011 From: lists at luigirosa.com (Luigi Rosa) Date: Thu, 24 Nov 2011 19:04:12 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322095616.24607.108.camel@hurina> References: <1322095616.24607.108.camel@hurina> Message-ID: <4ECE871C.3000804@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen said the following on 24/11/11 01:46: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz Works as advertised. I noticed this warining: Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < 4096), because of service auth { client_limit } What does exactly means? The server we are talking about is my home server, a CentOS/64 with a dozen of mailboxes. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Computers make excellent and efficient servants, but I have no wish to serve under them. Captain, a starship also runs on loyalty to one man. And nothing can replace it or him. --Spock, "The Ultimate Computer" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7OhxwACgkQ3kWu7Tfl6ZSJ0gCeIaS5ksyVHY3O9qOSCEyWUH2V By8AoMbUNSHBBc4102vScrozTOOkOeo+ =AZaL -----END PGP SIGNATURE----- From mlists at edicom.eu Thu Nov 24 20:23:00 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 19:23:00 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <4ECE871C.3000804@luigirosa.com> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> Message-ID: <201111241923.00215.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 19:04:12 Luigi Rosa escribi?: > Timo Sirainen said the following on 24/11/11 01:46: > > > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz > > Works as advertised. > > I noticed this warining: > > Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < > 4096), because of service auth { client_limit } > > What does exactly means? > > > The server we are talking about is my home server, a CentOS/64 with a dozen of > mailboxes. > The fd limit is the maximum number of concurrently open file descriptors per process. It can be configured on a per-user basis. You can check the actual value running 'ulimit -n'. It can be increased up to the value set in /proc/sys/fs/file-max. You can set it editing /etc/security/limits.conf. What the above comment means is that under max load the auth service could need up to 4096 file descriptors open, that's more than the default in most linux systems (1024). However, in your case with just a dozen of mailboxes I don't think you'll need to change it. From lists at luigirosa.com Thu Nov 24 20:29:41 2011 From: lists at luigirosa.com (Luigi Rosa) Date: Thu, 24 Nov 2011 19:29:41 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111241923.00215.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> Message-ID: <4ECE8D15.6050200@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Miguel Tormo said the following on 24/11/11 19:23: > The fd limit is the maximum number of concurrently open file descriptors > per process. It can be configured on a per-user basis. You can check the > actual value running 'ulimit -n'. It can be increased up to the value set > in /proc/sys/fs/file-max. You can set it editing > /etc/security/limits.conf. What the above comment means is that under max > load the auth service could need up to 4096 file descriptors open, that's > more than the default in most linux systems (1024). > > However, in your case with just a dozen of mailboxes I don't think you'll > need to change it. Thank you for the explaination. Ciao, luigi - -- / +--[Luigi Rosa]-- \ A woman was in love with fourteen soldiers. It was clearly platoonic. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7OjRUACgkQ3kWu7Tfl6ZQXAwCgyD6LAUYQX7/x1PIFda5CoFjx IYsAniqFYvdk6y982XjvWsWc4G5t9zEC =70Em -----END PGP SIGNATURE----- From olli.raisanen at telemail.fi Thu Nov 24 21:23:33 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-1?q?R=E4is=E4nen?=) Date: Thu, 24 Nov 2011 21:23:33 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <4ECE6D3B.6020208@whyscream.net> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> Message-ID: <201111242123.34156.olli.raisanen@telemail.fi> Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: > > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail > as mailstore? Well, yes - at least the home directory will be created that way. Still, PostfixAdmin will not store the path in the maibox table home field. Perhaps it is enough to include the path in the Dovecot user_query? > Otherwise, postfixadmin has a post-create hook that is able to execute > arbitrary scripts, creating homedirs should be among the included > examples. If you are referring to maildir_name_hook in config.inc.php script, no, homedirs are not included (in Postfix Admin version 2.3.4). Thank you anyway, Tom! I must consider your advice. Olli From tss at iki.fi Thu Nov 24 21:25:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 21:25:47 +0200 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111241923.00215.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> Message-ID: <1322162747.24607.129.camel@hurina> On Thu, 2011-11-24 at 19:23 +0100, Miguel Tormo wrote: > The fd limit is the maximum number of concurrently open file descriptors per process. It can be configured on a per-user basis. > You can check the actual value running 'ulimit -n'. It can be increased up to the value set in /proc/sys/fs/file-max. You can set it editing /etc/security/limits.conf. I'm not sure if changing /etc/security/limits.conf helps. It's probably only used by PAM when user logs in, so if Dovecot is started in system bootup it's unlikely to have been even read yet. Also I think some OSes override the limits in /etc/init.d/ scripts. Of course, I could be completely wrong in all of the above, I haven't really tested any of it :) > What the above comment means is that under max load the auth service could need up to 4096 file descriptors open, that's more than the default in most linux systems (1024). I wonder if there's a good reason for why auth default should be that high. Probably in earlier times imap/pop3 processes just weren't disconnecting early enough from auth. So I guess I'll just drop it back to default and this warning would go away in most systems. I had started to get a little bit annoyed at that warning myself also. From tss at iki.fi Thu Nov 24 21:29:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 21:29:46 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111242123.34156.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> <201111242123.34156.olli.raisanen@telemail.fi> Message-ID: <1322162990.24607.131.camel@hurina> On Thu, 2011-11-24 at 21:23 +0200, Olli R?is?nen wrote: > Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: > > > > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail > > as mailstore? > > Well, yes - at least the home directory will be created that way. Still, > PostfixAdmin will not store the path in the maibox table home field. Perhaps > it is enough to include the path in the Dovecot user_query? It would be better if you used PostfixAdmin's "maildir" field as the home directory, and then mail_location=~/Maildir. Like the sql queries being: password_query = \ SELECT username AS user, password, \ CONCAT('/home/vmail/', maildir) AS userdb_home, \ CONCAT('*:bytes=', quota) AS userdb_quota_rule \ FROM mailbox \ WHERE username = '%u' and ACTIVE = '1' user_query = \ SELECT CONCAT('/home/vmail/', maildir) AS home, \ CONCAT('*:bytes=', quota) AS quota_rule \ FROM mailbox \ WHERE username = '%u' AND active = '1' I used these in another PostfixAdmin server's install. From tom at whyscream.net Thu Nov 24 21:50:39 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 24 Nov 2011 20:50:39 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <1322162990.24607.131.camel@hurina> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> <201111242123.34156.olli.raisanen@telemail.fi> <1322162990.24607.131.camel@hurina> Message-ID: <4ECEA00F.9040306@whyscream.net> On 24-11-11 20:29, Timo Sirainen wrote: > On Thu, 2011-11-24 at 21:23 +0200, Olli R?is?nen wrote: >> Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: >>> >>> How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail >>> as mailstore? >> >> Well, yes - at least the home directory will be created that way. Still, >> PostfixAdmin will not store the path in the maibox table home field. Perhaps >> it is enough to include the path in the Dovecot user_query? > > It would be better if you used PostfixAdmin's "maildir" field as the > home directory, and then mail_location=~/Maildir. Like the sql queries > being: > > password_query = \ > SELECT username AS user, password, \ > CONCAT('/home/vmail/', maildir) AS userdb_home, \ > CONCAT('*:bytes=', quota) AS userdb_quota_rule \ > FROM mailbox \ > WHERE username = '%u' and ACTIVE = '1' > > user_query = \ > SELECT CONCAT('/home/vmail/', maildir) AS home, \ > CONCAT('*:bytes=', quota) AS quota_rule \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' > > I used these in another PostfixAdmin server's install. > I even simplified this up to: # taken from stock dovecot2 config userdb { driver = static args = uid=vmail gid=mail home=/var/spool/vmail/%u } mail_location = maildir:~/Maildir password_query = SELECT username AS user, password FROM mailbox WHERE username = '%u' AND active = '1' (no quota's, no special cases for home/mail directory configuration) -- Tom From mlists at edicom.eu Thu Nov 24 22:35:09 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 21:35:09 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322162747.24607.129.camel@hurina> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> Message-ID: <201111242135.09681.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: > I'm not sure if changing /etc/security/limits.conf helps. It's probably > only used by PAM when user logs in, so if Dovecot is started in system > bootup it's unlikely to have been even read yet. Also I think some OSes > override the limits in /etc/init.d/ scripts. Of course, I could be > completely wrong in all of the above, I haven't really tested any of > it Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). Another permanent way between reboots that I just googled is changing this line of includes/linux/fs.h: #define INR_OPEN 1024 /* Initial setting for nfile rlimits */ and recompile the kernel :P. From noel.butler at ausics.net Fri Nov 25 00:30:18 2011 From: noel.butler at ausics.net (Noel Butler) Date: Fri, 25 Nov 2011 08:30:18 +1000 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111242135.09681.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> <201111242135.09681.mlists@edicom.eu> Message-ID: <1322173818.6600.5.camel@tardis> On Thu, 2011-11-24 at 21:35 +0100, Miguel Tormo wrote: > El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: > > I'm not sure if changing /etc/security/limits.conf helps. It's probably > > only used by PAM when user logs in, so if Dovecot is started in system > > bootup it's unlikely to have been even read yet. Also I think some OSes > > override the limits in /etc/init.d/ scripts. Of course, I could be > > completely wrong in all of the above, I haven't really tested any of > > it > > Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). > much simpler way without rebuilding is as Timo suggested, just pick your value #!/bin/sh # Start/stop dovecot. ulimit -n 16384 ... Be careful, building in kernel based higher limits means /every/ service and can lead to DoS, this way limits dovecot to 16384, but keeps other stuff like mysql at 4096, and everything else at 1024, bit safer. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From olli.raisanen at telemail.fi Fri Nov 25 00:43:51 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-15?q?R=E4is=E4nen?=) Date: Fri, 25 Nov 2011 00:43:51 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <1322162990.24607.131.camel@hurina> References: <201111241122.21131.olli.raisanen@telemail.fi> <201111242123.34156.olli.raisanen@telemail.fi> <1322162990.24607.131.camel@hurina> Message-ID: <201111250043.51861.olli.raisanen@telemail.fi> Timo Sirainen kirjoitti 2011-11-24 kello 21:29:46: > > It would be better if you used PostfixAdmin's "maildir" field as the > home directory, and then mail_location=~/Maildir. Thank you, Timo! (And thank you, Tom, for the simplification example!) I never thought ~/Maildir could refer to anything but UNIX system users! :) Timo's queries (beneath) work indeed. However, it took quite a while of testing, and I wonder if it really would cause some trouble if homedir and maildir were kept the same... Regards, Olli > Like the sql queries being: > > password_query = \ > SELECT username AS user, password, \ > CONCAT('/home/vmail/', maildir) AS userdb_home, \ > CONCAT('*:bytes=', quota) AS userdb_quota_rule \ > FROM mailbox \ > WHERE username = '%u' and ACTIVE = '1' > > user_query = \ > SELECT CONCAT('/home/vmail/', maildir) AS home, \ > CONCAT('*:bytes=', quota) AS quota_rule \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' > > I used these in another PostfixAdmin server's install. From pw at wk-serv.de Fri Nov 25 00:58:01 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 23:58:01 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111124143321.GB28600@dibs.tanso.net> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> Message-ID: <4ECECBF9.5070507@wk-serv.de> Jan-Frode Myklebust schrieb: > We use a clusterfs (so it's not very important which backend server the > user is directed to), and have configured the dovecot director with: > > director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 > director_servers = 192.168.42.15 192.168.42.17 > passdb { > args = proxy=y nopassword=y > driver = static > } > > which gives us loadbalancing of new logins, and also failover should a > backend server fail. That looks nice. What is your frontend configuration? How many users do you have that you need that many backend servers? Are your index stored together with the mails? Patrick From janfrode at tanso.net Fri Nov 25 10:17:15 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 25 Nov 2011 09:17:15 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <4ECECBF9.5070507@wk-serv.de> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> Message-ID: <20111125081715.GA24352@dibs.tanso.net> On Thu, Nov 24, 2011 at 11:58:01PM +0100, Patrick Westenberg wrote: > > What is your frontend configuration? The frontend is two servers running dovecot director, and ucarp for managing failover ip-addresses. ===================================================================== # 2.0.14: /etc/dovecot/dovecot.conf # OS: x86_64 Red Hat Enterprise Linux Server release 5.x director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * lmtp_proxy = yes passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 2051 } service auth { client_limit = 4196 unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 1024 } service lmtp { inet_listener lmtp { address = * port = 24 } } service pop3-login { executable = pop3-login director process_limit = 1024 } ssl_cert = How many users do you have that you need that many backend servers? I don't think I'm allowed to share that externally, sorry. This is the mail-system of an ISP, so we have quite a few users.. :-) > Are your index stored together with the mails? They're not stored together, because we made the error of not creating a home directory for our users, but both maildirs and indexes are stored on similar cluster fs' (GPFS). -jf From pw at wk-serv.de Fri Nov 25 11:02:19 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Fri, 25 Nov 2011 10:02:19 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111125081715.GA24352@dibs.tanso.net> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> <20111125081715.GA24352@dibs.tanso.net> Message-ID: <19dc5fa7b626f394a4e54fb0105ac5e8@wk-serv.de> On Fri, 25 Nov 2011 09:17:15 +0100, Jan-Frode Myklebust wrote: > On Thu, Nov 24, 2011 at 11:58:01PM +0100, Patrick Westenberg wrote: >> >> What is your frontend configuration? > > The frontend is two servers running dovecot director, and ucarp for > managing failover ip-addresses. Two additional directors or the two directors from your posted config (.15/.17)? Will I have to configure external IPs for the backend servers or just internal for the connections to and from the directors? I'm not sure if the director "redirects" the connections to the backend servers or if they pass through on the internal network. >> How many users do you have that you need that many backend servers? > > I don't think I'm allowed to share that externally, sorry. This is > the > mail-system of an ISP, so we have quite a few users.. :-) Ok :-) From joseba.torre at ehu.es Fri Nov 25 12:48:14 2011 From: joseba.torre at ehu.es (Joseba Torre) Date: Fri, 25 Nov 2011 11:48:14 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111242135.09681.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> <201111242135.09681.mlists@edicom.eu> Message-ID: <4ECF726E.6020109@ehu.es> El 24/11/11 21:35, Miguel Tormo escribi?: > El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: >> I'm not sure if changing /etc/security/limits.conf helps. It's probably >> only used by PAM when user logs in, so if Dovecot is started in system >> bootup it's unlikely to have been even read yet. Also I think some OSes >> override the limits in /etc/init.d/ scripts. Of course, I could be >> completely wrong in all of the above, I haven't really tested any of >> it > > Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). I tested this "intensively" some time ago, and Timo is right -as usual :) -: modifications in limits.conf are only relevant after a login. This means that they are not enforced at system boot, but they are if you reboot the service by hand (because you have already logged in). The permanent solution is, as Noel Butler has pointed, modifying the startup script and using ulimit before actually starting dovecot. HTH From janfrode at tanso.net Fri Nov 25 13:38:13 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 25 Nov 2011 12:38:13 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322162747.24607.129.camel@hurina> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> Message-ID: <20111125113813.GA27970@dibs.tanso.net> On Thu, Nov 24, 2011 at 09:25:47PM +0200, Timo Sirainen wrote: > I'm not sure if changing /etc/security/limits.conf helps. It's probably > only used by PAM when user logs in, so if Dovecot is started in system > bootup it's unlikely to have been even read yet. Also I think some OSes > override the limits in /etc/init.d/ scripts. Of course, I could be > completely wrong in all of the above, I haven't really tested any of > it :) The OP was running Centos, and the Red Hat way of increasing nofile for services is to add the ulimit setting to /etc/sysconfig/dovecot: ulimit -n 32768 This is what we use, and it survives dovecot upgrades (since this is tagged as a configfile in RPM, not as script that will be overwritten on upgrades). -jf From ef at math.uni-bonn.de Fri Nov 25 14:49:03 2011 From: ef at math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) Date: Fri, 25 Nov 2011 13:49:03 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <1320438508.21919.177.camel@hurina> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> Message-ID: <20111125124902.GB27737@gumme.math.uni-bonn.de> > That shouldn't happen because of Dovecot's indexes. Hm, also if the indexes are local? Fine. > Then the 'S' flag is added to the current Maildir filename without > losing any other changes. And this is supposed to work even over NFS? Great. So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? From sf.rique at gmail.com Fri Nov 25 17:29:35 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Fri, 25 Nov 2011 13:29:35 -0200 Subject: [Dovecot] Which FTS to use! Message-ID: Hello, i am currently using squat dovecot, but some users that have lots os emails ( 1GB ) are complaining about speed. Iam using dovecot 2.0.13 on debian 6 my squat conf is plugin { fts = squat fts_squat = partial=4 full=5 } I am thinking in maybe start to use Solr, so any one can help me with an decision? I am probably gonna update dovecot to 2.1 next year. and them maybe change the FTS plugin. This year i will get 10k disk only to store the dovecot index, rigth now they are saved on the same storage that the mails are and also change the "full" parameter to something like 10 or 15 I know i will use more disk space for it, but it will make a lot of differences right? it will match the message without the need of opening it for any search smaller than 10 character i guess. But this will be enough? or Solr or Lucene still better? Anyway, i am not current updating the indexes automatic, but i am planing on it, just not sure what is better, the command doveadm index or doveadm search... with one is better to do the task? Thanks! []'sf.rique From tss at iki.fi Fri Nov 25 18:47:51 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 25 Nov 2011 18:47:51 +0200 Subject: [Dovecot] Which FTS to use! In-Reply-To: References: Message-ID: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> On 25.11.2011, at 17.29, Henrique Santos Fernandes wrote: > Hello, i am currently using squat dovecot, but some users that have lots os > emails ( 1GB ) are complaining about speed. > Iam using dovecot 2.0.13 on debian 6 > > my squat conf is > plugin { > fts = squat > fts_squat = partial=4 full=5 > } > > I am thinking in maybe start to use Solr, so any one can help me with an > decision? With v2.0 Solr is the only other choice. > I am probably gonna update dovecot to 2.1 next year. and them maybe change > the FTS plugin. With v2.1 Lucene is another possibility, and it's somewhat easier to install than Solr. Also v2.1's Solr backend is improved, so I wouldn't really recommend spending a lot of time and disk I/O on v2.0's Solr since you'll need to rebuild the index anyway in v2.1 (or keep using "solr_old" backend with some missing featues). > This year i will get 10k disk only to store the dovecot index, rigth now > they are saved on the same storage that the mails are and also change the > "full" parameter to something like 10 or 15 I know i will use more disk > space for it, but it will make a lot of differences right? it will match > the message without the need of opening it for any search smaller than 10 > character i guess. > > > But this will be enough? or Solr or Lucene still better? The main problem with Squat is its index updating performance, which sucks. > Anyway, i am not current updating the indexes automatic, but i am planing > on it, just not sure what is better, the command doveadm index or doveadm > search... with one is better to do the task? doveadm index From lists at wildgooses.com Fri Nov 25 18:59:22 2011 From: lists at wildgooses.com (Ed W) Date: Fri, 25 Nov 2011 16:59:22 +0000 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <4ECFC96A.9020204@wildgooses.com> On 25/11/2011 12:49, Edgar Fu? wrote: >> That shouldn't happen because of Dovecot's indexes. > Hm, also if the indexes are local? Fine. > >> Then the 'S' flag is added to the current Maildir filename without >> losing any other changes. > And this is supposed to work even over NFS? Great. > > So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? You don't have any Thunderbird clients accessing this box do you? I have some wierd issue where our TB with the option "don't mark message read", still triggers messages to be marked read... Wierdly it only does it on some messages and all those from specific senders - can't correlate it with anything obvious in the message itself though... Regards Ed W From ef at math.uni-bonn.de Fri Nov 25 19:20:55 2011 From: ef at math.uni-bonn.de (=?iso-8859-1?Q?Edgar_Fu=DF?=) Date: Fri, 25 Nov 2011 18:20:55 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <4ECFC96A.9020204@wildgooses.com> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> <4ECFC96A.9020204@wildgooses.com> Message-ID: <476867D4-69AE-4854-96F8-43C4559DC51F@math.uni-bonn.de> > You don't have any Thunderbird clients accessing this box do you? Yes, I have. But I also experienced the problem myself only using mutt and Apple Mail. > I have some w[ei]rd issue where our TB with the option > "don't mark message read", still triggers messages to be marked read It's the other way round. People read their messages, they are flagged as read and all of a sudden revert to unread. From e-frog at gmx.de Fri Nov 25 19:23:59 2011 From: e-frog at gmx.de (e-frog) Date: Fri, 25 Nov 2011 18:23:59 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <4ECFC96A.9020204@wildgooses.com> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> <4ECFC96A.9020204@wildgooses.com> Message-ID: <4ECFCF2F.50307@gmx.de> On 25.11.2011 17:59, wrote Ed W: > On 25/11/2011 12:49, Edgar Fu? wrote: > You don't have any Thunderbird clients accessing this box do you? I > have some wierd issue where our TB with the option "don't mark message > read", still triggers messages to be marked read... Wierdly it only does > it on some messages and all those from specific senders - can't > correlate it with anything obvious in the message itself though... I had the same issue a while ago with Thunderbird. On one installation it worked on another it didn't. After comparing their config I found the culprit. Changing mail.server.default.mime_parts_on_demand to **false** fixed this issue for me. This setting is discussed in... http://kb.mozillazine.org/Entire_message_fetched_when_opening_a_IMAP_message ... but not mentioning the "don't mark message as read" issue. Regards, e-frog * * Englisch * Deutsch * Englisch * Deutsch From tss at iki.fi Fri Nov 25 19:47:22 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 25 Nov 2011 19:47:22 +0200 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <5539B894-4826-480F-95F6-A7D8CBF9B147@iki.fi> On 25.11.2011, at 14.49, Edgar Fu? wrote: >> That shouldn't happen because of Dovecot's indexes. > Hm, also if the indexes are local? Fine. Yep. It just means that indexes then aren't fully up to date. Hmm. I guess possibly one potential problem is if 1. dovecot.index.log says that "set flags to ()" but then it doesn't actually finish that 2. Next time indexes are accessed from second server, and flags are set to (\Seen) 3. And then finally user logs in back to original server, which sees that "aha, flags should be set to ()" and does that. But the problem is the first step. It shouldn't normally do it only halfway through, unless it crashes or something.. Also typically flags aren't replaced, they're added/removed, but this completely depends on the client and I don't remember what Apple Mail does. >> Then the 'S' flag is added to the current Maildir filename without >> losing any other changes. > And this is supposed to work even over NFS? Great. > > So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? You could try enabling mail_log plugin with flag_change event to see when it actually changes. http://wiki.dovecot.org/Plugins/MailLog From noel.butler at ausics.net Sat Nov 26 02:19:13 2011 From: noel.butler at ausics.net (Noel Butler) Date: Sat, 26 Nov 2011 10:19:13 +1000 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <1322266753.6288.6.camel@tardis> On Fri, 2011-11-25 at 13:49 +0100, Edgar Fu? wrote: > > Then the 'S' flag is added to the current Maildir filename without > > losing any other changes. > And this is supposed to work even over NFS? Great. Absolutely, always has worked over NFS -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From rajeev at teamsg.in Sat Nov 26 08:32:48 2011 From: rajeev at teamsg.in (Rajeev Sharma) Date: Sat, 26 Nov 2011 12:02:48 +0530 Subject: [Dovecot] Dovecot 1.2 ACL problem. Message-ID: Sir, I have migrate from courier-imap to dovecot without any problem but on courier in have shared folder which access on the basis on rights. I have manage these folders in dovecot but i have one problem all the shared folder is accessible by all user whether they have rights or not i have place dovecot-acl in shared folder and assign user my dovecot-acl is user=rajeev at teamsg.in tr. But all other user which is not listing on dovecot-acl also able to access these folder. Here is the output of dovecot -n. # 1.2.17: /usr/local/etc/dovecot.conf # OS: Linux 2.6.18-194.3.1.el5 i686 CentOS release 5.5 (Final) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log ssl_listen: 192.168.0.251:993 ssl_cert_file: /var/qmail/control/servercert.pem ssl_key_file: /var/qmail/control/servercert.pem login_dir: /var/run/dovecot//login login_executable: /usr/local/libexec/dovecot/imap-login first_valid_uid: 508 last_valid_uid: 508 mail_location: maildir:~/Maildir namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes namespace: type: shared separator: . prefix: INBOX.Shared. location: maildir:/home/vpopmail/domains/sgcricket.com/Maildir/Out:INDEX=~/Maildir/Shared list: children auth default: user: vpopmail passdb: driver: vpopmail userdb: driver: vpopmail plugin: acl: vfile Please help whats wrong i am doing. Rajeev Sharma. From dan.colascione at gmail.com Sat Nov 26 08:36:39 2011 From: dan.colascione at gmail.com (Daniel Colascione) Date: Fri, 25 Nov 2011 22:36:39 -0800 Subject: [Dovecot] Assertion failure in 2.0.16 - SEARCH () Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nov 25 22:25:39 dancol dovecot: master: Error: service(imap): child 25115 killed with signal 6 (core dumps disabled) Nov 25 22:25:59 dancol dovecot: imap(dancol): Panic: file mail-search-build.c: line 59 (mail_search_build_key_int): assertion failed: (sarg->value.subargs != NULL) Nov 25 22:25:59 dancol dovecot: imap(dancol): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3d291) [0xb7636291] -> /usr/lib/dovecot/libdovecot.so.0(+0x3d2ff) [0xb76362ff] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0xb760bb7a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x24109) [0xb7692109] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x24177) [0xb7692177] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_build+0x82) [0xb7692242] - -> dovecot/imap [dancol 127.0.0.1 SEARCH](imap_search_args_build+0x4a) [0x805db5a] -> dovecot/imap [dancol 127.0.0.1 SEARCH](cmd_search+0xbf) [0x80537ef] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x80568cc] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x80569a2] -> dovecot/imap [dancol 127.0.0.1 SEARCH](client_handle_input+0x175) [0x8056c25] -> dovecot/imap [dancol 127.0.0.1 SEARCH](client_input+0x5f) [0x80574cf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x51) [0xb76430d1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0xb7644193] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0xb7643050] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2a) [0xb762ec1a] -> dovecot/imap [dancol 127.0.0.1 SEARCH](main+0x315) [0x805fef5] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb74b6c76] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x804eb81] $ telnet localhost imap Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. . LOGIN dancol XXXXXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in . SELECT rss * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8 EXISTS * 0 RECENT * OK [UIDVALIDITY 1322264663] UIDs valid * OK [UIDNEXT 11] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. . SEARCH () Connection closed by foreign host. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk7QiNcACgkQ17c2LVA10VvhHACgihmbmTjVNl/PbKcreIDRTKpC 5wkAoJd9FHXM4GhPfnVUjwrpdsqnVQ6E =RlpQ -----END PGP SIGNATURE----- From mark.zealey at webfusion.com Sat Nov 26 18:33:30 2011 From: mark.zealey at webfusion.com (Mark Zealey) Date: Sat, 26 Nov 2011 18:33:30 +0200 Subject: [Dovecot] using dsync to convert mailboxes looses caching options Message-ID: <4ED114DA.7030707@webfusion.com> Hi there, We're trying to convert users from Maildir to sdbox at present; I'm using dsync to achieve this (2.0.16) however when the user's have been converted we only get minimal information in the caching files. Is there some way to preserve all the caching decisions that were previously made so that when the user logs in to the new mailbox we don't have to cause an io storm rebuilding the cache that we know was good? Dovecot seems to be partially doing this - if i remove the logs/cache from the source mailbox no cache files are built in the conversion; if i put them back then we get a cache file built but it only contains a few bits of information (guid, date.save). Looking into this a bit further i find that when the caches are present at source the fields are preserved but the 'last used' date and caching decisions are not which I suspect means dsync doesn't bother caching on import - only fields with a yes decision in the source are copied (but their decision is only copied as a tmp with the date of import). For example: Source idxview cache: -- Cache fields -- # Name Type Size Dec Last used 0: flags bit 4 tmp 2011-11-25 16:09 1: date.received fix 4 yes 2011-11-26 16:17 2: size.virtual fix 8 tmp 2011-11-25 16:09 3: imap.bodystructure str - tmp 2011-11-25 16:09 4: mime.parts var - tmp 2011-11-25 16:09 5: hdr.IMPORTANCE hdr - tmp 2011-11-25 16:09 6: hdr.X-PRIORITY hdr - tmp 2011-11-25 16:09 7: hdr.CONTENT-TYPE hdr - tmp 2011-11-25 16:09 ... 18: date.sent fix 8 no 1970-01-01 01:00 19: date.save fix 4 yes 2011-11-26 16:17 20: size.physical fix 8 no 1970-01-01 01:00 21: imap.body str - no 1970-01-01 01:00 ... (24 total entries. stripped ones are just hdr. with a tmp decision) dst cache: # Name Type Size Dec Last used 0: flags bit 4 tmp 1970-01-01 01:00 1: date.sent fix 8 no 1970-01-01 01:00 2: date.received fix 4 tmp 1970-01-01 01:00 3: date.save fix 4 tmp 2011-11-26 16:19 4: size.virtual fix 8 tmp 1970-01-01 01:00 5: size.physical fix 8 no 1970-01-01 01:00 6: imap.body str - no 1970-01-01 01:00 7: imap.bodystructure str - tmp 1970-01-01 01:00 8: imap.envelope str - no! 1970-01-01 01:00 9: pop3.uidl str - no 1970-01-01 01:00 10: guid str - tmp 2011-11-26 16:19 11: mime.parts var - tmp 1970-01-01 01:00 12: hdr.IMPORTANCE hdr - tmp 1970-01-01 01:00 13: hdr.X-PRIORITY hdr - tmp 1970-01-01 01:00 ... and dst cached message for example: RECORD: seq=57, uid=12207, flags=0x09 (Seen Answered) - ext 1 modseq : 1 (0100000000000000) - ext 3 cache : 5516 (8c150000) - cache offset=5516 size=92, prev_offset = 0 - guid: 1321895282.XXXt,S=7399760,W=7496225 - date.save: 1322324286 (3e11d14e) src idx record for this message contains pretty much all fields. Thanks, Mark From sf.rique at gmail.com Sun Nov 27 15:17:16 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Sun, 27 Nov 2011 11:17:16 -0200 Subject: [Dovecot] Which FTS to use! In-Reply-To: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> References: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> Message-ID: So i guess i will just stay with squat until i upgrade to 2.1 and then move to solr.. And make the cron update daily for active users! Thanks a lot! []'sf.rique On Fri, Nov 25, 2011 at 2:47 PM, Timo Sirainen wrote: > On 25.11.2011, at 17.29, Henrique Santos Fernandes wrote: > > > Hello, i am currently using squat dovecot, but some users that have lots > os > > emails ( 1GB ) are complaining about speed. > > Iam using dovecot 2.0.13 on debian 6 > > > > my squat conf is > > plugin { > > fts = squat > > fts_squat = partial=4 full=5 > > } > > > > I am thinking in maybe start to use Solr, so any one can help me with an > > decision? > > With v2.0 Solr is the only other choice. > > > I am probably gonna update dovecot to 2.1 next year. and them maybe > change > > the FTS plugin. > > With v2.1 Lucene is another possibility, and it's somewhat easier to > install than Solr. Also v2.1's Solr backend is improved, so I wouldn't > really recommend spending a lot of time and disk I/O on v2.0's Solr since > you'll need to rebuild the index anyway in v2.1 (or keep using "solr_old" > backend with some missing featues). > > > This year i will get 10k disk only to store the dovecot index, rigth now > > they are saved on the same storage that the mails are and also change > the > > "full" parameter to something like 10 or 15 I know i will use more disk > > space for it, but it will make a lot of differences right? it will match > > the message without the need of opening it for any search smaller than > 10 > > character i guess. > > > > > > But this will be enough? or Solr or Lucene still better? > > The main problem with Squat is its index updating performance, which sucks. > > > Anyway, i am not current updating the indexes automatic, but i am planing > > on it, just not sure what is better, the command doveadm index or doveadm > > search... with one is better to do the task? > > doveadm index From Patrick.Wood at tups.net.au Mon Nov 28 05:24:54 2011 From: Patrick.Wood at tups.net.au (Patrick Wood) Date: Mon, 28 Nov 2011 03:24:54 +0000 Subject: [Dovecot] POP3/IMAP crash signal 10 Message-ID: <0D810DB78132A8488DE17E655BF29B685C5EAA5C@TUPSDC01.tups.net.au> Hi, I'm building a Postfix/Dovecot mail server and while I am able to send/receive emails using telnet, after establishing a connection to Dovecot via a client (Mail Live, Thunderbird etc) the following appears in the logs: Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.info] pop3-login: Login: user=, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, mpid=282 78 Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.error] master: Error: service(pop3-login): child 28277 killed with signal 10 (core not dumped - set service pop3-log in { drop_priv_before_exec=yes }) Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.info] pop3(user at domain.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/5, size=3250 I also experience very poor connectivity to the server, with Microsoft Mail and Thunderbird regularly failing to connect (connection errors in the client and no login attempt in the dovecot logs). I assume this is related to the issue above. I am running dovecot 2.0.16 on Solaris 10 SPARC using a ZFS filesystem for mailbox storage. Here is my configuration: # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4u zfs auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:/storage0/home/vmail/%d/%n/Maildir mail_plugins = quota passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=1G } postmaster_address = root at domain.com sendmail_path = /usr/lib/sendmail ssl_cert = Hello, I tried to upgrade from 2.0.15 to 2.1.rc1 over the weekend but ran into some problems with squat. My two largest mailboxes were corrupted, producing this error: Nov 27 17:38:12 kerio1 dovecot[42860]: imap(bigmailbox): Error: Corrupted squat uidlist file /var/mail/metro-email.com/bigmailbox/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.search.uids: wrong indexid There is a good chance that I had shot my own foot, by forgetting to update postfix's dovecot-lda reference to the new instance. I had also enabled an inet based LMTP listener, although it is not being used yet. Even using non-corrupted indexes, it seemed that FTS was not being used. A basic search such as this one took many minutes to complete: doveadm search -u jsample mailbox INBOX body eps I found messages like this in mail.log after performing the search: Nov 27 18:21:39 kerio1 dovecot[42860]: indexer-worker(jsample): Indexed 0 messages in Drafts Nov 27 18:21:39 kerio1 dovecot[42860]: indexer-worker(jsample): Indexed 0 messages in INBOX When I rolled back to 2.0.15, the same search returned almost immediately. The larger mailboxes went quickly again, but only after they had a chance to rebuild. A few other notes about our configuration: We are on PPC, with the stats service disabled (because of kqueue issue discussed earlier). I made some minor changes in my config since the previous installed version (diff below). mail_location override is in place for all users via password file, e.g.: userdb_mail=mdbox:/var/mail/metro-email.com/jsample/mdbox Thanks, Tony kerio1:~ metroadmin$ /opt/dovecot-2.1.rc1/bin/doveconf -n -c /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf # 2.1.rc1: /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_mechanisms = plain login lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_proxy = yes mail_location = maildir:%h mail_plugins = fts fts_squat mail_log notify mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader mdbox_rotate_size = 10 M namespace { inbox = yes location = prefix = separator = / } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users driver = passwd-file } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from subject sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at metro-email.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = metroadmin } } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 192.168.20.10 127.0.0.1 port = 24 } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_cert = # 2.1.rc1: /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf 9a10 > lmtp_proxy = yes 15c16 < managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave --- > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader 24c25 < args = scheme=CRYPT username_format=%u /opt/dovecot-2.0.15/etc/dovecot/users --- > args = scheme=CRYPT username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users 50a52,57 > service lmtp { > inet_listener lmtp { > address = 192.168.20.10 127.0.0.1 > port = 24 > } > } 53a61,65 > service stats { > fifo_listener stats-mail { > mode = 00 > } > } 57c69 < args = username_format=%u /opt/dovecot-2.0.15/etc/dovecot/users --- > args = username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users kerio1:~ metroadmin$ From msbrogli-dovecot at vialink.com.br Tue Nov 29 00:14:19 2011 From: msbrogli-dovecot at vialink.com.br (Marcelo Salhab Brogliato) Date: Mon, 28 Nov 2011 20:14:19 -0200 Subject: [Dovecot] Fault tolerant architecture Message-ID: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Hi, I'm new to this list and want your help. I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). I'm using dovecot 1.2.9. I guess you already had some of these problems or maybe in another architecture these neither exists. Thanks for any help, Marcelo From list at airstreamcomm.net Tue Nov 29 01:16:37 2011 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 28 Nov 2011 17:16:37 -0600 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <8c0fae947f9c32073f798299f027684c@mail.airstreamcomm.net> On Mon, 28 Nov 2011 20:14:19 -0200, Marcelo Salhab Brogliato wrote: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we > have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. > Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS > using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is > using two IP addresses in our DNS records. > > About NFS using local indexes, I'm configuring a test server. But how to > configure local indexes when my mail_location comes from sql (userdb_home > actually). > I'm using dovecot 1.2.9. > > I guess you already had some of these problems or maybe in another > architecture these neither exists. > > Thanks for any help, > > Marcelo Marcelo, There are a number of ways to bring HA to a cluster of mail servers, one that we have experimented with lately is a bit exotic, but might work for you. >From the base layer we are experimenting with GlusterFS, distributed and replicated file system that offers very simple management and high availablity. It does run in userspace, which according to some will suffer from performance bottlenecks, but so far we have not seen any serious problems while running on 15k disks in raid 10. Assuming you have two virtual machines you could create a distributed file system between them, and have a mirrored copy of the data on both. Next is dovecot/postfix/webmail which would be setup to use the local GlusterFS mount on the system containing the mail storage and indexes. To provide HA on the connectivity side we used ucarp, which creates a virtual IP address between two servers and fails that virtual IP over to another server in the event of a server going down. I personally would never use DNS load balancing (two IPs for one DNS name) as it would round robin to each server regardless of whether it is running or not. This is somewhat exotic, but it works and provides a very high level of availability. However with HA comes more complexity and management. Good luck and let me know if you would like more specifics. From seandarcy2 at gmail.com Tue Nov 29 01:50:18 2011 From: seandarcy2 at gmail.com (sean darcy) Date: Mon, 28 Nov 2011 18:50:18 -0500 Subject: [Dovecot] can not get t'bird to create folder Message-ID: dovecot-2.0.16 on Fedora 15. thunderbird-8.0 I can access dovecot from thunderbird, imap maildir. But when I try to create a new folder, nothing happens. I'm not using LAYOUT=fs, so I have added, tb-extra-mailbox-sep, which didn't seem relevant anyhow. Any help appreciated. sean From micah at riseup.net Tue Nov 29 02:04:41 2011 From: micah at riseup.net (Micah Anderson) Date: Mon, 28 Nov 2011 19:04:41 -0500 Subject: [Dovecot] Indexes to MLC-SSD References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: <87sjl7lrom.fsf@algae.riseup.net> Dovecot-GDH writes: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net > > Flashcache: https://github.com/facebook/flashcache/ That site has no information about what flashcache is. From seandarcy2 at gmail.com Tue Nov 29 04:28:29 2011 From: seandarcy2 at gmail.com (sean darcy) Date: Mon, 28 Nov 2011 21:28:29 -0500 Subject: [Dovecot] can't get sieve to sort virus into spam Message-ID: I can't get sieve to put virus files in the SPAM folder. dovecot -n # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.8-32.fc15.i686.PAE i686 Fedora release 15 (Lovelock) auth_debug_passwords = yes info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_access_groups = mail mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { antispam_backend = dspam-exec antispam_dspam_args = --deliver;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam = Spam antispam_trash = trash;Trash;Deleted Items; Deleted Messages sieve = /home/vmail/%d/%n/dovecot.sieve.script sieve_dir = /home/vmail/%d/%n/dovecot.sieve/ sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } user = vmail } ssl_cert = References: Message-ID: <1266621424.20111129030956@gmail.com> On Tuesday, November 29, 2011 at 02:28:29 UTC, seandarcy2 at gmail.com confabulated: > I can't get sieve to put virus files in the SPAM folder. > dovecot -n > # 2.0.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.38.8-32.fc15.i686.PAE i686 Fedora release 15 (Lovelock) > auth_debug_passwords = yes > info_log_path = /var/log/dovecot-info.log > log_path = /var/log/dovecot.log > mail_access_groups = mail > mail_home = /home/vmail/%d/%n > mail_location = maildir:~/mail > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date > mbox_write_locks = fcntl > passdb { > args = scheme=CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file > } > plugin { > antispam_backend = dspam-exec > antispam_dspam_args = --deliver;--user;%u > antispam_dspam_binary = /usr/bin/dspam > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam = Spam > antispam_trash = trash;Trash;Deleted Items; Deleted Messages > sieve = /home/vmail/%d/%n/dovecot.sieve.script > sieve_dir = /home/vmail/%d/%n/dovecot.sieve/ > sieve_global_dir = /var/lib/dovecot/sieve/global/ > sieve_global_path = /var/lib/dovecot/sieve/default.sieve > } > service auth { > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service lmtp { > inet_listener lmtp { > address = 127.0.0.1 > port = 24 > } > user = vmail > } > ssl_cert = ssl_key = userdb { > driver = passwd > } > userdb { > args = username_format=%u /etc/dovecot/users > driver = passwd-file > } > protocol lmtp { > mail_plugins = quota sieve > } > protocol imap { > mail_plugins = " antispam" > } > cat /var/lib/dovecot/sieve/default.sieve > require "fileinto"; > if header :contains "X-DSPAM-Result" "Virus" { > fileinto "SPAM"; > } > if header :contains "X-DSPAM-Result" "Spam" { > fileinto "SPAM"; > } > and there is a folder SPAM: > cat mail/subscriptions > Sent > Trash > Drafts > SPAM > yet : > Content-Type: multipart/alternative; > boundary=14dae9399d3b9b67cc04b2d60790 > X-DSPAM-Result: Virus > X-DSPAM-Processed: Mon Nov 28 21:03:47 2011 > X-DSPAM-Confidence: 1.0000 > X-DSPAM-Probability: 1.0000 > X-DSPAM-Signature: 4ed43d8341479178914901 > --14dae9399d3b9b67cc04b2d60790 > Content-Type: text/plain; charset=ISO-8859-1 > X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H* > does not end up into SPAM, or anyplace else, including inbox. > How do I make this work? What gets logged? If sieve has an issue storing something, it is logged. If you don't have the autocreate plugin loaded, the directories have to be created some other way. Does SPAM exist as a directory? Have you compiled the global sieve file using sievec? Does the particular mail account have sieve rules? Global rules are not executed if there are account rules. If that is the case, check out the sieve_before and sieve_after parameters. -- There are 10 kinds of people in the world... Those who understand binary, and those who don't. From rainer.frey at inxmail.de Tue Nov 29 09:19:21 2011 From: rainer.frey at inxmail.de (Rainer Frey) Date: Tue, 29 Nov 2011 08:19:21 +0100 Subject: [Dovecot] Authentication listener configuration Message-ID: <22BC7966-5C73-4085-AB63-3B5E20FCE335@inxmail.de> Hi, I'm wondering about the authentication listeners that dovecot uses. In many configurations I see: service auth { unix_listener auth-userdb { Does any component of dovecot use this listener internally (lmtp, lda ...) and does that read the config and search for the listener, or is the exact name "auth-userdb" expected? Is this maybe dependent on the userdb? Which ones need it? And if I want to use postfix SMTP auth, should I define a second listener, or just re-configure that default one to postfix's needs? Also, in which cases do I need to define an auth-worker service, and a unix_listener for it? I don't find this in the documentation, if that's caused by my lacking search skills, please just send apointer. Thanks Rainer From robert at schetterer.org Tue Nov 29 09:49:52 2011 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 29 Nov 2011 08:49:52 +0100 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <4ED48EA0.4060706@schetterer.org> Am 28.11.2011 23:14, schrieb Marcelo Salhab Brogliato: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. > > About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). > I'm using dovecot 1.2.9. > > I guess you already had some of these problems or maybe in another architecture these neither exists. > > Thanks for any help, > > Marcelo > i have run , store drbd with ocfs behind keepalived loadbalancers this should run with vm too, but it might not optimal, depening i.e how much mailboxes etc search this list archive, there were posts about equal issues -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stephan at rename-it.nl Tue Nov 29 10:00:39 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 29 Nov 2011 09:00:39 +0100 Subject: [Dovecot] can't get sieve to sort virus into spam In-Reply-To: <1266621424.20111129030956@gmail.com> References: <1266621424.20111129030956@gmail.com> Message-ID: <4ED49127.6020000@rename-it.nl> On 11/29/2011 4:09 AM, Duane Hill wrote: > On Tuesday, November 29, 2011 at 02:28:29 UTC, seandarcy2 at gmail.com confabulated: > >> I can't get sieve to put virus files in the SPAM folder. >> does not end up into SPAM, or anyplace else, including inbox. >> How do I make this work? > What gets logged? If sieve has an issue storing something, it is > logged. The message must be stored somewhere, even if Sieve fails entirely. If your Dovecot logs provide no indication that a message was handled, are you even sure that LTMP is getting a message at all? In this regard your MTA's logs are of importance as well. Regards, Stephan. From pw at wk-serv.de Tue Nov 29 16:24:22 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 29 Nov 2011 15:24:22 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111125081715.GA24352@dibs.tanso.net> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> <20111125081715.GA24352@dibs.tanso.net> Message-ID: <4ED4EB16.1030704@wk-serv.de> Hi again, I have a working test environment not but I don't feel very comfortable with a non encrypted connection between the proxy/director and the backend servers. If I set ssl = required on my backend server I get this error: imap-login: Disconnected (tried to use disabled plaintext auth) Is there a chance to encrypt this connection? # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 auth_debug = yes auth_verbose = yes director_mail_servers = 172.17.0.31 director_servers = 172.17.213.213 disable_plaintext_auth = no lmtp_proxy = yes mail_debug = yes passdb { args = proxy=y nopassword=y driver = static } service auth { unix_listener auth-userdb { mode = 0600 user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } ssl = required ssl_cert = Hi, after upgrading from 2.0.15 to 2.0.16 authentication with vpopmail won?t work anymore. 2.0.x and 1.2.x works (this output is from 2.0.15): Nov 28 18:29:21 mail dovecot: auth: Debug: auth client connected (pid=3855) Nov 28 18:29:21 mail dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=10.114.57.11 rip=10.114.57.11 lport=143 rport=37693 resp=d3NjaG9lbGxlckBodGwtcGVyZy5hYy5hdAB3c2Nob2VsbGVyQGh0bC1wZXJnLmFjLmF0AGRpYWJsbw== Nov 28 18:29:21 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: client out: OK 1 user=wschoeller at htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: master in: REQUEST 4001103873 3855 1 7a21a4d39cc6b187b3b14dc4cfbf077a Nov 28 18:29:21 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: master out: USER 4001103873 wschoeller at htl-perg.ac.at uid=89 gid=89 home=/var/vpopmail/domains/htl-perg.ac.at/wschoeller Nov 28 18:29:21 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=10.114.57.11, lip=10.114.57.11, mpid=3856, secured Nov 28 18:29:21 mail dovecot: imap(wschoeller at htl-perg.ac.at): Disconnected: Logged out bytes=29/405 Nov 28 18:29:21 mail dovecot: auth: Debug: auth client connected (pid=3858) 2.0.16 doesn?t work: Nov 28 18:43:15 mail dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/auth Nov 28 18:43:15 mail dovecot: auth: Debug: auth client connected (pid=27392) Nov 28 18:43:15 mail dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=10.114.57.11 rip=10.114.57.11 lport=143 rport=42768 resp=d3NjaG9lbGxlckBodGwtcGVyZy5hYy5hdAB3c2Nob2VsbGVyQGh0bC1wZXJnLmFjLmF0AGRpYWJsbw== Nov 28 18:43:15 mail dovecot: auth-worker: Debug: Loading modules from directory: /usr/lib/dovecot/auth Nov 28 18:43:15 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:43:15 mail dovecot: auth-worker: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): Password mismatch Nov 28 18:43:15 mail dovecot: auth-worker: Error: md5_verify(wschoeller at htl-perg.ac.at): Not a valid MD5-CRYPT or PLAIN-MD5 password Nov 28 18:43:15 mail dovecot: auth-worker: Error: smd5_verify(wschoeller at htl-perg.ac.at): SMD5 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha_verify(wschoeller at htl-perg.ac.at): SSHA password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha256_verify(wschoeller at htl-perg.ac.at): SSHA256 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha512_verify(wschoeller at htl-perg.ac.at): SSHA512 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb Nov 28 18:43:15 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): CLEARTEXT(mypassword) != '', try OTP scheme instead Nov 28 18:43:17 mail dovecot: auth: Debug: client out: FAIL 1 user=wschoeller at htl-perg.ac.at Nov 28 18:43:17 mail dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=10.114.57.11, lip=10.114.57.11, secured Dovecot -n: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.36-gentoo-r8 i686 Gentoo Base System release 1.12.14 auth_debug = yes auth_debug_passwords = yes disable_plaintext_auth = no first_valid_uid = 89 listen = * mail_location = maildir:~/.maildir namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . type = private } passdb { args = webmail=127.0.0.1 driver = vpopmail } service auth { user = root } ssl_cert = References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <87sjl7lrom.fsf@algae.riseup.net> Message-ID: <0DD532C1-7E22-49BA-A457-250194CF4828@rocketfish.com> https://github.com/facebook/flashcache/blob/master/doc/flashcache-doc.txt On Nov 28, 2011, at 4:04 PM, Micah Anderson wrote: > Dovecot-GDH writes: > >> If I/O performance is a concern, you may be interested in ZFS and Flashcache. >> >> Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) >> ZFS does run on Linux http://zfs-fuse.net >> >> Flashcache: https://github.com/facebook/flashcache/ > > That site has no information about what flashcache is. > > From dbrooks at mdah.state.ms.us Tue Nov 29 22:46:35 2011 From: dbrooks at mdah.state.ms.us (Donny Brooks) Date: Tue, 29 Nov 2011 14:46:35 -0600 Subject: [Dovecot] Best setup for performance and fault tolerance Message-ID: <6f4f-4ed54480-5-fba0340@98207411> Hello all. I am in need of some guidance. First a little background. Currently our mail server is on physical hardware (Dell server with 2x 2.8GHz Xeon w/ 4GB ram, raid5 array, single gigabit nic) running on Fedora 11 and postfix-2.5.6-3.fc11.x86_64 with dovecot-1.2.11-3.fc11.x86_64. Mailstore is via Maildir format that was converted from mbox about a year ago. This same machine is also our PDC with samba, Master LDAP, slave MySQL replication, primary DNS, and home server for about 20 users. We have approximately 200 end users that have mailboxes on the server ranging from 1KB to 20GB in size. Total mail store is currently at 300GB. About 75 of the users are currently POP access and their mail will be moved to the server soon and setup as IMAP. This is calculated to add roughly another 150GB of mail for a total of 450GB mail store. Being a state agency we have to keep the mail indefinitely for public record reasons. We use a mixture of Thunderbird as an IMAP client and SOGo for web access. Now to the problem: Recently we have been having super slow access to the mail server. Turns out the load was insanely high partially due to the samba home server portion, which is being moved off as we speak, and the other part is due to people searching their mail. Just yesterday one of our users nearly brought the entire agency to its knees by performing a search on her 8GB of mail via IMAP. Since the server is old in both hardware and software I have been tasked with moving it to newer hardware and a newer OS. We currently have 3 virtual servers running Xen and a SAN. The new setup will be placed in the virtual environment. I will probably run Fedora 16 as the OS but am open to Centos, Fedora, or Ubuntu. Now to the question: What is the best way to setup Dovecot so that it is tuned for performance and high available? We have been running with this single point of failure for years so as long as we are moving the mail server we might as well build in some redundancy. To solve the searching problem I thought of maybe setting up some type of indexing. I do kind of want to break the various services out on to separate virtual machines for a little more fault tolerance, but that is not totally necessary. What do you think of things like iRedmail? I see it's usefulness but the not being able to separate services kind of defeats the purpose, plus I want to setup a high available MySQL cluster and possibly OpenLDAP or 389 cluster so iRedMail may not be the best solution. Sorry for the long email but I am trying to get all the information out there at once so it will help get more directed responses in the shortest amount of time. I look forward to any and all input on this matter -- Donny B MDAH From Ralf.Hildebrandt at charite.de Tue Nov 29 23:42:56 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 29 Nov 2011 22:42:56 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders Message-ID: <20111129214256.GL14970@charite.de> While our webmail installation creates "Sent" as a Sent-Items Folder, Apple Mail creates "Sent Messages". Is there a way of "mapping" foldernames e.g. map "Sent Messages" to "Sent" map "Deletes Messages" to "Trash" and so on? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From micah at riseup.net Tue Nov 29 23:50:59 2011 From: micah at riseup.net (micah anderson) Date: Tue, 29 Nov 2011 16:50:59 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <87ehx8gic7.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> Message-ID: <87zkfek37g.fsf@algae.riseup.net> On Wed, 16 Nov 2011 11:09:12 -0500, micah anderson wrote: > On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen wrote: > > On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > > > I really like the feature where you can define quota rules with percents > > > which trigger off of the default values[0] (so you can set the Trash to > > > allow for 10% more of the user's quota for example). > > > > > > What I would really love in dovecot would be for the ability to > > > configure a quota rule for dovecot-lda. I would like to configure things > > > so we don't bounce emails for users until they are well over quota, the > > > IMAP quota plugin is a really great way to notify people that they are > > > over quota because it fails to write to other folders.... that should be > > > enough to get people's attention that they need to deal with things, but > > > bouncing is harsh. > > > > You can give LDA-specific quota settings. Like maybe: > > > > protocol lda { > > plugin { > > quota_rule2 = INBOX:storage=200% > > } > > } I just tried doing this, and I started bouncing mail for everyone because they were over quota (even when they arent), so I am clearly missing something in how I have it configured. This is what I have: namespace { separator = . prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no } plugin { quota = dict:Your mail quota::proxy::quota } dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } plugin { quota_exceeded_message = You are over quota. To avoid losing mail, immediately empty your Trash and Sent folders and delete emails with large attachments. quota_rule = *:bytes=100663296 # in case someone doesn't have sql quota quota_rule2 = Trash:bytes=+20%% # let trash be larger by 20% quota_rule3 = Spam:bytes=+10%% # let spam be larger by 10% quota_rule4 = INBOX.Trash:bytes=+20%% # repeat the above for namespaces quota_rule5 = INBOX.Spam:bytes=+10%% # same quota_rule6 = INBOX.restored:bytes=+500%% # allow for temporary folder } ... protocol lda { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins plugin { quota_rule7 = INBOX:bytes=+10%% } } Maybe I should have that quota_rule7 be *:bytes=+10%% ? thanks for any suggestions for what I am missing here! micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From p at state-of-mind.de Tue Nov 29 23:51:49 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 29 Nov 2011 22:51:49 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <20111129215149.GA29651@state-of-mind.de> * Ralf Hildebrandt : > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? There's an RFC, but it hasn't been adopted yet. Neither by IMAP servers nor by IMAP clients. ;) p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From tss at iki.fi Wed Nov 30 00:17:05 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 30 Nov 2011 00:17:05 +0200 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> On 29.11.2011, at 23.42, Ralf Hildebrandt wrote: > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? With v2.1 it would be somewhat easy to create a plugin that supports aliases, but each client would still see each others' Sent boxes and possibly synchronize everything twice.. From slusarz at curecanti.org Wed Nov 30 00:17:23 2011 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 29 Nov 2011 15:17:23 -0700 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129215149.GA29651@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> Message-ID: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Quoting Patrick Ben Koetter

: > * Ralf Hildebrandt : >> While our webmail installation creates "Sent" as a Sent-Items Folder, >> Apple Mail creates "Sent Messages". >> >> Is there a way of "mapping" foldernames e.g. >> >> map "Sent Messages" to "Sent" >> map "Deletes Messages" to "Trash" >> and so on? > > There's an RFC, but it hasn't been adopted yet. Neither by IMAP > servers nor by > IMAP clients. ;) Not true. We've (IMP) had special-use (RFC 6154) support since August 2010. Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. michael From p at state-of-mind.de Wed Nov 30 00:26:51 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 29 Nov 2011 23:26:51 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Message-ID: <20111129222651.GG10344@state-of-mind.de> * Michael M Slusarz : > Quoting Patrick Ben Koetter

: > > >* Ralf Hildebrandt : > >>While our webmail installation creates "Sent" as a Sent-Items Folder, > >>Apple Mail creates "Sent Messages". > >> > >>Is there a way of "mapping" foldernames e.g. > >> > >>map "Sent Messages" to "Sent" > >>map "Deletes Messages" to "Trash" > >>and so on? > > > >There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >servers nor by > >IMAP clients. ;) > > Not true. We've (IMP) had special-use (RFC 6154) support since August 2010. > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. Very nice. Thanks for the update. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From stan at hardwarefreak.com Wed Nov 30 07:59:33 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 29 Nov 2011 23:59:33 -0600 Subject: [Dovecot] Best setup for performance and fault tolerance In-Reply-To: <6f4f-4ed54480-5-fba0340@98207411> References: <6f4f-4ed54480-5-fba0340@98207411> Message-ID: <4ED5C645.1000605@hardwarefreak.com> On 11/29/2011 2:46 PM, Donny Brooks wrote: > Hello all. I am in need of some guidance. First a little background. Currently our mail server is on physical hardware (Dell server with 2x 2.8GHz Xeon w/ 4GB ram, raid5 array, single gigabit nic) running on Fedora 11 and postfix-2.5.6-3.fc11.x86_64 with dovecot-1.2.11-3.fc11.x86_64. Mailstore is via Maildir format that was converted from mbox about a year ago. This same machine is also our PDC with samba, Master LDAP, slave MySQL replication, primary DNS, and home server for about 20 users. > > We have approximately 200 end users that have mailboxes on the server ranging from 1KB to 20GB in size. Total mail store is currently at 300GB. About 75 of the users are currently POP access and their mail will be moved to the server soon and setup as IMAP. This is calculated to add roughly another 150GB of mail for a total of 450GB mail store. Being a state agency we have to keep the mail indefinitely for public record reasons. We use a mixture of Thunderbird as an IMAP client and SOGo for web access. > > Now to the problem: Recently we have been having super slow access to the mail server. Turns out the load was insanely high partially due to the samba home server portion, which is being moved off as we speak, and the other part is due to people searching their mail. Just yesterday one of our users nearly brought the entire agency to its knees by performing a search on her 8GB of mail via IMAP. > > Since the server is old in both hardware and software I have been tasked with moving it to newer hardware and a newer OS. We currently have 3 virtual servers running Xen and a SAN. The new setup will be placed in the virtual environment. I will probably run Fedora 16 as the OS but am open to Centos, Fedora, or Ubuntu. > > Now to the question: What is the best way to setup Dovecot so that it is tuned for performance and high available? We have been running with this single point of failure for years so as long as we are moving the mail server we might as well build in some redundancy. To solve the searching problem I thought of maybe setting up some type of indexing. I do kind of want to break the various services out on to separate virtual machines for a little more fault tolerance, but that is not totally necessary. > > What do you think of things like iRedmail? I see it's usefulness but the not being able to separate services kind of defeats the purpose, plus I want to setup a high available MySQL cluster and possibly OpenLDAP or 389 cluster so iRedMail may not be the best solution. > > Sorry for the long email but I am trying to get all the information out there at once so it will help get more directed responses in the shortest amount of time. I look forward to any and all input on this matter > -- > Donny B > MDAH Build an Enkive server: http://www.enkive.org/ and configure your SMTP MTAs to transparently copy all email to it (recipient_bcc for example). This fulfills your retention requirements. Since all emails are now archived by Enkive as they arrive, cron a nightly script on the Dovecot server to delete any emails over a week/month/etc old (depending on your short term access needs) from your active Dovecot mailboxes. This drastically reduces your Dovecot storage requirements. Searches will be performed by the Enkive server, removing that load from your Dovecot host. Search interface demo here: http://www.enkive.org/demo In addition your total mail storage (active+archive) requirement will be a fraction of what it is now because Enkive performs deduplication of email content and attachments so you save even more disk space. Enkive can run fine as a VM if you give it the required resources. Carve an appropriately sized LUN off the SAN array for the Enkive server storage. Format it with XFS for best performance. Enkive should be a good fit for your needs. Bear in mind installing/configuring it is not for the faint of heart: http://wiki.enkive.org/index.php/Installation_Instructions http://wiki.enkive.org/index.php/GettingMailIntoEnkive But thankfully administration is relatively easy: http://wiki.enkive.org/index.php/Administrator_Manual -- Stan From robert at schetterer.org Wed Nov 30 08:48:36 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:48:36 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <4ED5D1C4.8010607@schetterer.org> Am 29.11.2011 22:42, schrieb Ralf Hildebrandt: > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? > wish i would know a solution i know newest outlook can configured equal to tb folders default schema , except spam folder -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 08:50:39 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:50:39 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> References: <20111129214256.GL14970@charite.de> <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> Message-ID: <4ED5D23F.2030609@schetterer.org> Am 29.11.2011 23:17, schrieb Timo Sirainen: > On 29.11.2011, at 23.42, Ralf Hildebrandt wrote: > >> While our webmail installation creates "Sent" as a Sent-Items Folder, >> Apple Mail creates "Sent Messages". >> >> Is there a way of "mapping" foldernames e.g. >> >> map "Sent Messages" to "Sent" >> map "Deletes Messages" to "Trash" >> and so on? > > With v2.1 it would be somewhat easy to create a plugin that supports aliases, but each client would still see each others' Sent boxes and possibly synchronize everything twice.. that would be no good solution....in...my eyes > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 08:53:08 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:53:08 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Message-ID: <4ED5D2D4.2090603@schetterer.org> Am 29.11.2011 23:17, schrieb Michael M Slusarz: > Quoting Patrick Ben Koetter

: > >> * Ralf Hildebrandt : >>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>> Apple Mail creates "Sent Messages". >>> >>> Is there a way of "mapping" foldernames e.g. >>> >>> map "Sent Messages" to "Sent" >>> map "Deletes Messages" to "Trash" >>> and so on? >> >> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >> servers nor by >> IMAP clients. ;) > > Not true. We've (IMP) had special-use (RFC 6154) support since August > 2010. > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > > michael > what helps this, tb, outlook, apple mail, winmail and some mobile clients must follow , this is what people use, and it will take years after some of them might upgrade -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From akam at zakamye.ru Wed Nov 30 08:57:57 2011 From: akam at zakamye.ru (=?UTF-8?B?0JDQudC00LDRgCDQmtCw0LzQsNC70L7Qsg==?=) Date: Wed, 30 Nov 2011 10:57:57 +0400 Subject: [Dovecot] second active directory Message-ID: <4ED5D3F5.1020102@zakamye.ru> Hello! There is dovecot authenticating through AD. All is working well, if authentication is just one of AD. But there is another AD, and working with 2 AD's dovecot don't recognizes last AD user's home directory. For example, if user name in the second AD - 1developer and Mail - it-2c at zakamye.ru then dovecot creates folder /var/mail/zakamye.ru/1cdeveloper but it should be /var/mail/zakamye.ru/it-2c. What I do wrong?( And, if I change the order of viewing AD, does not work precisely the second AD: passdb ldap-DUSLYK { driver = ldap args = /etc/dovecot/dovecot-DUSLYK.conf } userdb ldap-DUSLYK { driver = ldap args = /etc/dovecot/dovecot-DUSLYK-users.conf } passdb ldap-DUSL-OFFICE { driver = ldap args = /etc/dovecot/dovecot-DUSL-OFFICE.conf } userdb ldap-DUSL-BASE { driver = ldap args = /etc/dovecot/dovecot-DUSL-OFFICE-users.conf } Sorry for my bad english, but maybe you can understand logs and configs: http://pastebin.com/mtk4XsNE -- ????? ?. ???????, ????????? ????????????? ??? "?? ??????" ?. ???. ?????, ???. ????????? ???.: +7 (8552) 47-01-99 ???. 3025 ???.: +7 (8552) 70-65-15 ???. 280 ???.: +7 (919) 621-28-52 E-mail: akam at zakamye.ru From p at state-of-mind.de Wed Nov 30 10:21:42 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 30 Nov 2011 09:21:42 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5D2D4.2090603@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> Message-ID: <20111130082141.GB1594@state-of-mind.de> * Robert Schetterer : > Am 29.11.2011 23:17, schrieb Michael M Slusarz: > > Quoting Patrick Ben Koetter

: > > > >> * Ralf Hildebrandt : > >>> While our webmail installation creates "Sent" as a Sent-Items Folder, > >>> Apple Mail creates "Sent Messages". > >>> > >>> Is there a way of "mapping" foldernames e.g. > >>> > >>> map "Sent Messages" to "Sent" > >>> map "Deletes Messages" to "Trash" > >>> and so on? > >> > >> There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >> servers nor by > >> IMAP clients. ;) > > > > Not true. We've (IMP) had special-use (RFC 6154) support since August > > 2010. > > > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > > > > michael > > > > what helps this, tb, outlook, apple mail, winmail and some mobile > clients must follow , this is what people use, and it will take years > after some of them might upgrade What helps this == Was hilft es? TB has it on the CR list. Apple has their own X-LIST feature together with google. p at rick > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Wed Nov 30 10:25:46 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 09:25:46 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111130082141.GB1594@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> Message-ID: <4ED5E88A.20205@schetterer.org> Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: > * Robert Schetterer : >> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>> Quoting Patrick Ben Koetter

: >>> >>>> * Ralf Hildebrandt : >>>>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>>>> Apple Mail creates "Sent Messages". >>>>> >>>>> Is there a way of "mapping" foldernames e.g. >>>>> >>>>> map "Sent Messages" to "Sent" >>>>> map "Deletes Messages" to "Trash" >>>>> and so on? >>>> >>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>> servers nor by >>>> IMAP clients. ;) >>> >>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>> 2010. >>> >>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>> >>> michael >>> >> >> what helps this, tb, outlook, apple mail, winmail and some mobile >> clients must follow , this is what people use, and it will take years >> after some of them might upgrade > > What helps this == Was hilft es? > > TB has it on the CR list. > Apple has their own X-LIST feature together with google. cool, *g left another hundred clients however having this widly solved would be a dream why i did take so long for it.........., its a Problem since years > > p at rick > > > > > >> >> -- >> Best Regards >> >> MfG Robert Schetterer >> >> Germany/Munich/Bavaria > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 10:33:24 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 09:33:24 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5E88A.20205@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> Message-ID: <4ED5EA54.7000007@schetterer.org> Am 30.11.2011 09:25, schrieb Robert Schetterer: > Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: >> * Robert Schetterer : >>> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>>> Quoting Patrick Ben Koetter

: >>>> >>>>> * Ralf Hildebrandt : >>>>>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>>>>> Apple Mail creates "Sent Messages". >>>>>> >>>>>> Is there a way of "mapping" foldernames e.g. >>>>>> >>>>>> map "Sent Messages" to "Sent" >>>>>> map "Deletes Messages" to "Trash" >>>>>> and so on? >>>>> >>>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>>> servers nor by >>>>> IMAP clients. ;) >>>> >>>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>>> 2010. >>>> >>>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>>> >>>> michael >>>> >>> >>> what helps this, tb, outlook, apple mail, winmail and some mobile >>> clients must follow , this is what people use, and it will take years >>> after some of them might upgrade >> >> What helps this == Was hilft es? >> >> TB has it on the CR list. >> Apple has their own X-LIST feature together with google. > > cool, *g left another hundred clients > > however having this widly solved would be a dream > why i did take so long for it.........., its a Problem since years hoping big spreaded clients will solve it soon perhaps a chance to block non fixed clients would be nice any chance ?, i see clients anounce their name and versions in debug log > >> >> p at rick >> >> >> >> >> >>> >>> -- >>> Best Regards >>> >>> MfG Robert Schetterer >>> >>> Germany/Munich/Bavaria >> > > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From p at state-of-mind.de Wed Nov 30 10:56:27 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 30 Nov 2011 09:56:27 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5E88A.20205@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> Message-ID: <20111130085626.GD1594@state-of-mind.de> * Robert Schetterer : > Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: > > * Robert Schetterer : > >> Am 29.11.2011 23:17, schrieb Michael M Slusarz: > >>> Quoting Patrick Ben Koetter

: > >>>> > >>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >>>> servers nor by IMAP clients. ;) > >>> > >>> Not true. We've (IMP) had special-use (RFC 6154) support since August > >>> 2010. > >>> > >>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > >>> > >>> michael > >>> > >> > >> what helps this, tb, outlook, apple mail, winmail and some mobile > >> clients must follow , this is what people use, and it will take years > >> after some of them might upgrade > > > > What helps this == Was hilft es? > > > > TB has it on the CR list. > > Apple has their own X-LIST feature together with google. > > cool, *g left another hundred clients > > however having this widly solved would be a dream > why i did take so long for it.........., its a Problem since years Yes, it is a problem, but having a standard doesn't solve it and open source software is not a guarant to have it implemented either, unless you contribute the code yourself or get to sponsor someone to do it. BTW: We'd contribute ? 500 if Timo implemented RFC 6154 in Dovecot 2.1. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Wed Nov 30 11:05:26 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 10:05:26 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111130085626.GD1594@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> <20111130085626.GD1594@state-of-mind.de> Message-ID: <4ED5F1D6.7000501@schetterer.org> Am 30.11.2011 09:56, schrieb Patrick Ben Koetter: > * Robert Schetterer : >> Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: >>> * Robert Schetterer : >>>> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>>>> Quoting Patrick Ben Koetter

: >>>>>> >>>>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>>>> servers nor by IMAP clients. ;) >>>>> >>>>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>>>> 2010. >>>>> >>>>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>>>> >>>>> michael >>>>> >>>> >>>> what helps this, tb, outlook, apple mail, winmail and some mobile >>>> clients must follow , this is what people use, and it will take years >>>> after some of them might upgrade >>> >>> What helps this == Was hilft es? >>> >>> TB has it on the CR list. >>> Apple has their own X-LIST feature together with google. >> >> cool, *g left another hundred clients >> >> however having this widly solved would be a dream >> why i did take so long for it.........., its a Problem since years > > Yes, it is a problem, but having a standard doesn't solve it and open source > software is not a guarant to have it implemented either, unless you contribute > the code yourself or get to sponsor someone to do it. > > BTW: We'd contribute ? 500 if Timo implemented RFC 6154 in Dovecot 2.1. very cool !!!!, as an add on idea, if it would be possible to identificate unfixed clients on the fly and doing some action on it, it might be a solution to allow imap at fixed clients only , others have to use pop3 then, not a solution everywhere but it could be implemented in some places ( i.e mine *g ) > > p at rick > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From info at simonecaruso.com Wed Nov 30 12:33:29 2011 From: info at simonecaruso.com (Simone Caruso) Date: Wed, 30 Nov 2011 11:33:29 +0100 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <4ED60679.1010903@simonecaruso.com> On 28/11/2011 23:14, Marcelo Salhab Brogliato wrote: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. > For imap i need only 1 VIP both for loadbalancing or HA, for SMTP u can use dns failover with multiple mx records. > About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). > I'm using dovecot 1.2.9. I suggest to not use local indexing (for high i/o after failover) , anyway NFS for HA setup works well! If u need a cluster (master/master) consider having an HA loadbancer (dovecot director): -- Simone Caruso IT Consultant +39 349 65 90 805 From akam at zakamye.ru Wed Nov 30 12:39:31 2011 From: akam at zakamye.ru (=?UTF-8?B?0JDQudC00LDRgCDQmtCw0LzQsNC70L7Qsg==?=) Date: Wed, 30 Nov 2011 14:39:31 +0400 Subject: [Dovecot] second active directory In-Reply-To: <4ED5D3F5.1020102@zakamye.ru> References: <4ED5D3F5.1020102@zakamye.ru> Message-ID: <4ED607E3.40805@zakamye.ru> SOLVED. Problem solved. There was double users in both AD domains. 30.11.2011 10:57, ????? ??????? ?????: > Hello! > There is dovecot authenticating through AD. All is working well, if > authentication is just one of AD. > But there is another AD, and working with 2 AD's dovecot don't > recognizes last AD user's home directory. > For example, if user name in the second AD - 1developer and Mail - > it-2c at zakamye.ru then dovecot creates folder > /var/mail/zakamye.ru/1cdeveloper but it should be > /var/mail/zakamye.ru/it-2c. > > What I do wrong?( > > And, if I change the order of viewing AD, does not work precisely the > second AD: > passdb ldap-DUSLYK { > driver = ldap > args = /etc/dovecot/dovecot-DUSLYK.conf > } > userdb ldap-DUSLYK { > driver = ldap > args = /etc/dovecot/dovecot-DUSLYK-users.conf > } > passdb ldap-DUSL-OFFICE { > driver = ldap > args = /etc/dovecot/dovecot-DUSL-OFFICE.conf > } > userdb ldap-DUSL-BASE { > driver = ldap > args = /etc/dovecot/dovecot-DUSL-OFFICE-users.conf > } > > Sorry for my bad english, but maybe you can understand logs and configs: > http://pastebin.com/mtk4XsNE > From heptalium at gmx.de Wed Nov 30 20:04:16 2011 From: heptalium at gmx.de (Jens =?UTF-8?B?TWVpw59uZXI=?=) Date: Wed, 30 Nov 2011 19:04:16 +0100 Subject: [Dovecot] lda/sieve ignores quota from ldap Message-ID: <20111130190416.641ede1f@meissner.pentament.dyndns.org> Hello, I'm using Dovecot 2.0 with Sieve as MDA and Postfix as MTA. The Postfix delivery command is /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT". I have set a default quota with quota_rule = *:storage=1G. Some users should have a larger quota. This is defined in a ldap directory and I have set user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mailQuota=quota_rule=*:storage=%$. The imap_quota plugin shows the right quota for all users but the lda ignores the quota rule from ldap. If a user is over default quota but not over his ldap defined quota I get the following error message: From: Mail Delivery Subsystem Your message to <[?]> was automatically rejected: Quota exceeded (mailbox for user is full) The log file shows the following: Nov 30 18:21:17 [?] dovecot: lda([?]): Error: sieve: msgid=<20111130172116.7F065A07D@[?]>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) Nov 30 18:21:17 [?] dovecot: lda([?]): Error: sieve: script /var/mail/[?]/dovecot.sieve failed with unsuccessful implicit keep (user logfile /var/mail/[?]/dovecot.sieve.log may reveal additional details) Nov 30 18:21:17 [?] dovecot: lda([?]): msgid=<20111130172116.7F065A07D@[?]>: rejected: Quota exceeded (mailbox for user is full) dovecot.sieve.log: error: msgid=<20111130172116.7F065A07D@[?]>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full). What is wrong? What must I do to getting this to work? Thanks in advance. Regards Jens doveconf -n: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.3 nfs mail_location = maildir:/var/mail/%u mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = maildir quota_rule = *:storage=1G sieve = /var/mail/%u/dovecot.sieve sieve_dir = /var/mail/%u/sieve } protocols = " imap sieve pop3" ssl_cert = From terry at cnysupport.com Wed Nov 30 22:36:46 2011 From: terry at cnysupport.com (Terry Carmen) Date: Wed, 30 Nov 2011 15:36:46 -0500 Subject: [Dovecot] MS Exchange IMAP Proxy Message-ID: <20111130153646.Horde.fehRaFeGiNBO1pPe5xWA0vA@www.cnysupport.com> I need to make the IMAP interface of an Exchange 2000 server available on the net, however I would like to give it a little protection, and believe Dovecot's IMAP proxy might be appropriate. Does anybody have a *really simple* config that would allow IMAP pass-through to a single Exchange server? The examples I've seen all assume some sort of load balancing, which isn't an issue here, so I'd like to eliminate the need for an unnecessary database server. Does Dovecot do any cleanup of the IMAP commands that would be useful in protecting Exchange (from this, http://www.cvedetails.com/cve/CVE-2007-0221/ for example), or am I barking up the wrong tree? If Dovecot isn't helpful for this, can anybody point me to a better resource? Thanks! Terry From nam_10 at hotmail.com Tue Nov 1 00:20:56 2011 From: nam_10 at hotmail.com (nuno marques) Date: Mon, 31 Oct 2011 22:20:56 +0000 Subject: [Dovecot] Imap/pop gateway Message-ID: Hello, How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. regards From tss at iki.fi Tue Nov 1 08:24:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Nov 2011 08:24:33 +0200 Subject: [Dovecot] dsync should sync sieve-dirs to! In-Reply-To: <201110290141.41963.p.heinlein@heinlein-support.de> References: <201110290141.41963.p.heinlein@heinlein-support.de> Message-ID: <6589FF32-A0F6-4E75-A6A3-65356164E563@iki.fi> On 29.10.2011, at 2.41, Peer Heinlein wrote: > Having dsync to make backups from existing mail-spaces, it would be nice > to make dsync syncing the sieve-dirs too. -Otherweise backups aren't > complete... dsync was originally meant to be a way to do two-way syncing of mailboxes without losing any changes. That's not really possible to do with Sieve scripts. I guess the best that could be done is to copy the one with the latest timestamp. But if you have multiple Sieve scripts even that doesn't work well, since dsync can't know if a script was added on one side or deleted from another side. Although if scripts were only managed via ManageSieve, the last problem could be solved with some log file. For "dsync backup" I guess it could be useful to have a setting with a list of files/dirs that dsync would always upload to remote.. From tss at iki.fi Tue Nov 1 08:32:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Nov 2011 08:32:47 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: On 30.10.2011, at 13.53, Lukas Wei? wrote: > ive just switched to dovecot imap because dbmail > i used before is not developed any further Oh? It doesn't yet seem dead to me. > and sql storage is difficult > to backup. Ive done that, by setting up dovecot with maildir format and > copy mails from old dbmail to new dovecot imap server with my mail > client (thunderbird). Thunderbird probably changes the INTERNALDATE of the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. From laxlaw at gmx.de Tue Nov 1 10:38:02 2011 From: laxlaw at gmx.de (=?UTF-8?Q?Lukas_Wei=C3=9F?=) Date: Tue, 01 Nov 2011 09:38:02 +0100 Subject: [Dovecot] =?utf-8?q?Bad_creation_timestamp_of_migrated_mails_=28m?= =?utf-8?q?aildir_format=29=3F?= In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: On Tue, 1 Nov 2011 08:32:47 +0200, Timo Sirainen wrote: > On 30.10.2011, at 13.53, Lukas Wei? wrote: > >> ive just switched to dovecot imap because dbmail i used before is not developed any further > Oh? It doesn't yet seem dead to me. > >> and sql storage is difficult to backup. Ive done that, by setting up dovecot with maildir format and copy mails from old dbmail to new dovecot imap server with my mail client (thunderbird). > Thunderbird probably changes the INTERNALDATE of the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. What is INTERNALDATE? The modification timestamp is correct ("ls -l"), and the "Date: ... " line, too. @dbmail: Dead is the wrong designation. But since the last 5 years i only noticed bugfixes, the most "ideas for developement" (STARTTLS for example) mentioned here [1] are still ideas. Links: ------ [1] http://www.dbmail.org/dokuwiki/doku.php/2.3 From taeuber at bbaw.de Tue Nov 1 13:21:49 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Tue, 1 Nov 2011 12:21:49 +0100 Subject: [Dovecot] antispam for dovecot 2.0 Message-ID: <20111101122149.b8028453.taeuber@bbaw.de> Hi! Is there anybody out there who has ported the antispam-storage-1.x.c to antispam-storage-2.0.c? Or is there an alternative to the antispam plugin? Thanks Lars From duihi77 at gmail.com Tue Nov 1 14:34:30 2011 From: duihi77 at gmail.com (Duane Hill) Date: Tue, 1 Nov 2011 07:34:30 -0500 (CDT) Subject: [Dovecot] antispam for dovecot 2.0 In-Reply-To: <20111101122149.b8028453.taeuber@bbaw.de> References: <20111101122149.b8028453.taeuber@bbaw.de> Message-ID: On Tue, 1 Nov 2011, Lars T?uber wrote: > Hi! > > Is there anybody out there who has ported the antispam-storage-1.x.c to > antispam-storage-2.0.c? > > Or is there an alternative to the antispam plugin? Don't know anything about antispam-storage. From searching I assume you mean the antispam plugin for Dovecot 2.x. http://wiki2.dovecot.org/Plugins/Antispam shows where you can get it from and compile. From taeuber at bbaw.de Tue Nov 1 15:58:55 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Tue, 1 Nov 2011 14:58:55 +0100 Subject: [Dovecot] antispam for dovecot 2.0 In-Reply-To: References: <20111101122149.b8028453.taeuber@bbaw.de> Message-ID: <20111101145855.d14e671c.taeuber@bbaw.de> Am Tue, 1 Nov 2011 07:34:30 -0500 (CDT) Duane Hill schrieb: > On Tue, 1 Nov 2011, Lars T?uber wrote: > > > Hi! > > > > Is there anybody out there who has ported the antispam-storage-1.x.c to > > antispam-storage-2.0.c? > > > > Or is there an alternative to the antispam plugin? > > Don't know anything about antispam-storage. From searching I assume you > mean the antispam plugin for Dovecot 2.x. > > http://wiki2.dovecot.org/Plugins/Antispam shows where you can get it from > and compile. Hallo Duane, thank you very much. This is what I was searching for. Thanks Lars From dvstillerfan74 at gmail.com Tue Nov 1 17:55:36 2011 From: dvstillerfan74 at gmail.com (David Varela) Date: Tue, 1 Nov 2011 09:55:36 -0600 Subject: [Dovecot] LDAP w/SASL "Active Directory" authentication failing. Message-ID: Hello, I am running a Dovecot server (version 1.2.17) on FreeBSD 8.2, using LDAP to authenticate Active Directory users. I can successfully bind and authenticate using PLAIN and LDAP without SASL, but obviously passwords for the bind user and the user being authenticated are being passed in plain text. I've attempted to configure my server to us SASL however when I attempt to authenticate a user I see authentication failures. I reviewed the security log on my domain controller and see that the bind user is binding properly, so the issue appears to be orginating from the user authentication, however I cannot determine what the issue is. Here is all the information regarding my configuration, along with the logs from the server: Dovecot -n # 1.2.17: /usr/local/etc/dovecot.conf # OS: FreeBSD 8.2-RELEASE i386 ufs log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot_info.log protocols: imap pop3 imaps pop3s ssl_cert_file: /etc/ssl/dovecot/cert.pem ssl_key_file: /etc/ssl//dovecot/key.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login valid_chroot_dirs: /usr/data/vmail verbose_proctitle: yes first_valid_uid: 1001 last_valid_uid: 1001 first_valid_gid: 1001 last_valid_gid: 1001 mail_privileged_group: mail mail_location: maildir:/usr/data/vmail/%u mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain gssapi ntlm login username_format: %Ln winbind_helper_path: /usr/local/bin/ntlm_auth verbose: yes debug: yes debug_passwords: yes use_winbind: yes passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf userdb: driver: static args: uid=vmail gid=vmail home=/usr/data/vmail/%u socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 dovecot-ldap.conf hosts = 192.168.0.240:3268 dn = cn=PostfixBind,ou=IT,ou=Central Office,ou=LMC,dc=smallmountain,dc=Local dnpass = y0urm0mma sasl_bind = yes #sasl_mech = GSSAPI #sasl_realm = smallmountain.local #sasl_authz_id = debug_level = -1 #auth_bind = yes ldap_version = 3 base = dc=smallmountain,dc=Local deref = never scope = subtree user_filter = (&(objectClass=person)(mail=%u)) pass_attrs = mail=user pass_filter = (&(objectClass=person)(sAMAccountName=%n)) dovecot.log Nov 01 09:09:48 dovecot: Warning: Killed with signal 15 (by pid=99758 uid=0 code=kill) Nov 01 09:09:48 auth(default): Error: ldap_pvt_sasl_getmech Nov 01 09:09:48 auth(default): Error: ldap_search Nov 01 09:09:48 auth(default): Error: put_filter: "(objectclass=*)" Nov 01 09:09:48 auth(default): Error: put_filter: simple Nov 01 09:09:48 auth(default): Error: put_simple_filter: "objectclass=*" Nov 01 09:09:48 auth(default): Error: ldap_build_search_req ATTRS: supportedSASLMechanisms Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_new_connection 1 1 0 Nov 01 09:09:48 auth(default): Error: ldap_int_open_connection Nov 01 09:09:48 auth(default): Error: ldap_connect_to_host: TCP 192.168.0.240:3268 Nov 01 09:09:48 auth(default): Error: ldap_new_socket: 11 Nov 01 09:09:48 auth(default): Error: ldap_prepare_socket: 11 Nov 01 09:09:48 auth(default): Error: ldap_connect_to_host: Trying 192.168.0.240:3268 Nov 01 09:09:48 auth(default): Error: ldap_pvt_connect: fd: 11 tm: -1 async: 0 Nov 01 09:09:48 auth(default): Error: ldap_open_defconn: successful Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 1 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 1, origid 1, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 message type search-entry Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 1, origid 1, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: * msgid 1, type 100 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 message type search-result Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 1, msgid 1) Nov 01 09:09:48 auth(default): Error: adding response ld 0x18611700 msgid 1 type 101: Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: ldap_get_values Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_sasl_interactive_bind: server supports: GSSAPI GSS-SPNEGO EXTERNAL DIGEST-MD5 Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: GSSAPI GSS-SPNEGO EXTERNAL DIGEST-MD5 Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_open: host=adminserver-1.smallmountain.local Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 2 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 2, origid 2, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 2 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: res_errno: 14, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 2, msgid 2) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: sasl_client_step: 2 Nov 01 09:09:48 auth(default): Error: sasl_client_step: 1 Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 3 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 3, origid 3, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 3 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: res_errno: 14, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 3, msgid 3) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: sasl_client_step: 0 Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 4 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 4, origid 4, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 4 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 4, msgid 4) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: ldap_pvt_sasl_generic_install Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:10:13 auth(default): Error: ldap_search Nov 01 09:10:13 auth(default): Error: put_filter: "(&(objectClass=person)(sAMAccountName=davidv))" Nov 01 09:10:13 auth(default): Error: put_filter: AND Nov 01 09:10:13 auth(default): Error: put_filter_list "(objectClass=person)(sAMAccountName=davidv)" Nov 01 09:10:13 auth(default): Error: put_filter: "(objectClass=person)" Nov 01 09:10:13 auth(default): Error: put_filter: simple Nov 01 09:10:13 auth(default): Error: put_simple_filter: "objectClass=person" Nov 01 09:10:13 auth(default): Error: put_filter: "(sAMAccountName=davidv)" Nov 01 09:10:13 auth(default): Error: put_filter: simple Nov 01 09:10:13 auth(default): Error: put_simple_filter: "sAMAccountName=davidv" Nov 01 09:10:13 auth(default): Error: ldap_build_search_req ATTRS: mail Nov 01 09:10:13 auth(default): Error: ldap_send_initial_request Nov 01 09:10:13 auth(default): Error: ldap_send_server_request Nov 01 09:10:13 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:10:13 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:10:13 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:10:13 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:10:13 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:10:13 auth(default): Error: last used: Tue Nov 1 09:10:13 2011 Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:10:13 auth(default): Error: * msgid 5, origid 5, status InProgress Nov 01 09:10:13 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:10:13 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:10:13 auth(default): Error: ldap_int_select Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid 5 message type search-entry Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid 5 message type search-result Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:10:13 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 5 Nov 01 09:10:13 auth(default): Error: request done: ld 0x18611700 msgid 5 Nov 01 09:10:13 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:10:13 auth(default): Error: ldap_free_request (origid 5, msgid 5) Nov 01 09:10:13 auth(default): Error: adding response ld 0x18611700 msgid 5 type 101: Nov 01 09:10:13 auth(default): Error: ldap_parse_result Nov 01 09:10:13 auth(default): Error: ldap_first_attribute Nov 01 09:10:13 auth(default): Error: ldap_get_values Nov 01 09:10:13 auth(default): Error: ldap_next_attribute Nov 01 09:10:13 auth(default): Error: ldap_msgfree Nov 01 09:10:13 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:10:13 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:10:13 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:10:13 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:10:13 auth(default): Error: refcnt: 1 status: Connected Nov 01 09:10:13 auth(default): Error: last used: Tue Nov 1 09:10:13 2011 Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 request count 0 (abandoned 0) Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:10:13 auth(default): Error: ldap_int_select Nov 01 09:13:26 auth(default): Error: ldap_search Nov 01 09:13:26 auth(default): Error: put_filter: "(&(objectClass=person)(sAMAccountName=davidv))" Nov 01 09:13:26 auth(default): Error: put_filter: AND Nov 01 09:13:26 auth(default): Error: put_filter_list "(objectClass=person)(sAMAccountName=davidv)" Nov 01 09:13:26 auth(default): Error: put_filter: "(objectClass=person)" Nov 01 09:13:26 auth(default): Error: put_filter: simple Nov 01 09:13:26 auth(default): Error: put_simple_filter: "objectClass=person" Nov 01 09:13:26 auth(default): Error: put_filter: "(sAMAccountName=davidv)" Nov 01 09:13:26 auth(default): Error: put_filter: simple Nov 01 09:13:26 auth(default): Error: put_simple_filter: "sAMAccountName=davidv" Nov 01 09:13:26 auth(default): Error: ldap_build_search_req ATTRS: mail Nov 01 09:13:26 auth(default): Error: ldap_send_initial_request Nov 01 09:13:26 auth(default): Error: ldap_send_server_request Nov 01 09:13:26 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:13:26 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:13:26 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:13:26 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:13:26 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:13:26 auth(default): Error: last used: Tue Nov 1 09:13:26 2011 Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:13:26 auth(default): Error: * msgid 6, origid 6, status InProgress Nov 01 09:13:26 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:13:26 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:13:26 auth(default): Error: ldap_int_select Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid 6 message type search-entry Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid 6 message type search-result Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:13:26 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 6 Nov 01 09:13:26 auth(default): Error: request done: ld 0x18611700 msgid 6 Nov 01 09:13:26 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:13:26 auth(default): Error: ldap_free_request (origid 6, msgid 6) Nov 01 09:13:26 auth(default): Error: adding response ld 0x18611700 msgid 6 type 101: Nov 01 09:13:26 auth(default): Error: ldap_parse_result Nov 01 09:13:26 auth(default): Error: ldap_first_attribute Nov 01 09:13:26 auth(default): Error: ldap_get_values Nov 01 09:13:26 auth(default): Error: ldap_next_attribute Nov 01 09:13:26 auth(default): Error: ldap_msgfree Nov 01 09:13:26 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:13:26 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:13:26 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:13:26 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:13:26 auth(default): Error: refcnt: 1 status: Connected Nov 01 09:13:26 auth(default): Error: last used: Tue Nov 1 09:13:26 2011 Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 request count 0 (abandoned 0) Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:13:26 auth(default): Error: ldap_int_select dovecot_info.log Nov 01 09:09:48 dovecot: Info: Dovecot v1.2.17 starting up Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99773 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99774 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99775 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99776 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99777 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99778 Nov 01 09:10:09 auth(default): Info: new auth connection: pid=99782 Nov 01 09:10:13 auth(default): Info: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=993 rport=55010 resp=AGRhdmlkdgBlbGkqNDJ0dW4= Nov 01 09:10:13 auth(default): Info: ldap(davidv,127.0.0.1): pass search: base=dc=smallmountain,dc=Local scope=subtree filter=(&(objectClass=person)(sAMAccountName=davidv)) fields=mail Nov 01 09:10:13 auth(default): Info: auth(davidv,127.0.0.1): username changed davidv -> davidv at smallmountain.net Nov 01 09:10:13 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): result: mail(user)=davidv at smallmountain.net Nov 01 09:10:13 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): No password returned (and no nopassword) Nov 01 09:10:15 auth(default): Info: client out: FAIL 1 user=davidv at smallmountain.net Nov 01 09:13:13 imap-login: Info: Disconnected: Inactivity (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Nov 01 09:13:21 auth(default): Info: new auth connection: pid=99797 Nov 01 09:13:26 auth(default): Info: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=993 rport=61080 resp=AGRhdmlkdgBlbGkqNDJ0dW4= Nov 01 09:13:26 auth(default): Info: ldap(davidv,127.0.0.1): pass search: base=dc=smallmountain,dc=Local scope=subtree filter=(&(objectClass=person)(sAMAccountName=davidv)) fields=mail Nov 01 09:13:26 auth(default): Info: auth(davidv,127.0.0.1): username changed davidv -> davidv at smallmountain.net Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): result: mail(user)=davidv at smallmountain.net Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): No password returned (and no nopassword) Nov 01 09:13:28 auth(default): Info: client out: FAIL 1 user=davidv at smallmountain.net Nov 01 09:13:36 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Your assistance is greatly appreciated, David From leo at strike.wu.ac.at Tue Nov 1 22:12:14 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Tue, 01 Nov 2011 21:12:14 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EAEDDDD.40206@gmx.de> References: <4EAED444.70608@strike.wu.ac.at> <4EAEDDDD.40206@gmx.de> Message-ID: <4EB0529E.2090808@strike.wu.ac.at> On 10/31/2011 06:41 PM, e-frog wrote: > On 31.10.2011 18:00, wrote Alexander 'Leo' Bergolth: >> Have environment based namespace settings been abandoned in 2.0? >> > I think you need to add them to USERDB_KEYS now... > > http://wiki2.dovecot.org/PostLoginScripting#Running_environment I have added NAMESPACE_1 NAMESPACE_1_SEP NAMESPACE_1_TYPE NAMESPACE_1_LIST NAMESPACE_1_PREFIX to USERDB_KEYS. Unfortunately the namespace still isn't used. :-( --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From pgnet.dev+rich-dovecot at gmail.com Tue Nov 1 22:53:47 2011 From: pgnet.dev+rich-dovecot at gmail.com (Rich) Date: Tue, 1 Nov 2011 13:53:47 -0700 Subject: [Dovecot] looking for Dovecot-code + SQL consultants Message-ID: Hi, We're using Dovecot2. Trying, given our own spread-too-thin bandwidth, to make it work within our evolving SQL application environment. When there's a problem, we post to this list (e.g., http://www.dovecot.org/list/dovecot/2011-October/061609.html), but aren't getting any/timely responses. We've decided to look for a consultant (hourly or retainer) that can be available for working with our in-house staff to straighten these issues out -- by helping us identify & fix our own mess, and by working to get fixes pushed to Dovecot project code, where appropriate. If you provide these services, rather than simply deployment or hosting, and are available, please drop me a line *offlist*. We're in the San Francisco area, and local is best, but remote work is certainly an option. Thanks, Rich From ghandidrivesahumvee at rocketfish.com Tue Nov 1 22:57:05 2011 From: ghandidrivesahumvee at rocketfish.com (Dovecot-GDH) Date: Tue, 1 Nov 2011 13:57:05 -0700 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EA916D1.30409@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: If I/O performance is a concern, you may be interested in ZFS and Flashcache. Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) ZFS does run on Linux http://zfs-fuse.net Flashcache: https://github.com/facebook/flashcache/ Both of these techniques can use a pair of SSDs in RAID1 rather than a single SSD. On Oct 27, 2011, at 1:31 AM, Ed W wrote: > On 27/10/2011 03:36, Stan Hoeppner wrote: >> On 10/26/2011 4:13 PM, Patrick Westenberg wrote: >>> Hi all, >>> >>> is anyone on this list who dares/dared to store his index files on a >>> MLC-SSD? >> I have not. But I can tell you that a 32GB Corsair MLC SSD in my >> workstation died after 4 months of laughably light duty. It had nothing >> to do with cell life but low product quality. This was my first foray >> into SSD. The RMA replacement is still kickin after 2 months, >> thankfully. I'm holding my breath... >> >> Scanning the reviews on Newegg shows early MLC SSD failures across most >> brands, early being a year or less. Some models/sizes are worse than >> others. OCZ has a good reputation overall, but reviews show some of >> their models to be grenades. >> >> Thus, if you were to put indexes on SSD, you should strongly consider >> using a mirrored pair. >> > > I don't think you are saying that the advice varies here compared with > HDDs? I do agree that some SSDs are showing very early failures, but > it's only a tweak to the probability parameter compared with any other > storage medium. They ALL fail at some point, and generally well within > the life of the rest of the server. Some kind of failure planning is > necessary > > Caveat the potentially higher failures vs HDDs I don't see any reason > why an SSD shouldn't work well? (even more so if you are using maildir > where indexes can be regenerated). > > More interestingly: for small sizes like 32GB, has anyone played with > the "compressed ram with backing store" thing in newer kernels (that I > forget the name of now). I think it's been marketed for swap files, but > assuming I got the theory it could be used as a ram drive with slow > writeback to permanent storage? > > Good luck > > Ed W From pw at wk-serv.de Tue Nov 1 23:19:11 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 01 Nov 2011 22:19:11 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: <4EB0624F.90307@wk-serv.de> Dovecot-GDH schrieb: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was thinking about a SSD based LUN for the indexes. As I'm using multiple servers this LUN will use OCFS2. From dswartz at druber.com Tue Nov 1 23:40:39 2011 From: dswartz at druber.com (Dan Swartzendruber) Date: Tue, 1 Nov 2011 17:40:39 -0400 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB0624F.90307@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com><4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> Message-ID: <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> I can't imagine running any kind of performance critical app on linux using fuse! There is a native ZFS port going on, but I don't know how stable it is yet. -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Patrick Westenberg Sent: Tuesday, November 01, 2011 5:19 PM To: dovecot at dovecot.org Subject: Re: [Dovecot] Indexes to MLC-SSD Dovecot-GDH schrieb: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was thinking about a SSD based LUN for the indexes. As I'm using multiple servers this LUN will use OCFS2. From janfrode at tanso.net Wed Nov 2 10:57:15 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 2 Nov 2011 09:57:15 +0100 Subject: [Dovecot] doveadm delete-user ? Message-ID: <20111102085715.GA4089@dibs.tanso.net> IMHO it would be great if we could both provision and un-provision users without having the provisioning system mess with files in the filesystem. Provisioning new users works fine since dovecot will create all files/directories on first login, but I haven?t found the tools to un-provison them. I see "doveadm mailbox delete" can delete individual mailboxes, but what about indexes, sieve-scripts and home-directories? Are there any tools to clean out these ? -jf From goetz.reinicke at filmakademie.de Wed Nov 2 15:59:21 2011 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Wed, 02 Nov 2011 14:59:21 +0100 Subject: [Dovecot] Upgrading dovecot 1.0.7 -> 2.0.14 / Redhat EL 5.7 Message-ID: <4EB14CB9.9030708@filmakademie.de> Hi, recently I debugged some load and performance and 'log out' problems and found an error in the dovecot logs: Error: pipe() failed: Too many open files Googling around and reading some posts led to the conclusion, that my problem is a bug, which will not be removed. So I'm thinking of upgrading to the latest prebuild rpm from here: http://packages.atrpms.net/dist/el5/dovecot/ As far as I understand the upgrading instructions, I'll be save as we dont use special setting, but I'll check all changes. My question: Dose anybody has done such an upgrade yet? Or are there any suggestion or hints? Thanks a lot and best regards. G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From arif at team.de.inter.net Wed Nov 2 16:28:05 2011 From: arif at team.de.inter.net (Kai Arif) Date: Wed, 02 Nov 2011 15:28:05 +0100 Subject: [Dovecot] Different default realm per listener Message-ID: <4EB15375.30106@team.de.inter.net> Hi all! I want create a large IMAP server setup where dovecot is listening on multiple interfaces and needs a different default realm on each interface. I'm using dovecot 2.0.5 on FreeBSD9 and authenticate against a LDAP database. I found an old mail thread from 2007 that said I'm basically screwed. Has that change since than? Is there any way to use different default realms per listener? Best regards Kai -- Kai Arif - System Administrator Inter.net Germany GmbH Knesebeckstra?e 59-61 10719 Berlin Germany Fon +49 30 25430 0 Fax +49 30 25430 499 arif at team.de.inter.net www.de.inter.net Legal-Information: Inter.net Germany GmbH, HRB #79136, Amtsgericht Berlin Charlottenburg, UST-IdNr:: DE 813 165 159 FA f?r K?rperschaften I Berlin, Gesch?ftsf?hrer: J?rn Lubkoll Zust?ndige Regulierungsbeh?rde: Bundesnetzagentur Chemnitz, Liselotte-Herrmann-Str. 20a, 09127 Chemnitz, Registriernummer: 06/164 From rick at havokmon.com Wed Nov 2 16:37:47 2011 From: rick at havokmon.com (Rick Romero) Date: Wed, 02 Nov 2011 09:37:47 -0500 Subject: [Dovecot] IMP, Dovecot and multiple namespaces Message-ID: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> Hey guys, I'm wondering what the best way is to be backwards compatible with Courier mailbox formats and not duplicate mailbox trees with Dovecot. Is anyone doing this right now? My Dovecot 2.0.13 is configured as follows for Namespaces: namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } So I should have an INBOX, plus additional root folders - rather than the Courier default of all folders being subfolders of the INBOX. So when I run LIST in IMAP I get - 08 LIST "" % * LIST (\HasChildren) "." "INBOX" * LIST (\HasChildren) "." "Spam" * LIST (\HasNoChildren) "." "Sent" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "Junk" * LIST (\HasNoChildren) "." "Drafts" Which looks fine. The problem is, IMP (old and new) show two sets of mailboxes (one for each namespace) like so: INBOX Spam Spam.Over10 Sent INBOX.Spam INBOX.Spam.Over10 INBOX.Sent (etc) I assume IMP is walking the INBOX because it says it has Children, and displaying those Children - The Horde team is telling me that's the root of the problem. It makes sense, but I'm not sure what the real solution is. Is it possible to disable Children for a mailbox when an alternate hidden namespace matches the mailbox name? Does that make sense? Or, is there another option I change change in the dovecot config to accomplish the same? Rick From jtam.home at gmail.com Thu Nov 3 02:43:04 2011 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 2 Nov 2011 17:43:04 -0700 (PDT) Subject: [Dovecot] doveadm delete-user ? In-Reply-To: References: Message-ID: Jan-Frode Myklebust writes: > IMHO it would be great if we could both provision and un-provision users > without having the provisioning system mess with files in the > filesystem. Provisioning new users works fine since dovecot will create > all files/directories on first login, but I haven?t found the tools to > un-provison them. > I see "doveadm mailbox delete" can delete individual > mailboxes, but what about indexes, sieve-scripts and home-directories? > Are there any tools to clean out these ? I coundn't find any, so I wrote my own executable cron job that scans the index cache and checks if the corresponding mailbox exists. If not, it echo the commands to remove the stale indices. From there, you can review it, or pipe it into sh. I can post it if there's any interest, but it's not that hard to write. This is only necessary, of course, if you store the indices in a different place than the mail folder. Joseph Tam From scott_the_musician at yahoo.com.au Thu Nov 3 07:31:54 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Wed, 2 Nov 2011 22:31:54 -0700 (PDT) Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? Message-ID: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Hi all, I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. Here's what happens: [root at mail ~]# whereis dovecot dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot ... checking whether to build static libraries... yes dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH to give path to compiled Dovecot sources or to a directory with the installed dovecot-config file. configure: error: dovecot-config not found -- I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: Avelsieve plugin details: backend = ManageSieve ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? thanks in advance! From lists at wildgooses.com Thu Nov 3 10:29:32 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 08:29:32 +0000 Subject: [Dovecot] Thunderbird slow in talking with dovecot IMAP AND to sendmail In-Reply-To: <4EA68C11.5060102@tlinx.org> References: <4EA68C11.5060102@tlinx.org> Message-ID: <4EB250EC.90204@wildgooses.com> On 25/10/2011 11:14, Linda Walsh wrote: > > > I'm trying to find out what's causing this slowdown -- it's > INTOLERABLE.... > > over 1 minute and less than 1% done. (400MB file)... > > After trying 3 times, I gave up and logged in using X to the server > and ran Tbird from there.... > > Mail sent out in < 1 minute, though the copy to dovecot took about 50% > longer. > > So... > > I looked at the network trace. > > and everyfrackin' body was using 4K packet sizes (at the application > level!, the window size on TCP was over 64K...but no one was using > it)....especially galling with my network's MTU at 9K, BTW, because > small packets are really bad on a 1Gb network. > Although larger packets might be helpful, I don't see that you shouldn't be getting much faster speed without it? Even the 64K window, whilst it looks too small, might be ok if your ping times are very low? Something else is limiting your performance I think? Ed W From maria_arrea at gmx.com Thu Nov 3 11:54:24 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 10:54:24 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103095424.67160@gmx.com> Hello. We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 Nov 3 09:23:07 buzon dovecot: imap(delolmo at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/15/77/delolmo/subscriptions Nov 3 09:23:07 buzon dovecot: imap(delolmo at mydomain): Connection closed bytes=0/295 I have been googling but I only see problems with remote NFS, our setup does not use NFS. I give you doveconf -n output & mount options, if more info es needed, please ask. doveconf -n output # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.el5 x86_64 Red Hat Enterprise Linux Server release 5.7 (Tikanga) ext4 auth_cache_negative_ttl = 10 secs auth_cache_size = 10 M auth_cache_ttl = 2 mins auth_master_user_separator = * auth_mechanisms = plain login auth_worker_max_count = 3500 base_dir = /var/run/dovecot/ default_client_limit = 5000 default_process_limit = 6500 disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep delay-newmail tb-lsub-flags lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_fsync = never mail_gid = entrega mail_home = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/ mail_location = mdbox:/buzones/mydomain/%2.26Hn/%2.200Hn/%n:INDEX=/indices_dovecot/indices/%2.26Hn/%2.200Hn/%n mail_max_userip_connections = 15000 mail_plugins = " zlib acl quota autocreate" mail_uid = entrega managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mdbox_rotate_interval = 1 days mdbox_rotate_size = 60 M passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/usuario_maestro.txt driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile autocreate = SPAM autocreate2 = Sent autocreate3 = Drafts autocreate4 = Trash autosubscribe = SPAM autosubscribe2 = Sent autosubscribe3 = Drafts autosubscribe4 = Trash lda_mailbox_autosubscribe = yes quota = dict:Cuota de usuario::file:/buzones/cuotas/%n quota_rule2 = Trash:storage=+10%% quota_warning = storage=90%% aviso_cuota 90 %u sieve = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/dovecot.sieve sieve_dir = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/sieve/ zlib_save = gz zlib_save_level = 9 } pop3_no_flag_updates = yes protocols = pop3 imap sieve service anvil { client_limit = 25000 } service auth { client_limit = 28000 unix_listener auth-master { user = entrega } unix_listener auth-userdb { user = entrega } user = root } service aviso_cuota { executable = script /usr/local/bin/quota-warning.sh unix_listener aviso_cuota { mode = 0666 } user = entrega } service imap-login { executable = /usr/libexec/dovecot/imap-login group = dovenull service_count = 0 } service imap { executable = /usr/libexec/dovecot/imap process_limit = 6000 } service managesieve-login { executable = /usr/libexec/dovecot/managesieve-login inet_listener sieve { port = 2000 } process_limit = 2000 } service managesieve { executable = /usr/libexec/dovecot/managesieve process_limit = 5000 } service pop3-login { executable = /usr/libexec/dovecot/pop3-login process_limit = 4000 service_count = 0 } service pop3 { executable = /usr/libexec/dovecot/pop3 process_limit = 4000 } ssl_ca = References: <20111103095424.67160@gmx.com> Message-ID: <4EB26C2F.1050505@wildgooses.com> Hi > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:30:09 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:30:09 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103103009.67160@gmx.com> We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:25 AM To: dovecot at dovecot.org Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Hi > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W From lists at wildgooses.com Thu Nov 3 12:31:21 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:31:21 +0000 Subject: [Dovecot] Imap/pop gateway In-Reply-To: References: Message-ID: <4EB26D79.1010206@wildgooses.com> On 31/10/2011 22:20, nuno marques wrote: > > > > Hello, > How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. > The question isn't entirely clear, but I *think* you just want to use the normal "proxy" feature of dovecot. This accepts connections on one machine, examines them until the end of the auth stage and passes them onto some other machine based on the results of the auth process Also there are other imap/pop proxies such as nginx That said I'm not sure how much security this really buys you versus port forwarding POP/IMAP ports to your real server? If the proxy machine were to get hacked (over imap?) then the same hack can jump from the proxy to the real server. Also your only exposure in each case is via POP/IMAP, which means you would be mainly chasing buffer overflow vulnerabilities and the like. These can also be mitigated by chrooting the server machine (please consider virtualisation options, it's usually simpler/faster/saner, eg see my favourite: linux-vservers), MAC controls on the dovecot process (grsec/selinux, etc), and compiler extensions (gcc hardened) Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:35:46 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:35:46 +0100 Subject: [Dovecot] Imap/pop gateway Message-ID: <20111103103546.67180@gmx.com> If you are going to use an imap proxy for security reasons, consider using a software DIFFERENT than in your real mailboxes. If you use dovecot in your backend, you could use perdition in the frontend. Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:31 AM To: Dovecot Mailing List Subject: Re: [Dovecot] Imap/pop gateway On 31/10/2011 22:20, nuno marques wrote: > > > > Hello, > How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. > The question isn't entirely clear, but I *think* you just want to use the normal "proxy" feature of dovecot. This accepts connections on one machine, examines them until the end of the auth stage and passes them onto some other machine based on the results of the auth process Also there are other imap/pop proxies such as nginx That said I'm not sure how much security this really buys you versus port forwarding POP/IMAP ports to your real server? If the proxy machine were to get hacked (over imap?) then the same hack can jump from the proxy to the real server. Also your only exposure in each case is via POP/IMAP, which means you would be mainly chasing buffer overflow vulnerabilities and the like. These can also be mitigated by chrooting the server machine (please consider virtualisation options, it's usually simpler/faster/saner, eg see my favourite: linux-vservers), MAC controls on the dovecot process (grsec/selinux, etc), and compiler extensions (gcc hardened) Good luck Ed W From giulio at dsi.unimi.it Thu Nov 3 12:38:17 2011 From: giulio at dsi.unimi.it (Giulio Casella) Date: Thu, 03 Nov 2011 11:38:17 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103103009.67160@gmx.com> References: <20111103103009.67160@gmx.com> Message-ID: <4EB26F19.1080105@dsi.unimi.it> This could be the problem. Double check the time also on your host system(s), not only on guest. Bye, gc Il 03/11/2011 11.30, Maria Arrea ha scritto: > We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:25 AM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > Hi> We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W > -- Giulio Casella giulio at dsi.unimi.it System and network manager Computer Science Dept. - University of Milano From lists at wildgooses.com Thu Nov 3 12:42:36 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:42:36 +0000 Subject: [Dovecot] How can we horizontally scale Dovecot across multiple servers? In-Reply-To: References: <003001cc97a9$addbeb80$0993c280$@mvs.us> <4EAE62AA.6080203@schetterer.org> <000301cc97b1$81100ca0$833025e0$@mvs.us> <4EAE71C1.7060400@schetterer.org> Message-ID: <4EB2701C.6060706@wildgooses.com> On 31/10/2011 11:28, Felipe Scarel wrote: > Quick question about the usage of DRBD: I'm thinking of a setup on my > organization here (15k+ users, 4TB of email data), but I'm holding back on > the clusterization due to the high volume of data. > > Using DRBD would implicate mirroring those 4TB of data across all cluster > nodes? If yes, I might go with a SAN-based solution, though I haven't I don't the technique with DRBD is something like having pairs of machines, each of which is a backup for the other. There were some old notes on the Dovecot website about such a setup? Roughly I seem to recall that each pair of machines ran two virtual machines, each of which ran active on one of the nodes each, but could migrate to the other if needed. Add a bunch of such paired nodes to get to the performance you require and put a dovecot proxy instance in front of the whole lot In contrast the SAN solution uses a clustered filesystem (opinion varies on which performs best) and then in theory every machine has access to every mailbox. In practice access to the SAN is relatively slow compared with local storage, so the technique seems to be to store indexes on the local machine and then using the front end proxy to be somewhat "sticky" in returning users to the same backend node so that the indexes can be re-used and not rebuilt The DRBD solution offers local disk access speed to the node and would on the surface give far faster performance (if disk were the limiting issue). However, it's likely to be more complex to maintain and manage and without buying licences you get only failover between pairs of machines. The SAN solution in theory looks like perfect scale up, big backend and just add more backend IMAP nodes as you need them, and all the clever stuff moves to the frontend load balancer to be "sticky" and obviously that's your main maintenance problem. However, based on evidence from users of big systems, IO is likely to be your main bottleneck and so just theoretically, the SAN will only scale as far as it doesn't run out of IOs... Using local disk for indexes would tend to reduce the amount of IOs needed (from the SAN) very dramatically, but you still have some limit out there and it's a question of whether you will reach it? DRBD has theoretical infinite scale out because each time you add another pair you get more IO as well as more CPU I don't have the fortune to have anything like the volume of users you have so I have no opinion to offer... However, I think the above accurately summarises your options. Others might help clarify the likely bounds on performance of each solution and maintenance headaches (eg some have had problems with maildir mounted on OCFS/GFS2 and fixed that by moving to dbox, etc) Please report on your results! Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:49:04 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:49:04 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103104904.67150@gmx.com> All the ESXs hosts and all the VM use the same NTP server. Any other idea? Regards Maria ----- Original Message ----- From: Giulio Casella Sent: 11/03/11 11:38 AM To: dovecot at dovecot.org Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube This could be the problem. Double check the time also on your host system(s), not only on guest. Bye, gc Il 03/11/2011 11.30, Maria Arrea ha scritto: > We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:25 AM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > Hi> We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W > -- Giulio Casella giulio at dsi.unimi.it System and network manager Computer Science Dept. - University of Milano From stephan at rename-it.nl Thu Nov 3 12:54:13 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 03 Nov 2011 11:54:13 +0100 Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Message-ID: <4EB272D5.1090403@rename-it.nl> Op 3-11-2011 6:31, Scott Lewis schreef: > Hi all, > > I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. > > Here's what happens: > > [root at mail ~]# whereis dovecot > dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz > > [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot > > ... > > checking whether to build static libraries... yes > dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH > to give path to compiled Dovecot sources or to a directory with the > installed dovecot-config file. configure: error: dovecot-config not found > > -- > > I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I'm not familiar with CentOS, but there usually is a separate package containing the Dovecot development headers and the dovecot-config file you need. By the looks of things, that is not installed at your end. Point the --with-dovecot to wherever the dovecot-config file is installed. Regards, Stephan. From lists at wildgooses.com Thu Nov 3 12:57:37 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:57:37 +0000 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103104904.67150@gmx.com> References: <20111103104904.67150@gmx.com> Message-ID: <4EB273A1.4030106@wildgooses.com> On 03/11/2011 10:49, Maria Arrea wrote: > All the ESXs hosts and all the VM use the same NTP server. > > Any other idea? > Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 13:03:36 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 12:03:36 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103110336.67190@gmx.com> We follow the guidelines about timekeeping RHEL in vmware vsphere located here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 These problems happens in peak hours. Any dovecot config parameter I could set to mitigate this problem? Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:57 AM To: Maria Arrea, Dovecot Mailing List Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube On 03/11/2011 10:49, Maria Arrea wrote: > All the ESXs hosts and all the VM use the same NTP server. > > Any other idea? > Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W From fbscarel at gmail.com Thu Nov 3 13:32:34 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Thu, 3 Nov 2011 09:32:34 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> Message-ID: I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ users, over 2TB of mail data) with little issues. Dedup and compression disabled, mind that. Dedup especially is a major source of trouble, I wouldn't recommend it for production just yet. Cheers, fbscarel On Tue, Nov 1, 2011 at 19:40, Dan Swartzendruber wrote: > > I can't imagine running any kind of performance critical app on linux using > fuse! There is a native ZFS port going on, but I don't know how stable it > is yet. > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On > Behalf Of Patrick Westenberg > Sent: Tuesday, November 01, 2011 5:19 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Indexes to MLC-SSD > > Dovecot-GDH schrieb: > > If I/O performance is a concern, you may be interested in ZFS and > Flashcache. > > > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive > Read Cache) > > ZFS does run on Linux http://zfs-fuse.net > > I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was > thinking about a SSD based LUN for the indexes. As I'm using multiple > servers this LUN will use OCFS2. > > From lists at wildgooses.com Thu Nov 3 16:27:16 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 14:27:16 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> Message-ID: <4EB2A4C4.2080100@wildgooses.com> On 03/11/2011 11:32, Felipe Scarel wrote: > I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ > users, over 2TB of mail data) with little issues. Dedup and compression > disabled, mind that. > OT: but what were the rough criteria that led you to using ZFS over say LVM with EXT4/XFS/btrfs? I can think of plenty for/against reasons for each, just wondering what criteria affected *your* situation? I'm guessing some kind of manageability reason is at the core, but perhaps you can expand on how it's all worked out for you? I have a fairly static server setup here so I have been "satisfied" with LVM, software raid and mainly ext4. The main thing I miss is simple to use snapshots Cheers Ed W From lists at wildgooses.com Thu Nov 3 17:21:01 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 15:21:01 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB0624F.90307@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> Message-ID: <4EB2B15D.1010005@wildgooses.com> > I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was > thinking about a SSD based LUN for the indexes. As I'm using multiple > servers this LUN will use OCFS2. Given that the SAN always has the network latency behind it, might you be better to look at putting the SSDs in the frontend machines? Obviously this then needs some way to make users "sticky" to one machine (or some few machines) where the indexes are stored? This seems theoretically likely to give you higher IOPs to the index than having them on the OCFS2 storage? (At a trade off with more complexity for the load balancer front end...) Ed W From alietsantiesteban at gmail.com Thu Nov 3 17:52:24 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Thu, 3 Nov 2011 11:52:24 -0400 Subject: [Dovecot] How to define ldap connection idle Message-ID: I'm having a problem with dovecot ldap connection when ldap server is in another firewall zone, firewall kills the ldap connection after a determined period of inactivity, this is good from the firewall point of view but is bad for dovecot because it never knows the connections has been dropped, this creates longs timeouts in dovecot and finally it reconnects, meanwhile many users fails to authenticate, I have seen this kind of post in the list for a while but can't find a solution for it, so my question is how to define a idle ldap time in dovecot so it can reconnect before the firewall has dropped the connection or just close the connection under inactivity so when a user authenticate doesn't fails for a while until dovecot detects that the connection has hanged. Is this a feature request or there is already a configuration for this??? Thank's in advance and congrats tu Timo for this great app. From pgnet.dev+rich-dovecot at gmail.com Thu Nov 3 18:37:13 2011 From: pgnet.dev+rich-dovecot at gmail.com (Rich) Date: Thu, 3 Nov 2011 09:37:13 -0700 Subject: [Dovecot] looking for Dovecot-code + SQL consultants In-Reply-To: References: Message-ID: Hi, I've already received a number of replies from providers offering to help out. I'll be in touch with each, and am certain we'll be able to find the right solution from among them. Thanks for the responses, Rich On Tue, Nov 1, 2011 at 1:53 PM, Rich wrote: > Hi, > > We're using Dovecot2. ?Trying, given our own spread-too-thin > bandwidth, to make it work within our evolving SQL application > environment. > > When there's a problem, we post to this list (e.g., > http://www.dovecot.org/list/dovecot/2011-October/061609.html), but > aren't getting any/timely responses. > > We've decided to look for a consultant (hourly or retainer) that can > be available for working with our in-house staff to straighten these > issues out -- by helping us identify & fix our own mess, and by > working to get fixes pushed to Dovecot project code, where > appropriate. > > If you provide these services, rather than simply deployment or > hosting, and are available, please drop me a line *offlist*. ?We're in > the San Francisco area, and local is best, but remote work is > certainly an option. > > Thanks, > > Rich > From pw at wk-serv.de Thu Nov 3 18:53:24 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 03 Nov 2011 17:53:24 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2B15D.1010005@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> Message-ID: <4EB2C704.1010107@wk-serv.de> Ed W schrieb: >> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >> thinking about a SSD based LUN for the indexes. As I'm using multiple >> servers this LUN will use OCFS2. > > Given that the SAN always has the network latency behind it, might you > be better to look at putting the SSDs in the frontend machines? > Obviously this then needs some way to make users "sticky" to one machine > (or some few machines) where the indexes are stored? Storing the indexes on several machines? In this case I have to synchronize them. From dswartz at druber.com Thu Nov 3 18:56:45 2011 From: dswartz at druber.com (Dan Swartzendruber) Date: Thu, 03 Nov 2011 12:56:45 -0400 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2C704.1010107@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> Message-ID: <4EB2C7CD.2090707@druber.com> Patrick Westenberg wrote: > Ed W schrieb: > >>> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >>> thinking about a SSD based LUN for the indexes. As I'm using multiple >>> servers this LUN will use OCFS2. >> >> Given that the SAN always has the network latency behind it, might you >> be better to look at putting the SSDs in the frontend machines? >> Obviously this then needs some way to make users "sticky" to one machine >> (or some few machines) where the indexes are stored? > > Storing the indexes on several machines? > In this case I have to synchronize them. > maybe i am missing something. if a client has to fetch the index, the server has to read the index from disk and pass it back. the network latency is unavoidable, but i don't see why putting the fastest possible SSD on the server isn't a win. possibly i am misunderstanding something? From thierry at mailhub.co.za Thu Nov 3 20:13:18 2011 From: thierry at mailhub.co.za (Thierry de Montaudry) Date: Thu, 3 Nov 2011 19:13:18 +0100 Subject: [Dovecot] Restricting IMAP access Message-ID: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Hi list, I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). Anybody has a trick for that? Running dovecot 2.0.13. I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. Regards, Thierry From fbscarel at gmail.com Thu Nov 3 20:24:06 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Thu, 3 Nov 2011 16:24:06 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2A4C4.2080100@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> Message-ID: Reasons to choose ZFS were snapshots, and mainly dedup and compression capabilities. I know, it's ironic since I'm not able to use them now due to severe performance issues with them (mostly dedup) turned on. I do like the emphasis on data integrity and fast on-the-fly configurability of ZFS to an extent, but I wouldn't recommend it highly for new users, especially for production. It works (in fact it's working right now), but has its fair share of troubles. We've started implementations to move our mail system to a more modular enviroment and we'll probably move away from ZFS. Was a nice experiment nonetheless, I learned quite a bit from it. On Thu, Nov 3, 2011 at 12:27, Ed W wrote: > On 03/11/2011 11:32, Felipe Scarel wrote: > > I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ > > users, over 2TB of mail data) with little issues. Dedup and compression > > disabled, mind that. > > > > OT: but what were the rough criteria that led you to using ZFS over say > LVM with EXT4/XFS/btrfs? I can think of plenty for/against reasons for > each, just wondering what criteria affected *your* situation? I'm > guessing some kind of manageability reason is at the core, but perhaps > you can expand on how it's all worked out for you? > > I have a fairly static server setup here so I have been "satisfied" with > LVM, software raid and mainly ext4. The main thing I miss is simple to > use snapshots > > Cheers > > Ed W > From robert at schetterer.org Thu Nov 3 20:46:16 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 03 Nov 2011 19:46:16 +0100 Subject: [Dovecot] Restricting IMAP access In-Reply-To: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> References: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Message-ID: <4EB2E178.8060103@schetterer.org> Am 03.11.2011 19:13, schrieb Thierry de Montaudry: > Hi list, > > I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. > I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). > Anybody has a trick for that? Running dovecot 2.0.13. > > I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. > > Regards, > > Thierry i am shot in time , with a databse this should be get to work i have it as flag for all users, so i can forbid imap to special ones as far i remember ther should be examples on the dovecot site , and it was written about here on the list before -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From simon.brereton at buongiorno.com Thu Nov 3 22:42:40 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 3 Nov 2011 16:42:40 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN Message-ID: Hi Could someone explain to me the difference between LOGIN and PLAIN? I've been googling for a while, but haven't found anything. Thanks. Simon From p at state-of-mind.de Thu Nov 3 23:00:24 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 3 Nov 2011 22:00:24 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <20111103210023.GB2312@state-of-mind.de> * Simon Brereton : > Could someone explain to me the difference between LOGIN and PLAIN? In SMTP these are: Both - are plaintext mechanisms. - base64 encode identification data before they send it over the wire - do not encrypt the indentification data and should therefore only be offered over an encrypted transport layer PLAIN - is an open standard supported by most clients - sends identification data as one string - sends an authentication ID, an authorization ID and the password LOGIN - is a proprietary standard supported by Microsofts clients - sends LOGIN, login name, password and optionally the domain name one after another I guess they are basically the same in IMAP, but others will know better. p at rick > I've been googling for a while, but haven't found anything. > > Thanks. > > Simon -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From stephan at rename-it.nl Thu Nov 3 23:01:17 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 03 Nov 2011 22:01:17 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <4EB3011D.80108@rename-it.nl> On 11/3/2011 9:42 PM, Simon Brereton wrote: > Hi > > Could someone explain to me the difference between LOGIN and PLAIN? > I've been googling for a while, but haven't found anything. The LOGIN SASL mechanism is an obsolete plain text mechanism. It is documented here: http://tools.ietf.org/html/draft-murchison-sasl-login-00 Some clients still support it, but I would not recommend using it when PLAIN or a better SASL mechanism is also available at both ends. The PLAIN mechanism is documented here: http://tools.ietf.org/html/rfc4616 The main technical difference between the two is that the PLAIN mechanism transfers both username and password in a single SASL interaction, where LOGIN needs two. The PLAIN mechanism also provides support for having an authorization id different from the authentication id, allowing for master user login for example. Regards, Stephan. From dovecot.user at seibercom.net Thu Nov 3 23:17:19 2011 From: dovecot.user at seibercom.net (Jerry) Date: Thu, 3 Nov 2011 17:17:19 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <20111103171719.55e0a7b9@scorpio> On Thu, 3 Nov 2011 16:42:40 -0400 Simon Brereton articulated: > Hi > > Could someone explain to me the difference between LOGIN and PLAIN? > I've been googling for a while, but haven't found anything. You could start here for some basic information: http://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer http://wiki.dovecot.org/Authentication/Mechanisms -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From simon.brereton at buongiorno.com Thu Nov 3 23:28:28 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 3 Nov 2011 17:28:28 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: <4EB3011D.80108@rename-it.nl> References: <4EB3011D.80108@rename-it.nl> Message-ID: On 3 November 2011 17:01, Stephan Bosch wrote: > On 11/3/2011 9:42 PM, Simon Brereton wrote: >> >> Hi >> >> Could someone explain to me the difference between LOGIN and PLAIN? >> I've been googling for a while, but haven't found anything. > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > documented here: > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > Some clients still support it, but I would not recommend using it when PLAIN > or a better SASL mechanism is also available at both ends. The PLAIN > mechanism is documented here: > > http://tools.ietf.org/html/rfc4616 > > The main technical difference between the two is that the PLAIN mechanism > transfers both username and password in a single SASL interaction, where > LOGIN needs two. The PLAIN mechanism also provides support for having an > authorization id different from the authentication id, allowing for master > user login for example. Thanks to both of you. Can I bet that Outlook doesn't support anything but plain? I'm not sure I've ever heard of a client supporting other than Evolution supporting MD5 passwords.. Simon From stan at hardwarefreak.com Fri Nov 4 01:01:29 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 03 Nov 2011 18:01:29 -0500 Subject: [Dovecot] How can we horizontally scale Dovecot across multiple servers? In-Reply-To: <003b01cc9565$c3b86960$4b293c20$@mvs.us> References: <003b01cc9565$c3b86960$4b293c20$@mvs.us> Message-ID: <4EB31D49.9090009@hardwarefreak.com> On 10/28/2011 6:35 AM, Arlin wrote: > How can we horizontally scale Dovecot across multiple servers? Do we require > to install independent instances of Dovecot on each server? The performance limiting factors for IMAP servers are disk IO, memory size, network bandwidth/latency, and processor power, usually in that order. To scale Dovecot horizontally would imply multiple servers each with independent storage handling a subset of the total concurrent user load. > We are planning to use a NAS/SAN device using ZFS or EFS for email storage. > Each logical unit will be of 10TB and similarly as the no: of user increases > we are planning to add multiple 10TB units. This does not describe horizontal maildir performance scaling. This simply describes vertical scaling of the IO, to a point, as the nodes are all attached to one controller with finite IOPS performance, which is dictated by the speed of the IO ASIC and cache memory bandwidth WRT writes, and aggregate spindle IOPS WRT reads. With this setup, you simply gain multi-node redundancy of the front end servers. However, you will *lose* performance compared to a single equivalent large host due to NFS/cluster filesystem overhead, latency, locking, etc, and your management/troubleshooting burden will be increased. If your goal is simply increased performance and storage expandability, go with a single large host system. I'd only go the cluster route if you believe you need the redundancy. -- Stan From stan at hardwarefreak.com Fri Nov 4 04:16:09 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 03 Nov 2011 21:16:09 -0500 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> Message-ID: <4EB34AE9.6070002@hardwarefreak.com> On 11/3/2011 1:24 PM, Felipe Scarel wrote: > Reasons to choose ZFS were snapshots, and mainly dedup and compression > capabilities. I know, it's ironic since I'm not able to use them now due to > severe performance issues with them (mostly dedup) turned on. > > I do like the emphasis on data integrity and fast on-the-fly > configurability of ZFS to an extent, but I wouldn't recommend it highly for > new users, especially for production. It works (in fact it's working right > now), but has its fair share of troubles. > > We've started implementations to move our mail system to a more modular > enviroment and we'll probably move away from ZFS. Was a nice experiment > nonetheless, I learned quite a bit from it. I find this all very interesting... "Please keep in mind the current 0.5.2 stable release does not yet support a mountable filesystem. This functionality is currently available only in the 0.6.0-rc6 release candidate." https://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz "Uploaded October 14, 2011" So in the past ~two weeks, you converted your 15K+ user production server to ZFS on Linux, as an experiment, and have now decided to change to another filesystem solution, a mere two weeks later? Or am I misinterpreting the date given that 0.6.0-rc6 was released? -- Stan From p at state-of-mind.de Fri Nov 4 00:56:11 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 3 Nov 2011 23:56:11 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: <4EB3011D.80108@rename-it.nl> Message-ID: <20111103225611.GA14437@state-of-mind.de> * Simon Brereton : > On 3 November 2011 17:01, Stephan Bosch wrote: > > On 11/3/2011 9:42 PM, Simon Brereton wrote: > >> > >> Hi > >> > >> Could someone explain to me the difference between LOGIN and PLAIN? > >> I've been googling for a while, but haven't found anything. > > > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > > documented here: > > > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > > > Some clients still support it, but I would not recommend using it when PLAIN > > or a better SASL mechanism is also available at both ends. The PLAIN > > mechanism is documented here: > > > > http://tools.ietf.org/html/rfc4616 > > > > The main technical difference between the two is that the PLAIN mechanism > > transfers both username and password in a single SASL interaction, where > > LOGIN needs two. The PLAIN mechanism also provides support for having an > > authorization id different from the authentication id, allowing for master > > user login for example. > > Thanks to both of you. Can I bet that Outlook doesn't support > anything but plain? Outlook > 2007 LOGIN, NTLM Outlook 2010 > LOGIN, NTLM2, DIGEST-MD5 > I'm not sure I've ever heard of a client supporting other than > Evolution supporting MD5 passwords.. Two come to mind: mutt, Thunderbird However DIGEST-MD5 has been marked deprecated this summer. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From javierdemiguel at us.es Fri Nov 4 11:52:22 2011 From: javierdemiguel at us.es (=?UTF-8?B?SmF2aWVyIGRlIE1pZ3VlbCBSb2Ryw61ndWV6?=) Date: Fri, 04 Nov 2011 10:52:22 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103110336.67190@gmx.com> References: <20111103110336.67190@gmx.com> Message-ID: <4EB3B5D6.10102@us.es> Same problem here, any hint about a fix or workaround? Regards Javier > We follow the guidelines about timekeeping RHEL in vmware vsphere located here > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 > > These problems happens in peak hours. Any dovecot config parameter I could set to mitigate this problem? > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:57 AM > To: Maria Arrea, Dovecot Mailing List > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > On 03/11/2011 10:49, Maria Arrea wrote:> All the ESXs hosts and all the VM use the same NTP server.> > Any other idea?> Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W > From dovecot.user at seibercom.net Fri Nov 4 01:24:43 2011 From: dovecot.user at seibercom.net (Jerry) Date: Thu, 3 Nov 2011 19:24:43 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: <4EB3011D.80108@rename-it.nl> Message-ID: <20111103192443.4e239384@scorpio> On Thu, 3 Nov 2011 17:28:28 -0400 Simon Brereton articulated: > On 3 November 2011 17:01, Stephan Bosch wrote: > > On 11/3/2011 9:42 PM, Simon Brereton wrote: > >> > >> Hi > >> > >> Could someone explain to me the difference between LOGIN and PLAIN? > >> I've been googling for a while, but haven't found anything. > > > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > > documented here: > > > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > > > Some clients still support it, but I would not recommend using it > > when PLAIN or a better SASL mechanism is also available at both > > ends. The PLAIN mechanism is documented here: > > > > http://tools.ietf.org/html/rfc4616 > > > > The main technical difference between the two is that the PLAIN > > mechanism transfers both username and password in a single SASL > > interaction, where LOGIN needs two. The PLAIN mechanism also > > provides support for having an authorization id different from the > > authentication id, allowing for master user login for example. > > Thanks to both of you. Can I bet that Outlook doesn't support > anything but plain? > > I'm not sure I've ever heard of a client supporting other than > Evolution supporting MD5 passwords.. You would lose the bet. In addition, Claws-Mail supports CRAM-MD5. -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From taeuber at bbaw.de Fri Nov 4 13:04:22 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 12:04:22 +0100 Subject: [Dovecot] doveadm altmove -r Message-ID: <20111104120422.85c93004.taeuber@bbaw.de> Hi there, there is the ubuntu version 2.0.13 ov dovecot installed on our server. But in contrast to http://wiki2.dovecot.org/Tools/Doveadm/Altmove the option -r isn't supported in this version. altmove: invalid option -- 'r' doveadm altmove [-u |-A] [-S ] In which version of dovecot is this option supported? Thanks Lars From taeuber at bbaw.de Fri Nov 4 13:57:31 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 12:57:31 +0100 Subject: [Dovecot] doveadm altmove -r In-Reply-To: <20111104120422.85c93004.taeuber@bbaw.de> References: <20111104120422.85c93004.taeuber@bbaw.de> Message-ID: <20111104125731.5f05d91f.taeuber@bbaw.de> Ok, I've found this option to be introduce in version 2.0.14. Maybe someone (with the permissions) should add this info to the wiki page. Best wishes Lars Am Fri, 4 Nov 2011 12:04:22 +0100 Lars T?uber schrieb: > Hi there, > > there is the ubuntu version 2.0.13 ov dovecot installed on our server. > But in contrast to http://wiki2.dovecot.org/Tools/Doveadm/Altmove > the option -r isn't supported in this version. > > altmove: invalid option -- 'r' > doveadm altmove [-u |-A] [-S ] > > In which version of dovecot is this option supported? > > Thanks > Lars From fbscarel at gmail.com Fri Nov 4 13:57:45 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Fri, 4 Nov 2011 09:57:45 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB34AE9.6070002@hardwarefreak.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> <4EB34AE9.6070002@hardwarefreak.com> Message-ID: I'm using the GIT version, that 0.5 version is quite a bit outdated. I was not all that worried about using ZFS on this experiment because we do have the old mail storage on ext3 synchronized and ready to switch back, and I could disable dedup and compression on-the-fly if needed (which eventually was). On Fri, Nov 4, 2011 at 00:16, Stan Hoeppner wrote: > On 11/3/2011 1:24 PM, Felipe Scarel wrote: > > Reasons to choose ZFS were snapshots, and mainly dedup and compression > > capabilities. I know, it's ironic since I'm not able to use them now due > to > > severe performance issues with them (mostly dedup) turned on. > > > > I do like the emphasis on data integrity and fast on-the-fly > > configurability of ZFS to an extent, but I wouldn't recommend it highly > for > > new users, especially for production. It works (in fact it's working > right > > now), but has its fair share of troubles. > > > > We've started implementations to move our mail system to a more modular > > enviroment and we'll probably move away from ZFS. Was a nice experiment > > nonetheless, I learned quite a bit from it. > > I find this all very interesting... > > "Please keep in mind the current 0.5.2 stable release does not yet > support a mountable filesystem. This functionality is currently > available only in the 0.6.0-rc6 release candidate." > > https://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz > > "Uploaded October 14, 2011" > > So in the past ~two weeks, you converted your 15K+ user production > server to ZFS on Linux, as an experiment, and have now decided to change > to another filesystem solution, a mere two weeks later? Or am I > misinterpreting the date given that 0.6.0-rc6 was released? > > -- > Stan > From duihi77 at gmail.com Fri Nov 4 14:17:56 2011 From: duihi77 at gmail.com (Duane Hill) Date: Fri, 4 Nov 2011 07:17:56 -0500 (CDT) Subject: [Dovecot] doveadm altmove -r In-Reply-To: <20111104125731.5f05d91f.taeuber@bbaw.de> References: <20111104120422.85c93004.taeuber@bbaw.de> <20111104125731.5f05d91f.taeuber@bbaw.de> Message-ID: On Fri, 4 Nov 2011, Lars T?uber wrote: > Ok, I've found this option to be introduce in version 2.0.14. > Maybe someone (with the permissions) should add this info to the wiki page. It's in the release notes for v2.0.15 (http://www.dovecot.org/doc/NEWS-2.0): v2.0.15 2011-09-16 Timo Sirainen + doveadm altmove: Added -r parameter to move mails back to primary storage. ... From taeuber at bbaw.de Fri Nov 4 14:41:08 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 13:41:08 +0100 Subject: [Dovecot] doveadm altmove -r In-Reply-To: References: <20111104120422.85c93004.taeuber@bbaw.de> <20111104125731.5f05d91f.taeuber@bbaw.de> Message-ID: <20111104134108.efa5c27e.taeuber@bbaw.de> Am Fri, 4 Nov 2011 07:17:56 -0500 (CDT) Duane Hill schrieb: > On Fri, 4 Nov 2011, Lars T?uber wrote: > > > Ok, I've found this option to be introduce in version 2.0.14. > > Maybe someone (with the permissions) should add this info to the wiki > > page. > > It's in the release notes for v2.0.15 (http://www.dovecot.org/doc/NEWS-2.0): > > v2.0.15 2011-09-16 Timo Sirainen > > + doveadm altmove: Added -r parameter to move mails back to > primary storage. ... Oh yes this was a typo. Lars From leo at strike.wu.ac.at Fri Nov 4 19:36:16 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Fri, 04 Nov 2011 18:36:16 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EB0529E.2090808@strike.wu.ac.at> References: <4EAED444.70608@strike.wu.ac.at> <4EAEDDDD.40206@gmx.de> <4EB0529E.2090808@strike.wu.ac.at> Message-ID: <4EB42290.8060806@strike.wu.ac.at> On 11/01/2011 09:12 PM, Alexander 'Leo' Bergolth wrote: > On 10/31/2011 06:41 PM, e-frog wrote: >> On 31.10.2011 18:00, wrote Alexander 'Leo' Bergolth: >>> Have environment based namespace settings been abandoned in 2.0? >>> >> I think you need to add them to USERDB_KEYS now... >> >> http://wiki2.dovecot.org/PostLoginScripting#Running_environment > > I have added NAMESPACE_1 NAMESPACE_1_SEP NAMESPACE_1_TYPE > NAMESPACE_1_LIST NAMESPACE_1_PREFIX to USERDB_KEYS. > > Unfortunately the namespace still isn't used. :-( Can anyone confirm that setting namespaces via environment isn't supported in 2.0 anymore? Is there another way to dynamically add namespaces? Thanks, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From apm at one.com Fri Nov 4 21:23:36 2011 From: apm at one.com (Peter Mogensen) Date: Fri, 04 Nov 2011 20:23:36 +0100 Subject: [Dovecot] Corrupted transaction log file Message-ID: <4EB43BB8.6090803@one.com> Hi, I'm experiencing a problem I need some pointers to debug. I'm running Dovecot 2.0.15 and have a client which keeps causing log-entries like: Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: indexid changed 1320419300 -> 1320419441 (sync_offset=0) Nov 4 15:10:42 mail dovecot: imap(test at aaaone.net): Error: Corrupted transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: Invalid transaction log size (4476 vs 4360): /mail/3340444/. TestMails/dovecot.index.log (sync_offset=4476) Nov 4 15:10:42 mail dovecot: imap(test at aaaone.net): Disconnected: IMAP session state is inconsistent, please relogin. bytes=549/100474 The client issues a lot of IMAP connections like this: A1_0 LOGIN "test at aaaone.net" "password" A1_1 ENABLE QRESYNC CONDSTORE * OK Waiting for authentication process to respond.. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. A1_0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in * ENABLED QRESYNC CONDSTORE A1_1 OK Enabled. A2_0 CREATE "INBOX.folder" A2_0 OK Create completed. A3_0 SUBSCRIBE "INBOX.folder" A3_0 OK Subscribe completed. A4_0 APPEND "INBOX.folder" ($CreatedByImapClient-test) {20487+} ... more similar appends. But it only seems to be reproducible, when there's another client involved. Starting up Thunderbird with a login to the account seems to be enough to cause this. (To my knowledge this causes a series of STATUS command and then IDLE). The installation is using Maildir and lock_method = fcntl on local disks. /Peter From tss at iki.fi Fri Nov 4 21:30:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:30:29 +0200 Subject: [Dovecot] Auth Worker failures In-Reply-To: References: <1318948640.21919.103.camel@hurina> Message-ID: <1320435030.21919.142.camel@hurina> On Thu, 2011-10-20 at 16:24 -0400, Simon Brereton wrote: > Oct 20 06:25:14 mail dovecot: auth-worker(default): > sql(simon at example.net,127.0.0.1): CRYPT() != 'RaNDomsTRinG' CRYPT() means that it attempted to log in with an empty password. > I'm still sure this is the webmail trying to log in though.. Yeah, could be. But why it tries it with an empty password, I've no idea. From tss at iki.fi Fri Nov 4 21:34:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:34:03 +0200 Subject: [Dovecot] Accessing a strange mailbox In-Reply-To: <8762jie6ut.fsf@algae.riseup.net> References: <8762jie6ut.fsf@algae.riseup.net> Message-ID: <1320435243.21919.144.camel@hurina> On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote: > I have a user who has a mailbox called: > > A----->B > > It seemed to work in courier fine, they managed to create it, and there > are mails in it. However, dovecot is not letting the user access it, the > IMAP server gives an error. I tried to rename it but I would also get an > error: > > # doveadm mailbox rename 'A----->B' AtoB > doveadm(root): Error: Can't rename mailbox A----->B to AtoB: Mailbox doesn't exist: A----->B Maybe it's not named that? What does it look like in filesystem? Worked fine with me: # doveadm mailbox list 'A*' A----->B # doveadm mailbox rename 'A----->B' AtoB # doveadm mailbox list 'A*' AtoB From tss at iki.fi Fri Nov 4 21:38:27 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:38:27 +0200 Subject: [Dovecot] Getting the SMTP envelope through IMAP? In-Reply-To: <4EA28848.6070303@msgid.danisch.de> References: <4EA19119.4020803@msgid.danisch.de> <20111021180241.GA6924@PC211.ikt.de> <4EA1EB82.10807@msgid.danisch.de> <20111022081510.GA26132@laptop-dg.leere.eu> <4EA28848.6070303@msgid.danisch.de> Message-ID: <1320435508.21919.147.camel@hurina> On Sat, 2011-10-22 at 11:09 +0200, Hadmut Danisch wrote: > And neither the Return-Path nor the Received-Lines reveal the precise > date of delivery as the From line contains. The delivery date in From_ line is available in IMAP INTERNALDATE. The address part isn't available without patching Dovecot (mail_get_special(mail, MAIL_FETCH_FROM_ENVELOPE, &value)) From tss at iki.fi Fri Nov 4 21:43:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:43:31 +0200 Subject: [Dovecot] Dovecot crashes totally In-Reply-To: <4EA317B5.3090209@uni-greifswald.de> References: <4EA317B5.3090209@uni-greifswald.de> Message-ID: <1320435812.21919.150.camel@hurina> On Sat, 2011-10-22 at 21:21 +0200, Gordon Grubert wrote: > Hello, > > our dovecot server crashes totally without any really useful > log messages. The error log can be found in the attachment. > The only way to get dovecot running again is a complete > system restart. How often does it break? If really a "complete system restart" is needed to fix it, it doesn't sound like a Dovecot problem. Check if it's enough to stop dovecot and then make sure there aren't any dovecot processes lying around afterwards. > Oct 11 09:55:23 mailserver2 dovecot: master: Error: service(imap): > Initial status notification not received in 30 seconds, killing the > process > Oct 11 09:56:23 mailserver2 dovecot: imap-login: Error: master(imap): > Auth request timed out (received 0/12 bytes) Kind of looks like auth process is hanging. You could see if stracing it shows anything useful. Also are any errors logged about LDAP? Is LDAP running on the same server? From tss at iki.fi Fri Nov 4 21:47:39 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:47:39 +0200 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <4EA2C213.7030204@mike2k.de> References: <4EA2C213.7030204@mike2k.de> Message-ID: <1320436059.21919.152.camel@hurina> On Sat, 2011-10-22 at 15:16 +0200, Michael Stilkerich wrote: > dovecot: imap(michael): Error: > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file > or directory This is some weird filesystem? fstat() isn't supposed to ever fail with ENOENT. > Another thing that irritates me is that dovecot seems to use the dotlock > locking method, although I explicitly set lock_method to > fcntl (also tried flock, same behavior). The lock_method setting only affects index files. There are several others that always use dotlocks, because they're rarely enough modified that the dotlock performance doesn't matter. That's also why I haven't bothered to set use_excl_lock for them. From tss at iki.fi Fri Nov 4 21:50:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:50:13 +0200 Subject: [Dovecot] iterate_query not working In-Reply-To: <4EA5BBC8.5090901@netexpert.pl> References: <4EA5BBC8.5090901@netexpert.pl> Message-ID: <1320436213.21919.153.camel@hurina> On Mon, 2011-10-24 at 21:26 +0200, Piotr NetExpert wrote: > Hi, > > I have got iterate_query defined in dovecot-sql.conf.ext. However when I > run a command like: > doveadm search -A BEFORE 10d > I receive an error message (Failed to iterate ...) > > There is an error message in a log file too: > Error: Trying to iterate users, but userdbs don't support it dovecot -n output? Maybe you're using only passdb sql, but userdb static? Maybe you have more than one userdb? From piotr-l at netexpert.pl Fri Nov 4 21:53:41 2011 From: piotr-l at netexpert.pl (Piotr NetExpert) Date: Fri, 04 Nov 2011 20:53:41 +0100 Subject: [Dovecot] iterate_query not working In-Reply-To: <1320436213.21919.153.camel@hurina> References: <4EA5BBC8.5090901@netexpert.pl> <1320436213.21919.153.camel@hurina> Message-ID: <4EB442C5.9000709@netexpert.pl> Thanks for a reply. In the meantime I have found the problem. Userdb was static as you say. I believe docs do not tell that iterate query is based on user query. -- Tre?? oryginalnej wiadomo?ci -- > On Mon, 2011-10-24 at 21:26 +0200, Piotr NetExpert wrote: >> Hi, >> >> I have got iterate_query defined in dovecot-sql.conf.ext. However when I >> run a command like: >> doveadm search -A BEFORE 10d >> I receive an error message (Failed to iterate ...) >> >> There is an error message in a log file too: >> Error: Trying to iterate users, but userdbs don't support it > > dovecot -n output? Maybe you're using only passdb sql, but userdb > static? Maybe you have more than one userdb? > > -- pozdrawiam Piotr Szafarczyk http://www.netexpert.pl From tss at iki.fi Fri Nov 4 21:55:00 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:55:00 +0200 Subject: [Dovecot] Expire plugin not executing for sieve plugin? In-Reply-To: <4EA5E5DF.1050508@rename-it.nl> References: <4EA03F1C.5010500@druber.com> <4EA5A5E1.3000202@druber.com> <4EA5E5DF.1050508@rename-it.nl> Message-ID: <1320436501.21919.155.camel@hurina> On Tue, 2011-10-25 at 00:25 +0200, Stephan Bosch wrote: > On 10/24/2011 7:52 PM, Dan Swartzendruber wrote: > > > > No ideas? I googled some more and eventually turned up this posting: > > > > http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-01/msg00716.html > > > > > > guy seems to be having the exact problem I am, unfortunately, no-one > > replied to his post either :( > > Hmm, wonder why no one answered that one. In theory, those plugins > should interact gracefully. Right. There shouldn't be anything special with expire plugin and Sieve. > Running dovecot 1.2.12 on ubuntu 10.10. This very likely works fine with v2.0 though, and I don't really have time to debug noncritical v1.x bugs anymore. From tss at iki.fi Fri Nov 4 22:01:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:01:23 +0200 Subject: [Dovecot] SSL renegotiation vulnerability In-Reply-To: References: <23D958A8-91DE-42EA-BDC5-B0E2050EEC9F@iki.fi> <90232072-D5CF-42E7-8B5A-3DFBF75C6B3E@iki.fi> <7CAD9FBB-73BF-4D68-939B-1055AFED4C79@iki.fi> <3E17898B-B716-413F-9582-619B472D7C2B@iki.fi> Message-ID: <1320436883.21919.158.camel@hurina> http://vincent.bernat.im/en/blog/2011-ssl-dos-mitigation.html -> "Things get worse" shows that it's easier to DoS the server with multiple connections than with renegotiations, so I don't know if there's much point in disabling renegotiations. Perhaps Dovecot could allow e.g. one renegotiation per minute, but is that really worth the trouble?.. Perhaps there even are some clients that do renegotiations and Dovecot would break them. From tss at iki.fi Fri Nov 4 22:02:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:02:46 +0200 Subject: [Dovecot] GSSAPI and deny=yes passdb In-Reply-To: <4E8ACB70.7060004@telros.ru> References: <4E8ACB70.7060004@telros.ru> Message-ID: <1320436966.21919.160.camel@hurina> On Tue, 2011-10-04 at 13:01 +0400, Sergey Urushkin wrote: > Hi. Is it possible to use GSSAPI authentication and deny passdb > together? Seems it doesn't work as I expect: GSSAPI doesn't check deny > passdb, so I'm not able to restrict access to GSSAPI-users. GSSAPI doesn't use any passdbs, so it doesn't work. It would be nice to change this some day (so e.g. proxying would also be possible with GSSAPI auth), but for now it's a rather low priority for me. From tss at iki.fi Fri Nov 4 22:04:54 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:04:54 +0200 Subject: [Dovecot] POP server migration and duplicate messages. In-Reply-To: References: Message-ID: <1320437094.21919.162.camel@hurina> On Thu, 2011-10-27 at 12:16 -0700, Ernie Dunbar wrote: > From a > random sampling, the UIDL of the messages checks out between both servers, Completely identical UIDL output with the same order of messages? If so, there's nothing you can do about it from Dovecot's side. > so I really don't understand what's going on here. If this is just a bug > in Evolution, then that means I can just go ahead with the full migration > and hope that people aren't going to be stuck with re-downloading their > entire mailbox which in some cases, means hundreds or thousands of > messages. Some clients redownload mails if you change the hostname. But I don't know why Evolution would redownload only some of the messages but not all. From tss at iki.fi Fri Nov 4 22:08:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:08:59 +0200 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> Message-ID: <1320437339.21919.165.camel@hurina> On Fri, 2011-10-28 at 00:55 +0200, Marco Carcano wrote: > I'm struggling against what seems a really serious trouble: sometimes > mail get lost > > I did all the possible checks, but they seems really lost - I know is > absurd, but it seems so > > it has been noticed by two users on my server: intially I tought that > was their mistake (mail deleted, mail into spam, ecc), but it wasn't Enable mail_log plugin to make sure of this. http://wiki2.dovecot.org/Plugins/MailLog > Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid= >: saved mail to INBOX If Dovecot logs this, then the message definitely was saved to INBOX. > maybe is there some kind of bug in the maildrop version I use? by the > way, I had this issue also before installing amavisd-new Maildrop? From tss at iki.fi Fri Nov 4 22:19:43 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:19:43 +0200 Subject: [Dovecot] doveadm 'proxy list' or 'who' not working ? In-Reply-To: <7A7D318C-0BA3-4FDB-AB7F-F36A540C8472@aquaray.com> References: <7A7D318C-0BA3-4FDB-AB7F-F36A540C8472@aquaray.com> Message-ID: <1320437984.21919.170.camel@hurina> On Fri, 2011-10-28 at 11:49 +0200, de Lafond Guillaume wrote: > Hi, > > I switched from dovecot 1.2.* to 2.0.* for a mail proxy. > I try to play with the new cool utility doveadm, but I don't manage to get results from it. > > # /usr/local/dovecot-2.0.15/bin/doveadm proxy list > username proto src ip dest ip port > # This should work. A quick test shows it working with me. Check if you have "ipc" process running? Check that you have /login/ipc-proxy fifo. Maybe try straceing the imap-login processes and see if any ipc connection arrives there when running the doveadm command. > As 'proxy list' does not return anything, I tried with 'who' > # /usr/local/dovecot-2.0.15/bin/doveadm who -a /var/run/dovecot-socket-auth/anvil > username # proto (pids) (ips) > # This isn't supposed to show anything. Although why do you give such a socket path parameter? From tss at iki.fi Fri Nov 4 22:22:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:22:31 +0200 Subject: [Dovecot] What owner/permissions for virtual homes? In-Reply-To: References: Message-ID: <1320438151.21919.172.camel@hurina> On Fri, 2011-10-28 at 16:04 -0400, sean darcy wrote: > On 10/28/2011 03:34 PM, sean darcy wrote: > > VirtualUsers.Home.txt: > > > > The directory layouts for home and mail directories could look like one of > > these (in the preferred order): > > > > 1. Mail directory under home, for example: home='/var/vmail/domain/user/' > > mail='/var/vmail/domain/user/mail/' > > > > Who should be the owner of these directories, with that permissions? Typically 0700 permissions, owned by the "mail user", whichever you've configured for that. http://wiki2.dovecot.org/UserIds > > Does it make a difference if it's /home/vmail rather than /var/vmail? Either one is fine. > it suggests the following configuration: > > mail_home = /var/vmail/%d/%n > mail_location = maildir:~/mail > > but where does mail_home appear? Also in 10-mail.conf? Doesn't matter. For example there. From tss at iki.fi Fri Nov 4 22:28:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:28:28 +0200 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111025110238.GB8900@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> Message-ID: <1320438508.21919.177.camel@hurina> On Tue, 2011-10-25 at 13:02 +0200, Edgar Fu? wrote: > We have two dovecot 1.2 instances sharing Maildirs on NFS. Indexes are local to the individual servers. > Occasionally (no idea how to trigger this), the Seen flag gets lost on some messages. I've verified that actually the ``S'' is missing from the filename. > I suspect something like server A caching the flags, server B setting Seen, and then server A flushing its cache for another change so overwriting what B changed. That shouldn't happen because of Dovecot's indexes. Dovecot doesn't force Maildir to match its indexes. Whenever doing flag changes, it stores them as "add \Seen flag to UID 1234". Then the 'S' flag is added to the current Maildir filename without losing any other changes. From ms at mike2k.de Fri Nov 4 22:29:16 2011 From: ms at mike2k.de (Michael Stilkerich) Date: Fri, 04 Nov 2011 21:29:16 +0100 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320436059.21919.152.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> Message-ID: <4EB44B1C.90505@mike2k.de> On 04.11.2011 20:47, Timo Sirainen wrote: >> dovecot: imap(michael): Error: >> > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file >> > or directory > This is some weird filesystem? fstat() isn't supposed to ever fail with > ENOENT. Nope, plain ext3. I have these issues since upgrading to Ubuntu 11.10, which was a dovecot upgrade from 1.2.15 to 2.0.13. Currently I use a self-built 2.0.15 dovecot. I'm still getting errors once in a while for the dovecot.index.log file: Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking failed for file /home/dovecot/isa/dovecot.index.log: No such file or directory Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with file /home/dovecot/isa/dovecot.index.log: No such file or directory These errors occur regardless of the lock_method used. And occasionally I also get errors like: Nov 3 19:03:34 keira dovecot: imap(mikey): Error: Corrupted transaction log file /home/dovecot/mikey/dovecot.index.log seq 15: offset points outside file (32708 + 8 > 32708) (sync_offset=32800) I believe the latter was recently reported by someone else on this list as well. The files are there, though I guess that dovecot recreates them when the errors occurs so I don't know what the situation was by the time the error occurred... -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4480 bytes Desc: S/MIME Cryptographic Signature URL: From tss at iki.fi Fri Nov 4 22:30:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:30:13 +0200 Subject: [Dovecot] post login script In-Reply-To: References: Message-ID: <1320438617.21919.179.camel@hurina> On Fri, 2011-10-28 at 12:49 +0200, Osvaldo Alvarez Pozo wrote: > Hello all, > > I am testing postlogin script with dovecot 2. > > I works but i would like that the imap and pop session get executed > even if the post-login script fail. How/why would the script fail? > #/usr/local/bin/postlogin.sh > echo "UPDATE mailbox SET last_login = now() WHERE username = '$USER'" > | mysql -upostlogin -pXXXXXXXX postfixadmin > exec "$@" Looks like it would always exec $@ even if mysql update fails. From tss at iki.fi Fri Nov 4 22:33:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:33:09 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: <1320438789.21919.181.camel@hurina> On Tue, 2011-11-01 at 09:38 +0100, Lukas Wei? wrote: > >> and sql storage is difficult > to backup. Ive done that, by setting up dovecot with maildir format and > copy mails from old dbmail to new dovecot imap server with my mail > client (thunderbird). > > Thunderbird probably changes the INTERNALDATE of > the messages. Copy the mails some other way, like offlineimap or maybe > some other IMAP client. > > What is INTERNALDATE? The modification > timestamp is correct ("ls -l"), and the "Date: ... " line, too. Maildir file's mtime is the INTERNALDATE, yes. Maybe the the problem isn't then any timestamps, but that the messages were copied in reverse order (newest message first), so their IMAP UIDs are reversed. From tss at iki.fi Fri Nov 4 22:38:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:38:08 +0200 Subject: [Dovecot] LDAP w/SASL "Active Directory" authentication failing. In-Reply-To: References: Message-ID: <1320439089.21919.184.camel@hurina> On Tue, 2011-11-01 at 09:55 -0600, David Varela wrote: > I am running a Dovecot server (version 1.2.17) on FreeBSD 8.2, using > LDAP to authenticate Active Directory users. I can successfully bind and > authenticate using PLAIN and LDAP without SASL, but obviously passwords for > the bind user and the user being authenticated are being passed in plain > text. I've attempted to configure my server to us SASL however when I > attempt to authenticate a user I see authentication failures. I reviewed > the security log on my domain controller and see that the bind user is > binding properly, so the issue appears to be orginating from the user > authentication, however I cannot determine what the issue is. Here is all > the information regarding my configuration, along with the logs from the > server: SASL binding currently works only for the initial "ldap admin user" authentication. It doesn't work for individual user authentication requests (auth_bind=yes). > #auth_bind = yes Here you're not even attempting to use auth binds. > pass_attrs = mail=user And you're also not returning a password for user. > Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): > No password returned (and no nopassword) So Dovecot has no way of authenticating user. I'd suggest forgetting about SASL and enabling TLS instead. From tss at iki.fi Fri Nov 4 22:41:26 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:41:26 +0200 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <20111102085715.GA4089@dibs.tanso.net> References: <20111102085715.GA4089@dibs.tanso.net> Message-ID: <1320439286.21919.187.camel@hurina> On Wed, 2011-11-02 at 09:57 +0100, Jan-Frode Myklebust wrote: > IMHO it would be great if we could both provision and un-provision users > without having the provisioning system mess with files in the > filesystem. Provisioning new users works fine since dovecot will create > all files/directories on first login, but I haven?t found the tools to > un-provison them. I see "doveadm mailbox delete" can delete individual > mailboxes, but what about indexes, sieve-scripts and home-directories? > Are there any tools to clean out these ? Do you have all of those files in separate directories? Normally I'd think everything is under home dir, so you can just: rm -rf `doveadm user -f home $username` For deleting indexes and other dirs that are not under home.. well.. I guess you'll need some more scripting. Anyway, I don't think there's any good and safe way to just go delete user's directories. Especially home dir. From tss at iki.fi Fri Nov 4 22:44:30 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:44:30 +0200 Subject: [Dovecot] Different default realm per listener In-Reply-To: <4EB15375.30106@team.de.inter.net> References: <4EB15375.30106@team.de.inter.net> Message-ID: <1320439470.21919.190.camel@hurina> On Wed, 2011-11-02 at 15:28 +0100, Kai Arif wrote: > Hi all! > > I want create a large IMAP server setup where dovecot is listening on multiple interfaces and needs > a different default realm on each interface. I'm using dovecot 2.0.5 on FreeBSD9 and authenticate > against a LDAP database. > I found an old mail thread from 2007 that said I'm basically screwed. Has that change since than? Is > there any way to use different default realms per listener? v2.0 can almost do it: local 1.2.3.4 { auth_default_realm = foo.org } local 1.2.3.5 { auth_default_realm = bar.org } Unfortunately that will currently fail, because per-IP auth settings don't work.. It will hopefully be fixed some day. From tss at iki.fi Fri Nov 4 22:47:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:47:57 +0200 Subject: [Dovecot] IMP, Dovecot and multiple namespaces In-Reply-To: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> References: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> Message-ID: <1320439677.21919.192.camel@hurina> On Wed, 2011-11-02 at 09:37 -0500, Rick Romero wrote: > Hey guys, > > I'm wondering what the best way is to be backwards compatible with > Courier mailbox formats and not duplicate mailbox trees with Dovecot. > Is anyone doing this right now? > > My Dovecot 2.0.13 is configured as follows for Namespaces: > > namespace { > inbox = yes > location = > prefix = > separator = . Perhaps use '/' separator. > } > namespace { > hidden = yes > inbox = no > list = no > location = > prefix = INBOX. > separator = . Add "alias_for = " here. I also added it to wiki2. > } > > So I should have an INBOX, plus additional root folders - rather than > the Courier default of all folders being subfolders of the INBOX. > > So when I run LIST in IMAP I get - > 08 LIST "" % > * LIST (\HasChildren) "." "INBOX" INBOX.% now matches both INBOX. namespace's mailboxes, and also the INBOX mailbox's children. If you change the separator to '/', there's a difference between INBOX.% (compatibility namespace) and INBOX/% (INBOX's children). From tss at iki.fi Fri Nov 4 22:59:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:59:04 +0200 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103095424.67160@gmx.com> References: <20111103095424.67160@gmx.com> Message-ID: <1320440344.21919.196.camel@hurina> On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the filesystem operations take more than a second. From tss at iki.fi Fri Nov 4 23:03:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:03:25 +0200 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: References: Message-ID: <1320440605.21919.198.camel@hurina> On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes wrote: > I'm having a problem with dovecot ldap connection when ldap server is in > another firewall zone, firewall kills the ldap connection after a > determined period of inactivity, this is good from the firewall point of > view but is bad for dovecot because it never knows the connections has been > dropped, this creates longs timeouts in dovecot and finally it reconnects, > meanwhile many users fails to authenticate, I have seen this kind of post > in the list for a while but can't find a solution for it, so my question is > how to define a idle ldap time in dovecot so it can reconnect before the > firewall has dropped the connection or just close the connection under > inactivity so when a user authenticate doesn't fails for a while until > dovecot detects that the connection has hanged. Is this a feature request > or there is already a configuration for this??? Can't the firewall be changed to reject the LDAP packets instead of dropping them? Then Dovecot would immediately notice that the connection has died, and with a recent enough version it wouldn't even log an error about it. I guess some kind of an "ldap_idle_disconnect = 30s" setting could be added, but it's not a very high priority for me. From tss at iki.fi Fri Nov 4 23:09:49 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:09:49 +0200 Subject: [Dovecot] Restricting IMAP access In-Reply-To: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> References: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Message-ID: <1320440989.21919.202.camel@hurina> On Thu, 2011-11-03 at 19:13 +0100, Thierry de Montaudry wrote: > Hi list, > > I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. > I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). > Anybody has a trick for that? Running dovecot 2.0.13. > > I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. So assuming all users can log in from 127.0.0.1 and domain.com only elsewhere, you can do stuff like: a) Change password_query so that it succeeds only if '%l' = '127.0.0.1' or '%d' = 'domain.com' b) Add a deny=yes passdb with e.g. sqlite backend that basically does the same as a) but reversed. http://wiki2.dovecot.org/Authentication/RestrictAccess has passwd-file example, example config files should have also something. c) Add a new Dovecot proxy facing the public internet where it lets in only those users with '%d'='domain.com' (again maybe sqlite) Possibly other ways too.. From janfrode at tanso.net Fri Nov 4 23:19:31 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 4 Nov 2011 22:19:31 +0100 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <1320439286.21919.187.camel@hurina> References: <20111102085715.GA4089@dibs.tanso.net> <1320439286.21919.187.camel@hurina> Message-ID: <20111104211931.GA5430@dibs.tanso.net> On Fri, Nov 04, 2011 at 10:41:26PM +0200, Timo Sirainen wrote: > On Wed, 2011-11-02 at 09:57 +0100, Jan-Frode Myklebust wrote: > > IMHO it would be great if we could both provision and un-provision users > > without having the provisioning system mess with files in the > > filesystem. Provisioning new users works fine since dovecot will create > > all files/directories on first login, but I haven?t found the tools to > > un-provison them. I see "doveadm mailbox delete" can delete individual > > mailboxes, but what about indexes, sieve-scripts and home-directories? > > Are there any tools to clean out these ? > > Do you have all of those files in separate directories? Normally I'd > think everything is under home dir, so you can just: Unfortunately, for historical reasons, yes. The users doesn't have a home-directory, or the inbox is directly in their home.. so I don't think sieve fits there, and indexes are kept elsewhere for performance reasons. > > rm -rf `doveadm user -f home $username` Ah, great. Looking up the home dir via doveadm helps. > For deleting indexes and other dirs that are not under home.. well.. I > guess you'll need some more scripting. What happens to the indexes when we delete the mailboxes via "doveadm mailbox delete" ? Will that clean up the indexes ? -jf From tss at iki.fi Fri Nov 4 23:18:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:18:56 +0200 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EAED444.70608@strike.wu.ac.at> References: <4EAED444.70608@strike.wu.ac.at> Message-ID: <1320441536.21919.206.camel@hurina> On Mon, 2011-10-31 at 18:00 +0100, Alexander 'Leo' Bergolth wrote: > NAMESPACE_2='maildir:/var/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep' > NAMESPACE_2_PREFIX=Shared/spamrep/ > NAMESPACE_2_SEP=/ > NAMESPACE_2_TYPE=public > NAMESPACE_2_LIST=yes The settings are now named. http://wiki2.dovecot.org/MailLocation#Custom_namespace_location shows the basics. So for example if you have: namespace foo { .. } You can change it via: NAMESPACE/FOO/PREFIX=Shared/spamrep/ NAMESPACE/FOO/SEP=/ ..etc.. The '/' character is a bit problematic with shells scripts though, see the wiki link how to do it via "env" at least. Another problem is if you want to add more namespaces than you have configured. IIRC it's possible to simply add: NAMESPACE=foo And more than one is probably either "foo bar" or "foo,bar". If you get something to work, please add example to http://wiki2.dovecot.org/PostLoginScripting :) From tss at iki.fi Fri Nov 4 23:26:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:26:29 +0200 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <4EB43BB8.6090803@one.com> References: <4EB43BB8.6090803@one.com> Message-ID: <1320441989.21919.212.camel@hurina> On Fri, 2011-11-04 at 20:23 +0100, Peter Mogensen wrote: > I'm running Dovecot 2.0.15 and have a client which keeps causing > log-entries like: > > > Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted > transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: > indexid changed 1320419300 -> 1320419441 (sync_offset=0) Session A had TestMails open and created with index file whose ID was 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 17:08:20 EET 2011). Session B came and recreated the index files 141 seconds later with ID 1320419441. Either it didn't see A's original index files for some reason or it simply decided to recreate them for some reason. Either way this shouldn't have happened. Session A then notices that the indexes were recreated, and logs an error. > But it only seems to be reproducible, when there's another client > involved. Starting up Thunderbird with a login to the account seems to > be enough to cause this. (To my knowledge this causes a series of STATUS > command and then IDLE). > > The installation is using Maildir and lock_method = fcntl on local disks. What filesystem? Have changed any service imap {} settings? Can you reproduce this by running stress test with imaptest? http://imapwiki.org/ImapTest For example: imaptest user=testuser pass=testpass host=localhost Or if it's really related to creating the initial files: while imaptest user=testuser pass=testpass host=localhost secs=1; do rm -rf /home/testuser/Maildir done From tss at iki.fi Fri Nov 4 23:27:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:27:46 +0200 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <20111104211931.GA5430@dibs.tanso.net> References: <20111102085715.GA4089@dibs.tanso.net> <1320439286.21919.187.camel@hurina> <20111104211931.GA5430@dibs.tanso.net> Message-ID: <1320442066.21919.213.camel@hurina> On Fri, 2011-11-04 at 22:19 +0100, Jan-Frode Myklebust wrote: > > For deleting indexes and other dirs that are not under home.. well.. I > > guess you'll need some more scripting. > > What happens to the indexes when we delete the mailboxes via "doveadm > mailbox delete" ? Will that clean up the indexes ? Yes. So you could create a script that lists all mailboxes, deletes them and then rmdirs the index root dir. From tss at iki.fi Fri Nov 4 23:39:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:39:18 +0200 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <4EB44B1C.90505@mike2k.de> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> Message-ID: <1320442758.21919.222.camel@hurina> On Fri, 2011-11-04 at 21:29 +0100, Michael Stilkerich wrote: > On 04.11.2011 20:47, Timo Sirainen wrote: > >> dovecot: imap(michael): Error: > >> > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file > >> > or directory > > This is some weird filesystem? fstat() isn't supposed to ever fail with > > ENOENT. > > Nope, plain ext3. I have these issues since upgrading to Ubuntu 11.10, > which was a dovecot upgrade from 1.2.15 to 2.0.13. Currently I use a > self-built 2.0.15 dovecot. I'm still getting errors once in a while for > the dovecot.index.log file: > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking > failed for file /home/dovecot/isa/dovecot.index.log: No such file or > directory > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with > file /home/dovecot/isa/dovecot.index.log: No such file or directory These simply shouldn't happen. I'd say it's a kernel bug. You're running a default Ubuntu kernel? I wonder if other Ubuntu users have this problem. > And occasionally I also get errors like: > > Nov 3 19:03:34 keira dovecot: imap(mikey): Error: Corrupted transaction > log file /home/dovecot/mikey/dovecot.index.log seq 15: offset points > outside file (32708 + 8 > 32708) (sync_offset=32800) > > I believe the latter was recently reported by someone else on this list > as well. The files are there, though I guess that dovecot recreates them > when the errors occurs so I don't know what the situation was by the > time the error occurred... Try if you can reproduce these more easily with imaptest? http://imapwiki.org/ImapTest Be sure to run it against a test account since it trashes the whole mailbox. From tss at iki.fi Fri Nov 4 23:47:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:47:10 +0200 Subject: [Dovecot] Rewrite the ostream output method, to save messages in another directory. In-Reply-To: References: Message-ID: <1320443230.21919.227.camel@hurina> On Mon, 2011-10-31 at 13:25 -0200, Alex Baule wrote: > struct ostream * > o_stream_create_X_list(struct ostream *output, struct istream *input, const > char *type) > { > struct emexis_ostream *X_stream; > struct ostream *new_output; > int fd; > > fd = open("/tmp/email",O_WRONLY); > > new_output = o_stream_create_fd_file(fd, 0, TRUE); > o_stream_cork(new_output); > > emexis_stream = i_new(struct emexis_ostream, 1); > emexis_stream->ostream.sendv = o_stream_X_sendv; > emexis_stream->ostream.flush = o_stream_X_flush; > emexis_stream->ostream.iostream.close = o_stream_X_close; > emexis_stream->output = new_output; > emexis_stream->input = input; > > o_stream_ref(new_output); > > return o_stream_create(&X_stream->ostream); > } This part of the code looks ok. > With this new o_stream, i got this error. > > Oct 27 15:14:24 brc dovecot: imap(alex at exemplo.com.br): Error: > o_stream_send_istream(/storage/gss/emexis/messages/ > exemplo.com.br/alex/Maildir/.Emexis.Ext-Blacklist/tmp/1319735664.M568409P30815.brc) > failed: Bad file descriptor I guess this should call your o_stream_X_sendv(). What do you do in that? I guess you're calling o_stream_sendv() for the new_output stream? Check if it returns -1, if it does then you're somehow breaking the new_output. Maybe you're closing it too early? Try what happens if you create it with: new_output = o_stream_create_fd_file(fd, 0, FALSE); Then it doesn't automatically close the fd, and it's at least clearer if the problem is because the stream is closed too early or if some completely wrong fd is attempted to be used. > There is something wrong Timo ? Like missing something or i need to rewrite > another funcion ? Nope. I think this is completely internal to your code. From apm at one.com Fri Nov 4 23:49:22 2011 From: apm at one.com (Peter Mogensen) Date: Fri, 04 Nov 2011 22:49:22 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EB45DE2.3030909@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: > What filesystem? XFS > Have changed any service imap {} settings? nope. But I have a special auth plugin. I doubt that's the reason though, since it's an entirely different process. Thanx, I'll try gather more info. /Peter From tss at iki.fi Fri Nov 4 23:49:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:49:28 +0200 Subject: [Dovecot] doveadm segfaults on TCP connect - version 2.0.15 In-Reply-To: <3316c960405142bf1d81f1db2b350739.squirrel@webmail.islandemail.com> References: <3316c960405142bf1d81f1db2b350739.squirrel@webmail.islandemail.com> Message-ID: <1320443369.21919.229.camel@hurina> On Wed, 2011-10-19 at 12:40 -0700, ghandidrivesahumvee at rocketfish.com wrote: > I just recently installed Dovecot 2.0.15. Unfortunately, doveadm segfaults > when I attempt to connect to the local dovecot instance. I fixed a lot of bugs related to this for v2.1. I'm not sure if I'll bother backporting them, since it can't work perfectly anyway without breaking other stuff.. Hopefully v2.1.0 won't be too far away now :) From tss at iki.fi Fri Nov 4 23:50:51 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:50:51 +0200 Subject: [Dovecot] Proxy and Virtual IMAP Folders In-Reply-To: References: Message-ID: <1320443452.21919.230.camel@hurina> On Wed, 2011-10-19 at 21:17 -0400, Pedro Paixao wrote: > Is is posible to use the virtual plugin, in a IMAP proxy configuration, in a > way that the virtual plugin searches messages on the real server? > > Here's my scenario: > > [Client]----[Dovecot Proxy]-----[Real Imap Server] > > Client connects to Dovecot Proxy and gets all the folders that are on the > Real Imap Server, plus some virtual ones (generated by Dovecot Proxy) that > search for messages on the real server. In my config the Dovecot Proxy has > no email messages, real folders, or local delivery. Yes, you should be able to do this with v2.1's imapc backend. From tss at iki.fi Fri Nov 4 23:52:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:52:13 +0200 Subject: [Dovecot] imap proxy for Gmail In-Reply-To: References: Message-ID: <1320443533.21919.231.camel@hurina> On Wed, 2011-10-19 at 23:31 -0400, Jumping Mouse wrote: > Hello, b > > I have a situation were I want to be able to give teachers access to their Google apps for education email through Squirrelmail. The network that they are on blocks gmail. I am able to get Squirrelmail to communicate with Gmail via imap. I am looking at getting persistant imap connections to improve the user experience. I have looked at IMAPPROXY but this application is not compatible with the gmail imaps (ssl-encrypted imap over port 993). I recall reading somewhere that Dovecot has a built in imapproxy that could be used for this. Dovecot doesn't provide persistent IMAP proxying. You should be able to combine imapproxy with stunnel to do this. From tss at iki.fi Fri Nov 4 23:56:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:56:04 +0200 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <4E9FEC22.8060105@webfusion.com> References: <4E9FEC22.8060105@webfusion.com> Message-ID: <1320443764.21919.233.camel@hurina> On Thu, 2011-10-20 at 12:38 +0300, Mark Zealey wrote: > I'm currently changing some systems to use the redirector service, which > means that to get local deliveries going I need to get lmtp set up so it > can be redirected as well. This is working fine, however we have a > number of different ports running on our servers depending on which > brand a customer is using. When the time comes to do the user look up, > we use (port, user, domain) to do a unique database lookup which works > fine for everything (proxied pop, imap, lmtp & straight pop, imap) but > not straight lmtp - perhaps because it doesn't have a 2-stage login > process unlike the pop/imap protocols. Looking at the SQL query we are > issuing, the %a port variable is set to 0 even though the connection is > coming in from TCP. Is there a way to change this? We are running > dovecot 2.0.12 however looking through the changelogs I can't see this > would be fixed in the newest version. A quick check in code shows that this is already supposed to work. Set auth_debug=yes and see if LMTP's auth lookup sends lip, rip, lport and rport fields to auth process? From tss at iki.fi Fri Nov 4 23:59:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:59:03 +0200 Subject: [Dovecot] lmtp performance In-Reply-To: <201110171642.45954.joseba.torre@ehu.es> References: <4E9BE1FB.6040902@upm.es> <4E9C3C25.7070802@upm.es> <201110171642.45954.joseba.torre@ehu.es> Message-ID: <1320443944.21919.235.camel@hurina> On Mon, 2011-10-17 at 16:42 +0200, Joseba Torre wrote: > > Add to service lmtp: > > process_min_avail = 20 > I had a similar problem, and found the same solution. In my setup, only 1 lmtp > process was available and a 2nd one never was created, so the process was > really busy. With this parameter (I setup it to 10) I have diferent process > and all of them are used. Hmm. I wonder if perhaps it would be better to set: service lmtp { client_limit = 1 } I think I'll make that the default.. From tss at iki.fi Sat Nov 5 00:07:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 00:07:59 +0200 Subject: [Dovecot] [PATCH] Use SSL_MODE_RELEASE_BUFFERS if available to keep memory usage low In-Reply-To: References: Message-ID: <1320444480.21919.236.camel@hurina> On Thu, 2011-10-13 at 16:23 -0300, Cristian Rodr?guez wrote: > Use SSL_MODE_RELEASE_BUFFERS if available to keep memory usage low. Hmm. Committed. Although I wonder how big of a difference it makes. And if it increases CPU usage much? From maria_arrea at gmx.com Sat Nov 5 01:57:11 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Sat, 05 Nov 2011 00:57:11 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111104235711.67190@gmx.com> Timo, we are not using NFS, we use remote iSCSI volumes with ext4. Regards Maria ----- Original Message ----- From: Timo Sirainen Sent: 11/04/11 09:59 PM To: Maria Arrea Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the filesystem operations take more than a second. From noah-list at enabled.com Sat Nov 5 02:32:09 2011 From: noah-list at enabled.com (Noah) Date: Fri, 04 Nov 2011 17:32:09 -0700 Subject: [Dovecot] dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 Message-ID: <4EB48409.90803@enabled.com> Hi there I keep getting the following errors scrolling in my dovecot logs. I am wondering how I can nail down what is creating that problem. dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 also I have a trash folder with over 100k messages in mbox format. I am wondering if migrating to Maildir format would make it easier and quicker for dovecot to index my mail. I am getting these types of errors from time to time. dovecot: 2011-10-30 20:32:58 Error: IMAP(): Timeout while waiting for lock for transaction log file /home//mail/.imap/INBOX/dovecot.index.log Cheers, Noah From stan at hardwarefreak.com Sat Nov 5 10:26:45 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 05 Nov 2011 03:26:45 -0500 Subject: [Dovecot] dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 In-Reply-To: <4EB48409.90803@enabled.com> References: <4EB48409.90803@enabled.com> Message-ID: <4EB4F345.80000@hardwarefreak.com> On 11/4/2011 7:32 PM, Noah wrote: > Hi there > > I keep getting the following errors scrolling in my dovecot logs. I am > wondering how I can nail down what is creating that problem. > > dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash > UID 42022 got too little data: 90900 vs 1529791 What version of Dovecot are you using? > also I have a trash folder with over 100k messages in mbox format. I am > wondering if migrating to Maildir format would make it easier and > quicker for dovecot to index my mail. As I answered on the Postfix list, the solution to this is ditching procmail for dovecot-lda so indexing of new messages is incremental upon delivery, instead of megalithic when your MUA connects to the mailbox. Switching to maildir in this case will simply trade on problem for another. Your memory consumption may go down, but your disk access will skyrocket. > I am getting these types of errors from time to time. > > dovecot: 2011-10-30 20:32:58 Error: IMAP(): Timeout while waiting > for lock for transaction log file > /home//mail/.imap/INBOX/dovecot.index.log It would be really helpful if you would simply post a big dump of your machine specs, software configuration, user base, mailbox sizes, total mail storage space, concurrent user load, etc. You've now posted a total of 3 seemingly different problems on two mailing lists, related to your Dovecot problems. I get the feeling that there's a much larger overall problem here, say something like hardware totally under powered for the workload you're throwing at it, or something to that affect. Or maybe just a really bad system setup. The more information you can provide, the more quickly and thoroughly we can provide a solution or salient advice. -- Stan From laxlaw at gmx.de Sat Nov 5 11:57:19 2011 From: laxlaw at gmx.de (=?UTF-8?Q?Lukas_Wei=C3=9F?=) Date: Sat, 05 Nov 2011 10:57:19 +0100 Subject: [Dovecot] =?utf-8?q?Bad_creation_timestamp_of_migrated_mails_=28m?= =?utf-8?q?aildir_format=29=3F?= In-Reply-To: <1320438789.21919.181.camel@hurina> References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> <1320438789.21919.181.camel@hurina> Message-ID: On Fri, 04 Nov 2011 22:33:09 +0200, Timo Sirainen wrote: > On Tue, 2011-11-01 at 09:38 +0100, Lukas Wei? wrote: >>>> and sql storage is difficult to backup. Ive done that, by setting up dovecot with maildir format and copy mails from old dbmail to new dovecot imap server with my mail client (thunderbird). >> >>> Thunderbird probably changes the INTERNALDATE of >> the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. What is INTERNALDATE? The modification timestamp is correct ("ls -l"), and the "Date: ... " line, too. > Maildir file's mtime is the INTERNALDATE, yes. Maybe the the problem isn't then any timestamps, but that the messages were copied in reverse order (newest message first), so their IMAP UIDs are reversed. Possible. How do i avoid this? What is the preffered way to migrate mails from one to another imap server? Offlineimap? Can i do a save sync with that, even if my dovecot has newer mails now? From lists at kokelnet.de Sat Nov 5 16:13:30 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 15:13:30 +0100 Subject: [Dovecot] Dovecot SASL on another machine Message-ID: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> Hello list, is there a way to use Dovecot SASL Authentication for submission on a different machine dovecot runs? e.g.: Dovecot machine <-> postfix machine(for submission) How can I authenticate my senders on a different physical machine against Dovecot SASL? Thanks, Tobias From tss at iki.fi Sat Nov 5 17:43:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 17:43:10 +0200 Subject: [Dovecot] fts-lucene vs SEARCH header Re: 2.1: problems compiling fts-lucene - CLucene git version necessary? In-Reply-To: References: <7C9B9C67-A1FD-472B-8514-BC54F1B01D00@iki.fi> <1316446540.12936.114.camel@hurina> <20110920071245.GA3617217@gabi.sernet.de> Message-ID: <1320507790.21919.237.camel@hurina> On Thu, 2011-09-22 at 13:24 +0200, Lutz Pre?ler wrote: > Next problem :-) side effect: my test setup (some commits ago) with fts = lucene > in plugin section (ftps_lucene plugin without stemmer support): > SEARCH header is not working (empty result sets, no logging). Fixed finally. From tss at iki.fi Sat Nov 5 18:01:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:01:47 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> Message-ID: <1320508907.21919.239.camel@hurina> On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > is there a way to use Dovecot SASL Authentication for submission on a > different machine dovecot runs? With v2.0 you should be able to add inet_listener for auth service, which acts as an auth-client socket. > Dovecot machine <-> postfix machine(for submission) Looking at Postfix v2.8.0 code, looks like Postfix also supports it. From tss at iki.fi Sat Nov 5 18:06:11 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:06:11 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> <1320438789.21919.181.camel@hurina> Message-ID: <1320509172.21919.241.camel@hurina> On Sat, 2011-11-05 at 10:57 +0100, Lukas Wei? wrote: > > Maildir file's mtime is the INTERNALDATE, yes. Maybe the the > problem isn't then any timestamps, but that the messages were copied in > reverse order (newest message first), so their IMAP UIDs are > reversed. > > Possible. How do i avoid this? What is the preffered way to > migrate mails from one to another imap server? Offlineimap? Can i do a > save sync with that, even if my dovecot has newer mails now? If you've already moved the mails, there's not much you can do.. Or if you don't care about losing IMAP UIDs (= clients redownloading mails), you could simply delete dovecot-uidlist and dovecot.index* files and let Dovecot reassign UIDs in the "correct" order. From tss at iki.fi Sat Nov 5 18:15:05 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:15:05 +0200 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111104235711.67190@gmx.com> References: <20111104235711.67190@gmx.com> Message-ID: <1320509705.21919.248.camel@hurina> Well, doesn't matter if it's NFS or not. It still looks as if Dovecot process was stuck for 45 seconds, most likely waiting for disk I/O to finish.. What happens is something like: 1. Get the current time ("now") 2. See if lock file exists 3. Create lock file 4. fstat() the created lock file 5. Log a warning if fstat's ctime differs from "now" more than 30 seconds. (Actually I think the 30 seconds threshold is way too generous, it should be less than 1 second usually.) So steps 2 and 3 took 45 seconds to finish. Basically I guess the disk I/O load was very high at that time, or alternatively there was some unintentional delay caused by iSCSI (kernel/network bug/problem). On Sat, 2011-11-05 at 00:57 +0100, Maria Arrea wrote: > Timo, we are not using NFS, we use remote iSCSI volumes with ext4. > > Regards > > Maria > > ----- Original Message ----- > From: Timo Sirainen > Sent: 11/04/11 09:59 PM > To: Maria Arrea > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the fil > esystem operations take more than a second. From tss at iki.fi Sat Nov 5 18:23:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:23:14 +0200 Subject: [Dovecot] Attachements with SIS : shauld all the same files share the same inode ? In-Reply-To: <4E9BD704.9080801@univ-nantes.fr> References: <4E9BD704.9080801@univ-nantes.fr> Message-ID: <1320510194.21919.251.camel@hurina> On Mon, 2011-10-17 at 09:19 +0200, Yann Dupont wrote: > 104 instance of -rw------- 104 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-ffd0d1xxxx > > The files are hardlinked, so it seems ok, but I also see 9 others files > in the same directory : > > > -rw------- 1 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-0519df2a94fc9a4e1f4100008a6b99b6 > -rw------- 1 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-1790b92994fc9a4e1a4100008a6b99b6 > > as you can see, same prefix, If I compare them with diff, they are all > the same files. I guess you have "mail_attachment_fs = sis posix" (default)? Sounds like there could be a bug where SIS does comparison of files but for some reason thinks they are different. You could try if you can merge them with: mkdir /tmp/queue touch /tmp/queue/6e2df299fa96daec9b4735d07c494046429be4d6-foo doveadm sis deduplicate /var/attachments /tmp/queue From lists at kokelnet.de Sat Nov 5 18:31:56 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 17:31:56 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320508907.21919.239.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> Message-ID: <91de83aeeb686e189fb47e454ce86a06@hachmer.de> Am 05.11.2011 17:01, schrieb Timo Sirainen: > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: >> is there a way to use Dovecot SASL Authentication for submission on >> a >> different machine dovecot runs? > > With v2.0 you should be able to add inet_listener for auth service, > which acts as an auth-client socket. Thanks for your reply. I think I didn't couch correct. I want a postfix machine to authenticate against Dovecot SASL for submission service which hasn't installed dovecot locally. The postfix machine has to ask my central dovecot which is at the moment dovecot v1.2.15 but will be soon v2. Is this possible, if yes, how? Greetz, Tobias From tss at iki.fi Sat Nov 5 18:35:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:35:01 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <91de83aeeb686e189fb47e454ce86a06@hachmer.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <91de83aeeb686e189fb47e454ce86a06@hachmer.de> Message-ID: <1320510902.21919.253.camel@hurina> On Sat, 2011-11-05 at 17:31 +0100, Tobias Hachmer wrote: > Am 05.11.2011 17:01, schrieb Timo Sirainen: > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > >> is there a way to use Dovecot SASL Authentication for submission on > >> a > >> different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > Thanks for your reply. I think I didn't couch correct. > I want a postfix machine to authenticate against Dovecot SASL for > submission service which hasn't installed dovecot locally. > The postfix machine has to ask my central dovecot which is at the > moment dovecot v1.2.15 but will be soon v2. Yes, this is how I understood it. > Is this possible, if yes, how? With v2.0: service auth { inet_listener { port = 12345 } } Postfix: smtpd_sasl_path = inet:dovecot.foobar.com:12345 From lists at kokelnet.de Sat Nov 5 18:39:50 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 17:39:50 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320510902.21919.253.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <91de83aeeb686e189fb47e454ce86a06@hachmer.de> <1320510902.21919.253.camel@hurina> Message-ID: <80202ce555c1e3ca63e1c1ee3e8cd870@hachmer.de> Am 05.11.2011 17:35, schrieb Timo Sirainen: > On Sat, 2011-11-05 at 17:31 +0100, Tobias Hachmer wrote: >> Am 05.11.2011 17:01, schrieb Timo Sirainen: >> > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: >> >> is there a way to use Dovecot SASL Authentication for submission >> on >> >> a >> >> different machine dovecot runs? >> > >> > With v2.0 you should be able to add inet_listener for auth >> service, >> > which acts as an auth-client socket. >> >> Thanks for your reply. I think I didn't couch correct. >> I want a postfix machine to authenticate against Dovecot SASL for >> submission service which hasn't installed dovecot locally. >> The postfix machine has to ask my central dovecot which is at the >> moment dovecot v1.2.15 but will be soon v2. > > Yes, this is how I understood it. OK, sorry. >> Is this possible, if yes, how? > > With v2.0: > > service auth { > inet_listener { > port = 12345 > } > } > > Postfix: > > smtpd_sasl_path = inet:dovecot.foobar.com:12345 Thanks a lot for your quick help!!! Greetz, Tobias From tss at iki.fi Sat Nov 5 18:44:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:44:06 +0200 Subject: [Dovecot] Group ACL In-Reply-To: References: Message-ID: <1320511448.21919.257.camel@hurina> On Thu, 2011-10-06 at 08:38 -0700, Daniel L. Miller wrote: > Using Dovecot 2.0, LDAP userdb & passdb, with prefetch: > > hosts = myhost.mydomain > dn = cn=x,cn=y > dnpass = xyz123 > auth_bind = yes > auth_bind_userdn = uid=%n,ou=users,dc=x > ldap_version = 3 > base = ou=Users, dc=x > user_attrs = =home=/var/mail/%d/%n, \ > =mail=mdbox:/var/mail/%d/%n/mdbox, \ > =uid=vmail, \ > =gid=mail Would be good to put acl_groups here also so non-prefetch stuff has access to it also. > user_filter = (&(objectClass=inetOrgPerson)(mail=%u)) > > pass_attrs = mail=user, \ > userPassword=password, \ > =userdb_home=/var/mail/%d/%n, \ > =userdb_mail=mdbox:/var/mail/%d/%n/mdbox, \ > =userdb_uid=vmail, \ > =userdb_gid=mail, \ > =userdb_acl_groups='allshared' I think the problem may simply be those '' characters in there. You're now in "'allshared'" group rather than in "allshared" group.. If that doesn't help: > I have added permissions for "$allshared" to a mailbox's Inbox. It does > not appear in the shared folders list. Other mailboxes, with explicit > permission for a given username, work fine. What am I doing wrong? 1. Make sure that acl_groups setting is enabled for that user by setting mail_debug=yes and looking at the log: imap: Debug: Added userdb setting: plugin/acl_groups=allshared 2. Check with: doveadm acl debug -u user From tss at iki.fi Sat Nov 5 19:20:19 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:20:19 +0200 Subject: [Dovecot] Restarting dovecot periodically fails with "Socket already exists: /var/run/dovecot/anvil" In-Reply-To: <4E97000D.7030908@ubuntu.com> References: <4E97000D.7030908@ubuntu.com> Message-ID: <1320513622.21919.260.camel@hurina> On Thu, 2011-10-13 at 16:13 +0100, James Page wrote: > restarting dovecot regularly fails with version 2.0.13 on Ubuntu 11.10 > due to a running child process exiting after the main dovecot process. > > Oct 13 15:04:58 ubuntu dovecot: master: Error: service(anvil): Socket > already exists: /var/run/dovecot/anvil > This is an Ubuntu specific issue as we use upstart to manage dovecot > which monitors the status of the main dovecot process; the Debian init > script appears to make provision for this happening. http://hg.dovecot.org/dovecot-2.1/rev/6894298ae5fd should apply to v2.0 as well, but since it's a bit large change I don't know if I should backport it, at least without some more testing. From mpapet at yahoo.com Sat Nov 5 19:33:20 2011 From: mpapet at yahoo.com (Michael Papet) Date: Sat, 5 Nov 2011 10:33:20 -0700 (PDT) Subject: [Dovecot] Newbie Question/Feature Request Message-ID: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> Hi, I'm working on a plugin for qpsmtpd (a perl smtp frontend) that delivers mail via dovecot's LDA. I can only get mail delivered by writing the email to file, then calling 'deliver -d someone -f someone-else < /path/to/temp/email' All good, expected behavior. A perl snippet: local $/=undef; open FH, "< ./demo.mail"; my $stringified = ; # this is roughly equivalent to how other LDA queues work. close FH; open (CMD_OUT, " /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom $stringified |"); my $test = ; print $test; The above code errors out because the expected behavior is $stringified is supposed to be a file name. It would be great if deliver could be fed the email as a string so I don't have to add a step of writing to file. Or, maybe there's an undocumented method? Thanks. mpapet From tss at iki.fi Sat Nov 5 19:39:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:39:50 +0200 Subject: [Dovecot] dsync with quotas In-Reply-To: <8762kozbzo.fsf@algae.riseup.net> References: <8762kozbzo.fsf@algae.riseup.net> Message-ID: <1320514790.21919.262.camel@hurina> On Mon, 2011-09-19 at 13:19 -0400, Micah Anderson wrote: > I have been working on converting people from courier maildir -> dovecot > mdbox and during some of the dsync runs I'm seeing the > quota_exceeded_message be printed as an Error: > > dsync(): Error: Can't save message to mailbox INBOX: You are over quota. To avoid losing mail, immediately empty your Trash and Sent folders and \ > delete emails with large attachments. > dsync(): Info: INBOX: Couldn't keep all uids^M > > Its possible the user was over quota on the originating courier side, > but I would still like to migrate their mail proprely to mdbox, but it > seems like being over quota is inhibiting that. Are the quota > calculations including both the maildir files as well as the converted > mdbox files, resulting in a double counting? Depends on how you did the conversion and what quota backend you were using. If they were using different quota files, there shouldn't be double counting. But if for example you did a migration within the same server and quota was set to dict quota, it was possibly being counted double (both thought they owned the quota file). > I'm not entirely sure if the messages above indicate that the migration > failed for that user or not, so I've been manually increasing their > quota, then redoing the dsync mirror until it works properly. > > I wonder if it would be better if I turned off quota entirely during > migration so I don't run into this problem? Added: http://hg.dovecot.org/dovecot-2.0/rev/00157fd523ea http://hg.dovecot.org/dovecot-2.0/rev/0cf3215cdf2d From tss at iki.fi Sat Nov 5 19:43:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:43:09 +0200 Subject: [Dovecot] Newbie Question/Feature Request In-Reply-To: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> References: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> Message-ID: <1320514990.21919.264.camel@hurina> On Sat, 2011-11-05 at 10:33 -0700, Michael Papet wrote: > open (CMD_OUT, " /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom $stringified |"); > my $test = ; > print $test; > > The above code errors out because the expected behavior is $stringified is supposed to be a file name. It would be great if deliver could be fed the email as a string so I don't have to add a step of writing to file. Or, maybe there's an undocumented method? I don't remember exactly how Perl works, and this is untested, but I think it should be something like: open (CMD_OUT, "| /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom"); print CMD_OUT $stringified; my $test = ; print $test; The idea is that you write the string to the pipe to deliver that you open, and deliver reads it from stdin. From tss at iki.fi Sat Nov 5 20:00:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 20:00:50 +0200 Subject: [Dovecot] Safely restoring backups In-Reply-To: <87sjn4rg7y.fsf@algae.riseup.net> References: <87sjn4rg7y.fsf@algae.riseup.net> Message-ID: <1320516051.21919.274.camel@hurina> On Fri, 2011-10-07 at 11:09 -0400, Micah Anderson wrote: > I'm trying to determine what the best way to restore mail with mdbox > is. Restoring using maildir was trivial, I just used rsync > --ignore-existing which wrote any mails that were removed and didn't > touch things that already existed[1]. If a mail had changed flag, the maildir file got duplicated, which Dovecot complained about if it noticed it. > With mdbox things have become more > complicated, and I haven't found a way to restore mail that doesn't > result in many message duplicates. Do you need to restore mails so often that this is really a problem? :) > 1. I tried rsyncing the different backup directories back to the mail > storage host, and then doing: > > 'dsync -R backup -u $user mdbox:/path/to/to/daily.1/mdbox'[2] > 'dsync -R backup -u $user mdbox:/path/to/to/daily.2/mdbox' > > This works ok, but it gives duplicates of every mail that already exists > for every daily/weekly I dsync. This is what the rsync --ignore-existing > avoided. This is particularly annoying if I restore the weekly, and > multiple daily directories, because then you get a copy for every one > you restore. I had thought that the individual message's GIDs would keep > them from being duplicated? GUIDs can be used to identify messages, but there's no automatic deduplication. It's fine to e.g. copy a message from INBOX to INBOX, which duplicates it. Dovecot shouldn't prevent that. > Is there a way I can restore things from backup and avoid duplicates? > Is there another method I should try? Here's one way, although somewhat slow (and not fully tested): doveadm -f tab fetch -u user at domain 'mailbox guid' all | sort > guids1 doveadm -f tab fetch -o mdbox:/backups/user -u user at domain 'mailbox-guid guid' all | sort > guids2 diff -u guids1 guids2 | grep '^+[+]' | sed 's/^+//' | awk { system("doveadm import -u user at domain mdbox:/backups/user restored mailbox-guid "$1" guid "$2); } > 3. why doesn't doveadm import have a -s option to subscribe? I suppose it could.. Added to v2.1: http://hg.dovecot.org/dovecot-2.1/rev/afec4ceda8e1 From apm at one.com Sat Nov 5 21:17:04 2011 From: apm at one.com (Peter Mogensen) Date: Sat, 05 Nov 2011 20:17:04 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EB58BB0.2090900@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: >> Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted >> transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: >> indexid changed 1320419300 -> 1320419441 (sync_offset=0) > > Session A had TestMails open and created with index file whose ID was > 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 > 17:08:20 EET 2011). > > Session B came and recreated the index files 141 seconds later with ID > 1320419441. Either it didn't see A's original index files for some > reason or it simply decided to recreate them for some reason. Either way > this shouldn't have happened. > > Session A then notices that the indexes were recreated, and logs an > error. Oh... wait a minute... The timestamp is UTC, so 17:08:20 is about 2:22 before the log line. 2:22 is 142 seconds. So... given that the errors doesn't appear every time the client runs the series of APPEND requests, but (now I come to think of it) probably never the first time he runs it, but the second time - and that he did run the script a few minutes before this log line with out errors, - then... the problem might be that the first run of the script doesn't finish correctly. If session A is the first run of the script, then it should have finished and logged out long before session B. But maybe the problem is the first run not finishing properly. /Peter From p at state-of-mind.de Sat Nov 5 21:43:50 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sat, 5 Nov 2011 20:43:50 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320508907.21919.239.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> Message-ID: <20111105194350.GA3782@state-of-mind.de> * Timo Sirainen : > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > is there a way to use Dovecot SASL Authentication for submission on a > > different machine dovecot runs? > > With v2.0 you should be able to add inet_listener for auth service, > which acts as an auth-client socket. > > > Dovecot machine <-> postfix machine(for submission) > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. Document it and Wietse will support it on the mailing list. If its not documented support will be declined. Should I? p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From lists at wildgooses.com Sat Nov 5 22:09:38 2011 From: lists at wildgooses.com (Ed W) Date: Sat, 05 Nov 2011 20:09:38 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2C704.1010107@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> Message-ID: <4EB59802.9040205@wildgooses.com> On 03/11/2011 16:53, Patrick Westenberg wrote: > Ed W schrieb: > >>> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >>> thinking about a SSD based LUN for the indexes. As I'm using multiple >>> servers this LUN will use OCFS2. >> >> Given that the SAN always has the network latency behind it, might you >> be better to look at putting the SSDs in the frontend machines? >> Obviously this then needs some way to make users "sticky" to one machine >> (or some few machines) where the indexes are stored? > > Storing the indexes on several machines? > In this case I have to synchronize them. > See the "sticky" in my reply. You use one of several techniques to ensure that users always end up on the server with the indexes on. That way much of the IO is served from that local machine and you only access the SAN for the (in theory much less frequent) access to the mail files themselves. Clearly if the machine with the indexes on dies then the load balancer needs to pick a new machine and there will be delay/io/etc while the indexes are regenerated. Various techniques could mitigate this... I don't have such a larger system - please ignore all my advice... The basis for the suggestion is that I understand file access (locking in particular) is "expensive" on OCFS2/GFS. Therefore I read here on this list that others have found performance issues accessing maildir over OCFS2? It's also not hard to find benchmarks that show OCFS2/GFS are "fast", but slower than accessing the same storage without using a cluster filesystem - this makes sense. Hence it seems like a trade between convenience of storing everything on a central store and "some" performance improvement from a more complex system... I think if you search on benchmarks of DRBD vs OCFS2 and read here on the list about the "director" and "proxy" services you can see the point? I'm just trying to help you see the effects you might want to measure! (I don't have a system large enough to know much about this stuff from experience...) Good luck! Ed W From dovecot at knutejohnson.com Sun Nov 6 00:15:02 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sat, 05 Nov 2011 15:15:02 -0700 Subject: [Dovecot] Converting from 1.12 to 2.0 questions about mail location? Message-ID: <4EB5B566.7070808@knutejohnson.com> I'm building a new server and using a newer version of ubuntu server that uses Dovecot 2.0. I thought I would like to use Maildir instead of the mbox that was used in the previous server. What do I have to do to convert my mbox mail to Maildir? Or should I just use mbox and copy it over :-)? Thanks, -- Knute Johnson From stan at hardwarefreak.com Sun Nov 6 00:46:57 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 05 Nov 2011 17:46:57 -0500 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2B15D.1010005@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> Message-ID: <4EB5BCE1.3020608@hardwarefreak.com> On 11/3/2011 10:21 AM, Ed W wrote: > >> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >> thinking about a SSD based LUN for the indexes. As I'm using multiple >> servers this LUN will use OCFS2. > > Given that the SAN always has the network latency behind it, might you > be better to look at putting the SSDs in the frontend machines? The latency of the GbE IP network, iSCSI HBAs, GbE switches, etc, is but a fraction of the overhead of the out of band OCFS metadata exchange between cluster members, and the general overhead of OCFS, or of cluster filesystems in general. > Obviously this then needs some way to make users "sticky" to one machine > (or some few machines) where the indexes are stored? > > This seems theoretically likely to give you higher IOPs to the index > than having them on the OCFS2 storage? (At a trade off with more > complexity for the load balancer front end...) Following this logic, simply using local mechanical disk would yield improvement without the cost of SSDs. Depending on the number of nodes, putting a couple of SSDs in the SAN controller may likely be cheaper overall than adding mech disks to each node, let alone SSDs to each node. The random IO latency of SSD is so considerably lower than mechanical disk, even with the OCFS and iSCSI SAN overhead, overall read/write latency will likely be lower than using local mech disk in the nodes. And you get to retain centralized storage of the indexes, eliminating stickiness complexity issues. Something else to consider is the read/write caching performance of NexentaStor (I've never used it, know nothing about it). If it's very good, and the NexentaStor host has gobs of RAM (think 64-128GB), then adding SSDs for indexes may not improve performance much, if any, depending on the concurrent user load. I've read cases where adding SLC-SSD to high end FC SAN controllers with gobs of writeback cache RAM yielded little benefit with similar random IO workloads, simply because the cache was never taxed enough to force regular flushing. If your cache is large and fast enough to buffer most of your IOPS, then your current spindle speed is already irrelevant. In such a case adding SSD will yield little, or no, advantage. -- Stan From knute2011 at knutejohnson.com Sun Nov 6 01:35:46 2011 From: knute2011 at knutejohnson.com (Knute Johnson) Date: Sat, 05 Nov 2011 16:35:46 -0700 Subject: [Dovecot] Converting from 1.12 to 2.0 questions about mail location? In-Reply-To: <4EB5B566.7070808@knutejohnson.com> References: <4EB5B566.7070808@knutejohnson.com> Message-ID: <4EB5C852.5060301@knutejohnson.com> On 11/5/2011 3:15 PM, Knute Johnson wrote: > I'm building a new server and using a newer version of ubuntu server > that uses Dovecot 2.0. I thought I would like to use Maildir instead of > the mbox that was used in the previous server. What do I have to do to > convert my mbox mail to Maildir? Or should I just use mbox and copy it > over :-)? > > Thanks, > After looking at it some more, I think I'm just going to keep the mbox format mail. Thanks, -- Knute Johnson knute2011 at knutejohnson.com From e-frog at gmx.de Sun Nov 6 18:03:47 2011 From: e-frog at gmx.de (e-frog) Date: Sun, 06 Nov 2011 17:03:47 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub Message-ID: <4EB6AFE3.3020001@gmx.de> Hi Timo, I'm using the virtual plugin and have 2 mailboxes configured in the "virtual" namespace (dovecot -n attached). Using lsub with dovecot 2.0.15 leads to the following result: 1 lsub "" "virtual/*" * LSUB () "/" "virtual/Lists" * LSUB () "/" "virtual/unread" 1 OK Lsub completed. However with dovecot 2.1 (latest hg as of today) and the same config there is a 3rd mailbox "virtual" listed which doesn't exist: 1 lsub "" "virtual/*" * LSUB () "/" "virtual" * LSUB () "/" "virtual/Lists" * LSUB () "/" "virtual/unread" 1 OK Lsub completed. Thunderbird shows the "virtual" mailbox as selectable and not grayed out as with 2.0.15. Clicking on that mailbox gives an error message "Mailbox doesn't exist: virtual". Thanks, e-frog -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf-n.2.0.15.txt URL: From mds at schinhammer.de Mon Nov 7 01:54:28 2011 From: mds at schinhammer.de (Michael Schinhammer) Date: Mon, 07 Nov 2011 00:54:28 +0100 Subject: [Dovecot] Problems with password verification on a BSD box against PAM Message-ID: <4EB71E34.5050706@schinhammer.de> Spam detection software, running on the system "ns.089.at", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hi ML, i want to use my local users database on my Freebsd 8.2 box and got the following error in the maillog. The password is ok and it working on the console. i can establish the ssl connection after login "a login ux password" the error occures i cant login. [...] Content analysis details: (6.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.2 MR_NOT_ATTRIBUTED_IP Beta rule: an non-attributed IPv4 found in headers 0.6 J_CHICKENPOX_65 BODY: 6alpha-pock-5alpha 0.6 J_CHICKENPOX_73 BODY: 7alpha-pock-3alpha 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS 0.0 NO_RDNS2 Sending MTA has no reverse DNS 3.5 TO_NO_BRKTS_DIRECT To: misformatted and direct-to-MX -------------- next part -------------- An embedded message was scrubbed... From: Michael Schinhammer Subject: Problems with password verification on a BSD box against PAM Date: Mon, 07 Nov 2011 00:54:28 +0100 Size: 2066 URL: From pw at wk-serv.de Mon Nov 7 02:08:23 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 07 Nov 2011 01:08:23 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB59802.9040205@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> Message-ID: <4EB72177.8090801@wk-serv.de> Ed W schrieb: > See the "sticky" in my reply. You use one of several techniques to > ensure that users always end up on the server with the indexes on. That > way much of the IO is served from that local machine and you only access > the SAN for the (in theory much less frequent) access to the mail files > themselves. I know you can afford that (IMAP) users always end up on one particular server but afaik this only works for incoming IMAP connections. My mail exchangers use dovecot-lda and I think indexes will be written from these servers too or am I wrong with this? Patrick From komodo at uvt.cz Mon Nov 7 12:10:20 2011 From: komodo at uvt.cz (komodo) Date: Mon, 7 Nov 2011 11:10:20 +0100 Subject: [Dovecot] Missing public folder Message-ID: <201111071110.20143.komodo@uvt.cz> Hi i am using dovecot 2.0.13 and i can't see public folder in my mail client. But what is strange, it was working after instalation, and i don't know when it dissapeared. Maybe after some upgrade ? When i enable debug, then i can see in my log that folder exists and no errors. Please where should i find the problem ? Thanks Martin Here is log Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: Namespace : type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: maildir++: root=/home/komodo/Maildir, index=, control=, inbox=/home/komodo/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: Namespace : type=public, prefix=spam_learner., sep=., inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/shared/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: maildir++: root=/home/shared/Maildir, index=, control=, inbox= Here is my conf # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab037.1 x86_64 CentOS release 5.5 (Final) default_client_limit = 1300 disable_plaintext_auth = no hostname = somehost.com mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { location = maildir:/home/shared/Maildir prefix = spam_learner. separator = . subscriptions = no type = public } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/before.d/ sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /usr/sbin/sendmail.exim service imap-login { process_limit = 1000 process_min_avail = 30 } ssl_cert = References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> Message-ID: <4EB7BF9A.9020900@webfusion.com> 04-11-2011 23:56, Timo Sirainen yazm??: > On Thu, 2011-10-20 at 12:38 +0300, Mark Zealey wrote: > >> I'm currently changing some systems to use the redirector service, which >> means that to get local deliveries going I need to get lmtp set up so it >> can be redirected as well. This is working fine, however we have a >> number of different ports running on our servers depending on which >> brand a customer is using. When the time comes to do the user look up, >> we use (port, user, domain) to do a unique database lookup which works >> fine for everything (proxied pop, imap, lmtp& straight pop, imap) but >> not straight lmtp - perhaps because it doesn't have a 2-stage login >> process unlike the pop/imap protocols. Looking at the SQL query we are >> issuing, the %a port variable is set to 0 even though the connection is >> coming in from TCP. Is there a way to change this? We are running >> dovecot 2.0.12 however looking through the changelogs I can't see this >> would be fixed in the newest version. > A quick check in code shows that this is already supposed to work. Set > auth_debug=yes and see if LMTP's auth lookup sends lip, rip, lport and > rport fields to auth process? From my reading of the code in 2.0.15, you pass the local & remote IP in lmtp/commands.c:427 but the struct mail_storage_service_input only has remote_ip and local_ip fields, not port fields (unlike the auth_user_info struct which has both ip's and port entries as well). I think that the mail_storage_service_input struct needs the uint local_port, remote_port fields adding in & appropriate code changes to pass these through? Mark From gordon.grubert at uni-greifswald.de Sun Nov 6 20:56:05 2011 From: gordon.grubert at uni-greifswald.de (Gordon Grubert) Date: Sun, 06 Nov 2011 19:56:05 +0100 Subject: [Dovecot] Dovecot crashes totally In-Reply-To: <1320435812.21919.150.camel@hurina> References: <4EA317B5.3090209@uni-greifswald.de> <1320435812.21919.150.camel@hurina> Message-ID: <4EB6D845.7040208@uni-greifswald.de> On 11/04/2011 08:43 PM, Timo Sirainen wrote: > On Sat, 2011-10-22 at 21:21 +0200, Gordon Grubert wrote: >> Hello, >> >> our dovecot server crashes totally without any really useful >> log messages. The error log can be found in the attachment. >> The only way to get dovecot running again is a complete >> system restart. > > How often does it break? If really a "complete system restart" is needed > to fix it, it doesn't sound like a Dovecot problem. Check if it's enough > to stop dovecot and then make sure there aren't any dovecot processes > lying around afterwards. Currently, the problem occurred three times. The last time some days ago. The last "crash" was in the night and, therefore, we used the chance for a detailed debugging of the system. You could be right, that it's not a dovecot problem. Next to dovecot, we found other processes hanging and could not be killed by "kill -9". Additionally, we found a commonness of all of these processes: They hanged while trying to access the mailbox volume. Therefore, we repaired the filesystem. Now, we're watching the system ... >> Oct 11 09:55:23 mailserver2 dovecot: master: Error: service(imap): >> Initial status notification not received in 30 seconds, killing the >> process >> Oct 11 09:56:23 mailserver2 dovecot: imap-login: Error: master(imap): >> Auth request timed out (received 0/12 bytes) > > Kind of looks like auth process is hanging. You could see if stracing it > shows anything useful. Also are any errors logged about LDAP? Is LDAP > running on the same server? Dovecot authenticates against postfix and postfix has an LDAP connection. The LDAP is running on an external cluster. Here, no errors are reported. We hope, that the filesystem error was the reason for the problem and, that the problem is fixed by repairing it. Best regards, Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5396 bytes Desc: S/MIME Cryptographic Signature URL: From yuichi at sklc.co.jp Mon Nov 7 17:12:33 2011 From: yuichi at sklc.co.jp (Yuuichi Ikeda (SKLC)) Date: Tue, 08 Nov 2011 00:12:33 +0900 Subject: [Dovecot] POP3/IMAPv4 CRAM-MD5 Authentication failed.(Re-post) Message-ID: <20111108001232.2ACB.1986BE3C@sklc.co.jp> Hi, I'm Yuuichi Ikeda from Japan User. OS:Solaris 10 9/10 s10x_u9wos_14a X86 Mem:8GB HDD:3TB gcc:gcc (GCC) 4.1.2 gcc-prefix:/unsupported/gcc Dovecot Version:2.0.15 configure:./configure --prefix=/opt/dovecot_2 --sysconfdir=/opt/dovecot_2/conf --mandir=/opt/man --enable-shared --with-mysql --with-zlib --with-sqlite --with-sql=plugin --with-ssldir=/opt/openssl --with-rundir=/var/run --with-libiconv-prefix=/opt/libiconv Dovecot Configuration > # 2.0.15: /opt/dovecot_2/conf/dovecot/dovecot.conf > # OS: SunOS 5.10 i86pc > auth_debug = yes > auth_mechanisms = cram-md5 > auth_ssl_require_client_cert = yes > auth_ssl_username_from_cert = yes > auth_verbose = yes > base_dir = /var/run/dovecot/ > doveadm_worker_count = 10 > log_path = /var/log/dovecot/dovecot.log > login_greeting = ready. > login_trusted_networks = 192.168.1.0/24 > mail_location = maildir:~/Maildir > passdb { > driver = pam > } > passdb { > args = /opt/dovecot_2/conf/dovecot/passwd > driver = passwd-file > } > plugin { > acl = vfile:/opt/dovecot_2/conf/dovecot/global-acls:cache_secs=300 > acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes > } > protocols = imap pop3 > service auth { > executable = /opt/dovecot_2/libexec/dovecot/auth > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > } > service imap-login { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/imap-login > inet_listener imap { > port = 143 > ssl = no > } > inet_listener imaps { > port = 993 > ssl = yes > } > } > service imap { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/imap > } > service lmtp { > unix_listener lmtp { > mode = 0666 > } > } > service pop3-login { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/pop3-login > inet_listener pop3 { > port = 110 > ssl = no > } > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service pop3 { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/pop3 > } > ssl_ca = ssl_cert = ssl_key = ssl_verify_client_cert = yes > userdb { > args = blocking=yes > driver = passwd > } > protocol imap { > imap_logout_format = bytes=%i/%o > imap_max_line_length = 64 k > mail_max_userip_connections = 10 > mail_plugins = > } > protocol lda { > hostname = mailsv.sklc.co.jp > info_log_path = /var/log/dovecot/deliver.log > log_path = /var/log/dovecot/deliver.log > mail_plugins = > postmaster_address = postmaster at sklc.co.jp > sendmail_path = /usr/lib/sendmail > } > protocol lmtp { > mail_plugins = > } > protocol pop3 { > mail_plugins = > pop3_save_uidl = yes > pop3_uidl_format = %v-%u > } If it attests by connecting by POP3 or IMAPv4, the following messages will be displayed and attestation will go wrong. > Nov 07 23:12:40 auth: Debug: auth client connected (pid=20018) > Nov 07 23:12:40 auth: Debug: client in: AUTH 1 CRAM-MD5 service=pop3 secured no-penalty lip=192.168.1.1 rip=192.168.1.110 lport=110 rport=57054 > Nov 07 23:12:40 auth: Info: CRAM-MD5(?,192.168.1.110): Client didn't present valid SSL certificate > Nov 07 23:12:40 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate > Nov 07 23:12:40 pop3-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.110, lip=192.168.1.1, secured > Nov 07 23:16:32 auth: Debug: auth client connected (pid=20126) > Nov 07 23:16:32 auth: Debug: client in: AUTH 1 CRAM-MD5 service=imap secured no-penalty lip=192.168.1.1 rip=192.168.1.1 lport=143 rport=58734 > Nov 07 23:16:32 auth: Info: CRAM-MD5(?,192.168.1.1): Client didn't present valid SSL certificate > Nov 07 23:16:32 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate > Nov 07 23:16:32 imap-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.1, lip=192.168.1.1, secured What will you do and will become like this? If some people know ways of coping, please let me know. ============================================================= Information-system part. Sankei-Koumuten Co.,Ltd. Yuuichi Ikeda Mail:yuichi at sklc.co.jp Tel.+81-3-3623-6474 Fax.+81-3-3623-6475 Our company promotes "Team minus 6 percent" jus, Hatena Joined member. LPIC-2 Certified. ============================================================= From micah at riseup.net Mon Nov 7 19:13:07 2011 From: micah at riseup.net (micah anderson) Date: Mon, 07 Nov 2011 12:13:07 -0500 Subject: [Dovecot] Accessing a strange mailbox In-Reply-To: <1320435243.21919.144.camel@hurina> References: <8762jie6ut.fsf@algae.riseup.net> <1320435243.21919.144.camel@hurina> Message-ID: <87aa87onz0.fsf@algae.riseup.net> Hi, thanks for the reply! On Fri, 04 Nov 2011 21:34:03 +0200, Timo Sirainen wrote: > On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote: > > I have a user who has a mailbox called: > > > > A----->B > > > > It seemed to work in courier fine, they managed to create it, and there > > are mails in it. However, dovecot is not letting the user access it, the > > IMAP server gives an error. I tried to rename it but I would also get an > > error: > > > > # doveadm mailbox rename 'A----->B' AtoB > > doveadm(root): Error: Can't rename mailbox A----->B to AtoB: Mailbox doesn't exist: A----->B > > Maybe it's not named that? What does it look like in filesystem? Worked > fine with me: I moved the user from courier maildir to mdbox, and I've still got the maildir folders around, the filesystem shows it as: drwx------ 6 mail mail 4096 Oct 20 07:56 .A----->B I just realized my mistake, my 'doveadm mailbox rename' command wasn't limited to the particular user (I was not passing -u ) so it was looking in the wrong namespace for that user. Once I specified the user, it worked properly. Micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From user+dovecot at localhost.localdomain.org Mon Nov 7 22:13:25 2011 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 07 Nov 2011 21:13:25 +0100 Subject: [Dovecot] POP3/IMAPv4 CRAM-MD5 Authentication failed.(Re-post) In-Reply-To: <20111108001232.2ACB.1986BE3C@sklc.co.jp> References: <20111108001232.2ACB.1986BE3C@sklc.co.jp> Message-ID: <4EB83BE5.5030006@localhost.localdomain.org> On 11/07/2011 04:12 PM Yuuichi Ikeda (SKLC) wrote: > Hi, I'm Yuuichi Ikeda from Japan User. > > > Dovecot Configuration >> ? >> ssl_ca = > ssl_cert = > ssl_key = > ssl_verify_client_cert = yes > ? > If it attests by connecting by POP3 or IMAPv4, the following messages will > be displayed and attestation will go wrong. > >> Nov 07 23:12:40 auth: Info: CRAM-MD5(?,192.168.1.110): Client didn't present valid SSL certificate >> Nov 07 23:12:40 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate >> Nov 07 23:12:40 pop3-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.110, lip=192.168.1.1, secured > >> Nov 07 23:16:32 auth: Info: CRAM-MD5(?,192.168.1.1): Client didn't present valid SSL certificate >> Nov 07 23:16:32 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate >> Nov 07 23:16:32 imap-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.1, lip=192.168.1.1, secured > > What will you do and will become like this? If some people know ways of > coping, please let me know. Are you sure you want to verify the client's certificate (ssl_verify_client_cert = yes)? If not, just remove this line and try again. Regards, Pascal -- The trapper recommends today: cafebabe.1131121 at localdomain.org From alietsantiesteban at gmail.com Tue Nov 8 01:02:23 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Mon, 7 Nov 2011 18:02:23 -0500 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: <1320440605.21919.198.camel@hurina> References: <1320440605.21919.198.camel@hurina> Message-ID: We checked with the firewall admins and they can not change the drop action, this model doesn't support reject, only drops, but for testing they disabled the ldap protocol idle timeout wich was set to 30 mins to never so the firewall never drops ldap idle connections, we also verified the clientidletimeout option in Openldap but is set to 0 wich means never close a idle connection. After testing again we see the connection hanging again after user inactivity, we will keep looking for other issues and maybe do some packet captures to see what is really happening. best regards, btw it would be great this ldap_idle_disconnect = 30s 2011/11/4 Timo Sirainen > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes wrote: > > I'm having a problem with dovecot ldap connection when ldap server is in > > another firewall zone, firewall kills the ldap connection after a > > determined period of inactivity, this is good from the firewall point of > > view but is bad for dovecot because it never knows the connections has > been > > dropped, this creates longs timeouts in dovecot and finally it > reconnects, > > meanwhile many users fails to authenticate, I have seen this kind of post > > in the list for a while but can't find a solution for it, so my question > is > > how to define a idle ldap time in dovecot so it can reconnect before the > > firewall has dropped the connection or just close the connection under > > inactivity so when a user authenticate doesn't fails for a while until > > dovecot detects that the connection has hanged. Is this a feature request > > or there is already a configuration for this??? > > Can't the firewall be changed to reject the LDAP packets instead of > dropping them? Then Dovecot would immediately notice that the connection > has died, and with a recent enough version it wouldn't even log an error > about it. > > I guess some kind of an "ldap_idle_disconnect = 30s" setting could be > added, but it's not a very high priority for me. > > > From tss at iki.fi Tue Nov 8 01:11:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 08 Nov 2011 01:11:02 +0200 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: References: <1320440605.21919.198.camel@hurina> Message-ID: <1320707463.21919.337.camel@hurina> If you set openldap server to close idle clients sooner than the connection itself is dropped by firewall (or whatever), then Dovecot sees the disconnection and won't hang. So you could try something like clientidletimeout=5 mins On Mon, 2011-11-07 at 18:02 -0500, Aliet Santiesteban Sifontes wrote: > We checked with the firewall admins and they can not change the drop > action, this model doesn't support reject, only drops, but for testing > they disabled the ldap protocol idle timeout wich was set to 30 mins > to never so the firewall never drops ldap idle connections, we also > verified the clientidletimeout option in Openldap but is set to 0 wich > means never close a idle connection. After testing again we see the > connection hanging again after user inactivity, we will keep looking > for other issues and maybe do some packet captures to see what is > really happening. > best regards, btw it would be great this ldap_idle_disconnect = 30s > > 2011/11/4 Timo Sirainen > > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes > wrote: > > I'm having a problem with dovecot ldap connection when ldap > server is in > > another firewall zone, firewall kills the ldap connection > after a > > determined period of inactivity, this is good from the > firewall point of > > view but is bad for dovecot because it never knows the > connections has been > > dropped, this creates longs timeouts in dovecot and finally > it reconnects, > > meanwhile many users fails to authenticate, I have seen this > kind of post > > in the list for a while but can't find a solution for it, so > my question is > > how to define a idle ldap time in dovecot so it can > reconnect before the > > firewall has dropped the connection or just close the > connection under > > inactivity so when a user authenticate doesn't fails for a > while until > > dovecot detects that the connection has hanged. Is this a > feature request > > or there is already a configuration for this??? > > > Can't the firewall be changed to reject the LDAP packets > instead of > dropping them? Then Dovecot would immediately notice that the > connection > has died, and with a recent enough version it wouldn't even > log an error > about it. > > I guess some kind of an "ldap_idle_disconnect = 30s" setting > could be > added, but it's not a very high priority for me. > > > From alietsantiesteban at gmail.com Tue Nov 8 04:41:12 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Mon, 7 Nov 2011 21:41:12 -0500 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: <1320707463.21919.337.camel@hurina> References: <1320440605.21919.198.camel@hurina> <1320707463.21919.337.camel@hurina> Message-ID: We will try this as next step to find a workaround, the problem with client idletimeout=5 mins in openldap server is that is a global server definition and have the net effect of changing replication refreshAndPersit into type refreshOnly which is not a welcome side effect, we will look other options, still the better candidate is ldap_idle_disconnect in dovecot side or any other kind of logic able to detect this kind of problems. best regards 2011/11/7 Timo Sirainen > If you set openldap server to close idle clients sooner than the > connection itself is dropped by firewall (or whatever), then Dovecot > sees the disconnection and won't hang. So you could try something like > clientidletimeout=5 mins > > On Mon, 2011-11-07 at 18:02 -0500, Aliet Santiesteban Sifontes wrote: > > We checked with the firewall admins and they can not change the drop > > action, this model doesn't support reject, only drops, but for testing > > they disabled the ldap protocol idle timeout wich was set to 30 mins > > to never so the firewall never drops ldap idle connections, we also > > verified the clientidletimeout option in Openldap but is set to 0 wich > > means never close a idle connection. After testing again we see the > > connection hanging again after user inactivity, we will keep looking > > for other issues and maybe do some packet captures to see what is > > really happening. > > best regards, btw it would be great this ldap_idle_disconnect = 30s > > > > 2011/11/4 Timo Sirainen > > > > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes > > wrote: > > > I'm having a problem with dovecot ldap connection when ldap > > server is in > > > another firewall zone, firewall kills the ldap connection > > after a > > > determined period of inactivity, this is good from the > > firewall point of > > > view but is bad for dovecot because it never knows the > > connections has been > > > dropped, this creates longs timeouts in dovecot and finally > > it reconnects, > > > meanwhile many users fails to authenticate, I have seen this > > kind of post > > > in the list for a while but can't find a solution for it, so > > my question is > > > how to define a idle ldap time in dovecot so it can > > reconnect before the > > > firewall has dropped the connection or just close the > > connection under > > > inactivity so when a user authenticate doesn't fails for a > > while until > > > dovecot detects that the connection has hanged. Is this a > > feature request > > > or there is already a configuration for this??? > > > > > > Can't the firewall be changed to reject the LDAP packets > > instead of > > dropping them? Then Dovecot would immediately notice that the > > connection > > has died, and with a recent enough version it wouldn't even > > log an error > > about it. > > > > I guess some kind of an "ldap_idle_disconnect = 30s" setting > > could be > > added, but it's not a very high priority for me. > > > > > > > > > From Yann.Dupont at univ-nantes.fr Tue Nov 8 12:05:42 2011 From: Yann.Dupont at univ-nantes.fr (Yann Dupont) Date: Tue, 08 Nov 2011 11:05:42 +0100 Subject: [Dovecot] Attachements with SIS : shauld all the same files share the same inode ? In-Reply-To: <1320510194.21919.251.camel@hurina> References: <4E9BD704.9080801@univ-nantes.fr> <1320510194.21919.251.camel@hurina> Message-ID: <4EB8FEF6.7000208@univ-nantes.fr> Hi timo, thanks for your answer Le 05/11/2011 17:23, Timo Sirainen a ?crit : > On Mon, 2011-10-17 at 09:19 +0200, Yann Dupont wrote: >> 104 instance of -rw------- 104 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-ffd0d1xxxx >> >> The files are hardlinked, so it seems ok, but I also see 9 others files >> in the same directory : >> >> >> -rw------- 1 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-0519df2a94fc9a4e1f4100008a6b99b6 >> -rw------- 1 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-1790b92994fc9a4e1a4100008a6b99b6 >> >> as you can see, same prefix, If I compare them with diff, they are all >> the same files. > > I guess you have "mail_attachment_fs = sis posix" (default)? Sounds like yes that's the case. > there could be a bug where SIS does comparison of files but for some > reason thinks they are different. > Well I can't try anymore on this, because this particular attachment is not there. In fact this server keeps only 2 weeks of mails for 5000+ accounts (a form of backup for the main servers). I'll take a new one : -> rw------- 103 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c0c2230ecdd1b74ec91600008a6b99b6 -rw------- 103 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c1c1290ecdd1b74ec51600008a6b99b6 -rw------- 1 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 So f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 has only one link, but diff f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c1c1290ecdd1b74ec51600008a6b99b6 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 tell nothing : they are same files. > You could try if you can merge them with: > > mkdir /tmp/queue > touch /tmp/queue/6e2df299fa96daec9b4735d07c494046429be4d6-foo > doveadm sis deduplicate /var/attachments /tmp/queue > > hum I didn't get what you want to do ? Does putting a 0 byte file there force a complete rescan ? Tried it : the 0 byte length /tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo disappear from the /tmp/queue, but f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 still has one link. A strace reveal this stat("/vmail/attachments-deduplic", {st_mode=S_IFDIR|0700, st_size=8192, ...}) = 0 open("/tmp/queue", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 8 getdents(8, /* 3 entries */, 32768) = 112 stat("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 link("/vmail/attachments-deduplic/f7/c9/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo", "/vmail/attachments-deduplic/f7/c9/hashes/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667") = -1 ENOENT (No such file or directory) mkdir("/vmail/attachments-deduplic/f7/c9/hashes", 0700) = -1 EEXIST (File exists) unlink("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo") = 0 getdents(8, /* 0 entries */, 32768) = 0 close(8) = 0 Tried with a copy of the file but this doesn't work either : stat("/vmail/attachments-deduplic", {st_mode=S_IFDIR|0700, st_size=8192, ...}) = 0 open("/tmp/queue", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 8 getdents(8, /* 3 entries */, 32768) = 144 stat("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6", {st_mode=S_IFREG|0600, st_size=571037, ...}) = 0 write(2, "doveadm(root): Fatal: /tmp/queue"..., 170doveadm(root): Fatal: /tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 is not a valid sis-queue file, is the queue directory correct? How can I help further ? Thanks, -- Yann Dupont - Service IRTS, DSI Universit? de Nantes Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont at univ-nantes.fr From listas.correo at yahoo.es Tue Nov 8 12:20:22 2011 From: listas.correo at yahoo.es (mailing lists) Date: Tue, 8 Nov 2011 10:20:22 +0000 (GMT) Subject: [Dovecot] Failing to share folders when listescape is enabled (2.0.14) Message-ID: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> Hello, How I can share a folder with dots when the listescape plugin is enabled? In this example user001 is sharing two foders named "docs-abc" and "docs-a.b.c" to user002. The first folder (without dots) is seen by user002 but the second is not found by dovecot because it search a system folder named ".docs-a.b.c" but the system folder is ".docs-a\2eb\2ec" Nov? 8 11:03:52 imap2 dovecot: imap(user002): Debug: acl vfile: file /var/virtual-maildir/user001/.docs-a.b.c/dovecot-acl not found drwx------ 2 vmail vmail 4096 2011-11-08 10:54 cur drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .docs-a\2eb\2ec drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .docs-abc -rw------- 1 vmail vmail?? 20 2011-11-08 10:57 dovecot-acl-list -rw------- 1 vmail vmail? 248 2011-11-08 10:55 dovecot.index.log -rw------- 1 vmail vmail?? 96 2011-11-08 10:54 dovecot.mailbox.log -rw------- 1 vmail vmail?? 51 2011-11-08 10:55 dovecot-uidlist -rw------- 1 vmail vmail??? 8 2011-11-08 10:57 dovecot-uidvalidity -r--r--r-- 1 vmail vmail??? 0 2011-11-08 10:54 dovecot-uidvalidity.4eb8fc5b drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .Drafts -rw------- 1 vmail vmail?? 15 2011-11-08 10:54 maildirsize drwx------ 2 vmail vmail 4096 2011-11-08 10:54 new drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Sent drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Spam -rw------- 1 vmail vmail?? 23 2011-11-08 10:54 subscriptions drwx------ 2 vmail vmail 4096 2011-11-08 10:54 tmp drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Trash # cat /var/virtual-maildir/user001/subscriptions Trash Sent Drafts Spam docs-abc docs-a\2eb\2ec # telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. . login user001 XXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in . create docs-abc??? . OK Create completed. . create docs-a.b.c . OK Create completed. . setacl docs-abc user002 lrsw . OK Setacl complete. . setacl docs-a.b.c user002 lrsw . OK Setacl complete. . subscribe docs-abc . OK Subscribe completed. . subscribe docs-a.b.c . OK Subscribe completed. . logout * BYE Logging out . OK Logout completed. Connection closed by foreign host. # telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. . login user002 XXXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in . subscribe shared/user001/docs-abc . OK Subscribe completed. . subscribe shared/user001/docs-a.b.c . NO Mailbox doesn't exist: shared/user001/docs-a.b.c . logout * BYE Logging out . OK Logout completed. Connection closed by foreign host. ??? /-----------------/ # dovecot -n # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34.7-0.7-xen x86_64 openSUSE 11.3 (x86_64) auth_debug = yes auth_verbose = yes base_dir = /var/run/dovecot/ dict { ? expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_proxy_port = 24244 doveadm_socket_path = localhost:24244 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_debug = yes mail_fsync = always mail_gid = 5000 mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = acl quota notify expire listescape mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { ? inbox = yes ? list = yes ? location = ? prefix = ? separator = / ? subscriptions = yes ? type = private } namespace { ? list = children ? location = maildir:/var/virtual-maildir/%%n:INDEX=~/shared.%%n ? prefix = shared/%%n/ ? separator = / ? subscriptions = no ? type = shared } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } plugin { ? acl = vfile ? acl_shared_dict = file:/var/maildir/vol00/shared-mailboxes ? autocreate = Trash ? autocreate2 = Sent ? autocreate3 = Drafts ? autocreate4 = Spam ? autosubscribe = Trash ? autosubscribe2 = Sent ? autosubscribe3 = Drafts ? autosubscribe4 = Spam ? expire = Trash ? expire2 = Trash/* ? expire5 = Spam ? expire6 = Spam/* ? expire_dict = proxy::expire ? mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append ? mail_log_fields = uid box msgid from subject size vsize flags ? mail_plugins = " mail_log notify" ? quota = maildir:Quota ? quota_exceeded_message = Quota limit reached ? quota_rule2 = Trash:storage=+50M ? quota_warning = storage=95%% quota-warning 95 %u ? quota_warning2 = storage=90%% quota-warning 90 %u ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } service auth { ? unix_listener auth-userdb { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service dict { ? unix_listener dict { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service doveadm { ? inet_listener { ??? port = 24244 ? } } service lmtp { ? inet_listener lmtp { ??? port = 24 ? } ? unix_listener lmtp { ??? group = vmail ??? mode = 0666 ??? user = vmail ? } } service pop3-login { ? inet_listener pop3 { ??? port = 110 ? } } service quota-warning { ? executable = script /etc/dovecot/quota-warning.sh ? unix_listener quota-warning { ??? user = vmail ? } } ssl = no userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } verbose_proctitle = yes protocol lmtp { ? mail_plugins = acl quota notify expire listescape sieve } protocol lda { ? mail_plugins = acl quota notify expire listescape sieve } protocol imap { ? mail_max_userip_connections = 100 ? mail_plugins = acl quota notify expire listescape imap_acl autocreate imap_quota } protocol pop3 { ? pop3_uidl_format = %v-%u } local 10.1.245.101/28/28 { ? doveadm_password = secret } From tss at iki.fi Tue Nov 8 14:26:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 14:26:10 +0200 Subject: [Dovecot] Failing to share folders when listescape is enabled (2.0.14) In-Reply-To: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> References: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> Message-ID: <3F867C40-86D5-4572-A6A7-8B6C685245B9@iki.fi> On 8.11.2011, at 12.20, mailing lists wrote: > How I can share a folder with dots when the listescape plugin is enabled? Sorry, doesn't work in v2.0 and I don't think it's possible to fix it without major changes. Those major changes are done in v2.1 code tree though, and it works there. From p.heinlein at heinlein-support.de Tue Nov 8 15:50:25 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 14:50:25 +0100 Subject: [Dovecot] Performance-Tuning Message-ID: <201111081450.25798.p.heinlein@heinlein-support.de> Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff). I have to change my setup, maybe somebody else here have hard facts: *) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout? *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Thanks for any hints an tricks, Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From ricardo at wenn.com Tue Nov 8 15:59:01 2011 From: ricardo at wenn.com (Ricardo Branco) Date: Tue, 8 Nov 2011 13:59:01 +0000 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <28618879-1320760744-cardhu_decombobulator_blackberry.rim.net-908347421-@b13.c3.bise7.blackberry> What is the setup on the EVA, FC or iSCSI? Sent from my BlackBerry? wireless device -----Original Message----- From: Peer Heinlein Sender: dovecot-bounces at dovecot.org Date: Tue, 8 Nov 2011 14:50:25 To: Subject: [Dovecot] Performance-Tuning Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff). I have to change my setup, maybe somebody else here have hard facts: *) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout? *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Thanks for any hints an tricks, Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From mstevens at imt-systems.com Tue Nov 8 16:03:07 2011 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 08 Nov 2011 15:03:07 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> On 08.11.2011 14:50, Peer Heinlein wrote: > *) Is mdbox really faster? I'd like to have mdbox to have better > performance > in running my backup-processes. But does it bring some performance > boosts > to? Hi Peer, We have switched our mailbox storage format from maildir to mdbox! Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly. Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4. Best regards, Morten From javierdemiguel at us.es Tue Nov 8 16:15:39 2011 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_de_Miguel_Rodr=EDguez?=) Date: Tue, 08 Nov 2011 15:15:39 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EB9398B.1000904@us.es> We are very happy with mdbox+zlib+ext4 + iSCSI SAN (HP Lefthand in our setup) If you have CPU to spare, consider using zlib with mdbox. You are trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Mdbox has halved our backup windows (2,8 TB uncompressed mailboxes, 2 TB compressed) and backup software is happier because there are few (100.000+ files with mdbox) to backup instead of several millions (Maildir) Regards Javier > Hi, > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. > > I always wanted to make some mesurements about several influences to the > performance (switch to ext4, switch to mdbox), but I never had enough time > to do that. > > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). > > I have to change my setup, maybe somebody else here have hard facts: > > *) Is ext4 faster? How much faster? > *) Is it faster because of the ext4 kernel-module (which can be used on ext3 > to) or because of the ext4 filesystem layout? > > > *) Is mdbox really faster? I'd like to have mdbox to have better performance > in running my backup-processes. But does it bring some performance boosts > to? > > > Thanks for any hints an tricks, > > Peer > > From Ralf.Hildebrandt at charite.de Tue Nov 8 16:16:12 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 8 Nov 2011 15:16:12 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> Message-ID: <20111108141612.GE27296@charite.de> * Morten Stevens : > We have switched our mailbox storage format from maildir to mdbox! I wonder how I can incrementally change over from Maildir to mdbox? I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa). That would allow for a smooth transition... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From p.heinlein at heinlein-support.de Tue Nov 8 16:19:05 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:19:05 +0100 Subject: [Dovecot] Multiple Patitions with with mdbox Message-ID: <201111081519.05867.p.heinlein@heinlein-support.de> Having > 10 TByte mailstore filesystem-checks takes too much time. At the moment we have four different partitions, but I don't like to set symlinks or LDAP-flags to sort customers and their domains to there individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate the path automatically. How do you handle >> 10 TB mailstore? I'm very interested in the feature "alternative mailstore" with mdbox, because that makes it very easy to use at least TWO filesystems without any tricky configuration. I think I'd love to have alternative mailstores. Why does dbox doesn't look for its m.*-files in more then two directorys? Sure, looking in 4 directorys would lead to four disc operations, but maybe it could be very helpful. Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Tue Nov 8 16:22:24 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:22:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EB9398B.1000904@us.es> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EB9398B.1000904@us.es> Message-ID: <201111081522.24386.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodr?guez: Hi, > If you have CPU to spare, consider using zlib with mdbox. You are > trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Hey. This point is great. I hadn't realized that. Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks. > compressed) and backup software is happier because there are few > (100.000+ files with mdbox) to backup instead of several millions > (Maildir) Yes, that#s the main reason why I want to switch to mbox. At the moment our roundtrip-time for the backup is > 24h... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From javierdemiguel at us.es Tue Nov 8 16:29:02 2011 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_de_Miguel_Rodr=EDguez?=) Date: Tue, 08 Nov 2011 15:29:02 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081522.24386.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EB9398B.1000904@us.es> <201111081522.24386.p.heinlein@heinlein-support.de> Message-ID: <4EB93CAE.3080308@us.es> Other important thing to consider is message expunging. With mdbox you are "delaying" the I/O associated with deleting e-mails. We have a nightly cronjob that expunge messages from mdboxes. If you have en EVA (wich one? 4.400? 6.400? ) you also can consider RAID 1+0 or SSD for indexes. Indexes are hammered in mdbox. Regards Javier > Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodr?guez: > > > Hi, > >> If you have CPU to spare, consider using zlib with mdbox. You are >> trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). > Hey. This point is great. I hadn't realized that. > > Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks. > >> compressed) and backup software is happier because there are few >> (100.000+ files with mdbox) to backup instead of several millions >> (Maildir) > Yes, that#s the main reason why I want to switch to mbox. At the moment our > roundtrip-time for the backup is> 24h... > > > Peer > > From p.heinlein at heinlein-support.de Tue Nov 8 16:34:51 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:34:51 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <201111081534.52068.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: Hi, > I wonder how I can incrementally change over from Maildir to mdbox? If you have double diskspace: Just use "dsync mirror" in the background to prepare the change. After that it's just a short downtime to migrate incremental the last changes, or it's just a question of a short login-script. if [ -d ~/Maildir ] ; then dsync mirror voodoo-magic rm -R ~/Maildr fi > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or > vice versa). I wonder about that problem too. Even the last-last-last-quick sync would be so much IO, that I can't handle it in realtime in the morning at 9 a.m. Looks like a nightly downtime for the last incremental run. It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes. In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So having users with maildir and mdbox mixed up will break their shared folders... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Tue Nov 8 16:36:38 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:36:38 +0100 Subject: [Dovecot] dsync and zlib Message-ID: <201111081536.38710.p.heinlein@heinlein-support.de> Will dsync also use zlib-compression when copying mails from one side to another? Wouldn't this be a good way to compress existing mails? http://wiki2.dovecot.org/Plugins/Zlib says, there's no way to do that. But dsync should respect the zlib-Plugin... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From jester at arcor.de Tue Nov 8 16:42:45 2011 From: jester at arcor.de (Oliver Eales) Date: Tue, 08 Nov 2011 15:42:45 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <4EB93FE5.8020903@arcor.de> On 08.11.2011 15:16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! I never tried it, but it should be possible to provide the mail_location from the user repsoitory (LDAP, SQL, whatever) So you can keep your global config, and use a script to convert one mailbox after another, and add a mail_location extra userdb field in the user repository to overwrite the global setting on a per-user-basis. Regards, Oliver From Ralf.Hildebrandt at charite.de Tue Nov 8 16:44:55 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 8 Nov 2011 15:44:55 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <20111108144455.GG27296@charite.de> * Peer Heinlein : > Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: > > Hi, > > > I wonder how I can incrementally change over from Maildir to mdbox? > > If you have double diskspace: haha :) no. I thought of a per-user migration, that way I don't need extra space. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Nov 8 16:57:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 16:57:42 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <1F29E1C1-7D8F-49F6-AF01-7B50D69F78DB@iki.fi> On 8.11.2011, at 16.34, Peer Heinlein wrote: >> I can of course use dsync to mirror Maildir: to mdbox:, but how can I >> make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or >> vice versa). > > I wonder about that problem too. Even the last-last-last-quick sync would be > so much IO, that I can't handle it in realtime in the morning at 9 a.m. > > Looks like a nightly downtime for the last incremental run. > > It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP > attributes. Easy! > In this case the whole migration process could be split up into > groups. Unfortunately we have shared folders and I don't know a way to read > the *remote* mailbox-format from LDAP... So having users with maildir and > mdbox mixed up will break their shared folders... Not so easy.. Only the home directory can be currently looked up from userdb for shared folders. There is also automatic detection of Maildir and mbox when mail_location isn't set, but no such code for mdbox. It could be added without much trouble though. But for shared folders, assuming you'd want per-user \seen flags, it would also need something like: mail_location = auto::INDEX=~/shared-indexes This "auto" doesn't exist yet either. And then there's the biggest problem: You can't have per-user \seen flags with mdbox, because you can't change the index file path without breaking mdbox. From tss at iki.fi Tue Nov 8 16:59:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 16:59:03 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <877AD4C4-F8A0-48F5-812D-5AAC6FCF13C2@iki.fi> On 8.11.2011, at 16.16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! > > I wonder how I can incrementally change over from Maildir to mdbox? > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or > vice versa). > > That would allow for a smooth transition... If you don't have shared folders (as explained in previous mail) and you can have per-user mail_location in the userdb, this should be pretty easy. The man page for dsync lists the steps that can be used for online migration. From tss at iki.fi Tue Nov 8 17:00:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:00:29 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> Message-ID: On 8.11.2011, at 16.03, Morten Stevens wrote: > We have switched our mailbox storage format from maildir to mdbox! > > Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly. > > Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4. You don't happen to have any specific numbers/graphs that can be used to compare maildir vs. mdbox in the same hardware? I'd be interested in seeing those, such as a graph of disk iops spanning a month before/after mdbox switch. From dirk.jahnke-zumbusch at desy.de Tue Nov 8 17:03:24 2011 From: dirk.jahnke-zumbusch at desy.de (Jahnke-Zumbusch, Dirk) Date: Tue, 8 Nov 2011 16:03:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EB93FE5.8020903@arcor.de> References: <201111081450.25798.p.heinlein@heinlein-support.de><7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com><20111108141612.GE27296@charite.de> <4EB93FE5.8020903@arcor.de> Message-ID: <7C0189D0CDB51646B7814F4377D356F501536694@ADXV4.win.desy.de> Hi there, >I never tried it, but it should be possible to provide the mail_location >from the user repsoitory (LDAP, SQL, whatever) Actually this works :-) Our userdb looks similar to: account1:xyz:000:000::/account1s/home/dir::userdb_mail=maildir:/account1s/home/dir/Maildir account2:xyz:000:000::/account2s/home/dir::userdb_mail=mdbox:/ account2s/home/dir http://wiki2.dovecot.org/UserDatabase/ExtraFields Concerning Maildir backups: what about a backup-to-disc-to-tape scheme using snapshots for the "to-disc" part and something like perpetual incrementals afterwards for the "top-tape" (secondary store) ? Regards Dirk -- Dirk Jahnke-Zumbusch Deutsches Elektronen-Synchrotron DESY IT Information Fabrics Member of the Helmholtz Association D-22603 Hamburg Notkestrasse 85 / 22607 Hamburg T: +49-40-899.81760 F: +49-40-899.41760 dirk.jahnke-zumbusch at desy.de >So you can keep your global config, and use a script to convert one >mailbox after another, and add a mail_location extra userdb field in the >user repository to overwrite the global setting on a per-user-basis. > >Regards, >Oliver From tss at iki.fi Tue Nov 8 17:04:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:04:07 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: On 8.11.2011, at 15.50, Peer Heinlein wrote: > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). A few more ideas for Maildir if you haven't done yet: - maildir_very_dirty_syncs = yes - pop3_no_flag_updates = yes - Switching to dict file quota instead of Maildir++ quota. From tss at iki.fi Tue Nov 8 17:10:11 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:10:11 +0200 Subject: [Dovecot] dsync and zlib In-Reply-To: <201111081536.38710.p.heinlein@heinlein-support.de> References: <201111081536.38710.p.heinlein@heinlein-support.de> Message-ID: On 8.11.2011, at 16.36, Peer Heinlein wrote: > Will dsync also use zlib-compression when copying mails from one side to > another? As long as the zlib plugin is globally enabled and zlib_save setting is set. > Wouldn't this be a good way to compress existing mails? > > http://wiki2.dovecot.org/Plugins/Zlib > > says, there's no way to do that. But dsync should respect the zlib-Plugin... Well .. You could think of compressing existing mails with dsync the same as "migrating from maildir" to "migrating to maildir with zlib enabled" :) I'll add a note to the wiki page. From tss at iki.fi Tue Nov 8 17:37:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:37:01 +0200 Subject: [Dovecot] Multiple Patitions with with mdbox In-Reply-To: <201111081519.05867.p.heinlein@heinlein-support.de> References: <201111081519.05867.p.heinlein@heinlein-support.de> Message-ID: <608C2C35-40AD-41C7-9924-9D4F0BE232A2@iki.fi> On 8.11.2011, at 16.19, Peer Heinlein wrote: > Having > 10 TByte mailstore filesystem-checks takes too much time. > > At the moment we have four different partitions, but I don't like to set > symlinks or LDAP-flags to sort customers and their domains to there > individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate > the path automatically. Why not symlinks? You could use e.g. /mail/%16Hu/%d/%n as part of the path which would give you one hex letter when hashing the username (0..9, a..f). So then you could create symlinks like: /mail/0 -> /storage1 /mail/1 -> /storage2 ..etc.. > I'm very interested in the feature "alternative mailstore" with mdbox, > because that makes it very easy to use at least TWO filesystems without any > tricky configuration. > > I think I'd love to have alternative mailstores. Why does dbox doesn't > look for its m.*-files in more then two directorys? Sure, looking in 4 > directorys would lead to four disc operations, but maybe it could be very > helpful. The current implementation uses a simple message flag to tell when/where to move the mail, so the code would have to get much more complicated with more alt paths. From michael.abbott at apple.com Tue Nov 8 18:43:17 2011 From: michael.abbott at apple.com (Mike Abbott) Date: Tue, 08 Nov 2011 10:43:17 -0600 Subject: [Dovecot] another sizeof tweak Message-ID: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { Should that be sizeof(*old_rec)? From tss at iki.fi Tue Nov 8 18:46:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 18:46:07 +0200 Subject: [Dovecot] another sizeof tweak In-Reply-To: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> References: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Message-ID: On 8.11.2011, at 18.43, Mike Abbott wrote: > Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: > memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { > Should that be sizeof(*old_rec)? Yep, looks like was fixed in v2.1 already. I'll fix it for v2.0 too. From rostetter at mail.utexas.edu Tue Nov 8 18:53:32 2011 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Tue, 08 Nov 2011 10:53:32 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> Quoting Peer Heinlein : > It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP > attributes. In this case the whole migration process could be split up into > groups. Unfortunately we have shared folders and I don't know a way to read > the *remote* mailbox-format from LDAP... So having users with maildir and > mdbox mixed up will break their shared folders... May not work for you, but... The way I did this when I migrated was to run two dovecot instances, and have perdition software on a front-end (could be on the same machine instead of a front-end, I just happen to have a front-end machine to do it). Perdition will query ldap for the info per user/connection, and send the connection to the correct dovecot instance based on the ldap lookup. Worked for me, your milage may vary... -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From jeroen at bizworx-it.nl Tue Nov 8 20:34:30 2011 From: jeroen at bizworx-it.nl (Jeroen Grusewski) Date: Tue, 8 Nov 2011 19:34:30 +0100 Subject: [Dovecot] Dovecot alias and mailbox problem Message-ID: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> Hi All, My Postfix / Dovecot / MySQL is running fine but I have the following issue: When I create the following mailboxes test at example.com and test1 at example.com it is working fine. I can create an alias test2 at examples.com => test1 at example.com that is also working without a problem, only when I create test at example.com => test1 at example.com email is only delivered to test1 at .... instead to both test at .... and test1 at .... I believe it is properly just a setting but please help me out here ! Let me know if you need any more information before I post configs etc. Best regards, Jeroen From tom at whyscream.net Tue Nov 8 20:54:46 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Tue, 08 Nov 2011 19:54:46 +0100 Subject: [Dovecot] Dovecot alias and mailbox problem In-Reply-To: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> References: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> Message-ID: <4EB97AF6.1020606@whyscream.net> On 08-11-11 19:34, Jeroen Grusewski wrote: > > Hi All, > > My Postfix / Dovecot / MySQL is running fine but I have the following issue: > > When I create the following mailboxes test at example.com and test1 at example.com it is working fine. > I can create an alias test2 at examples.com => test1 at example.com that is also working without a problem, > only when I create test at example.com => test1 at example.com email is only delivered to test1 at .... instead to > both test at .... and test1 at .... > > I believe it is properly just a setting but please help me out here ! > > Let me know if you need any more information before I post configs etc. > You probably want to create the alias as test at example.com => test1 at example.com,test at example.com. Depending on how you setup postfix, this should deliver to both mailboxes (and not create an infinite loop). Note that aliasing is handled within Postfix, not in Dovecot, so if you have further questions, you're probably on the wrong list. -- Regards, Tom From p.heinlein at heinlein-support.de Tue Nov 8 21:40:08 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 20:40:08 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> References: <201111081450.25798.p.heinlein@heinlein-support.de> <201111081534.52068.p.heinlein@heinlein-support.de> <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> Message-ID: <201111082040.08183.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 17:53:32 schrieb Eric Rostetter: > May not work for you, but... > > The way I did this when I migrated was to run two dovecot instances, and > have perdition software on a front-end (could be on the same machine > instead of a front-end, I just happen to have a front-end machine to do > it). You could do that with Dovecot, too. > Perdition will query ldap for the info per user/connection, and send the > connection to the correct dovecot instance based on the ldap lookup. > Worked for me, your milage may vary... The problem is: You're running in problems with shared folders. You can't read your neighbors storage-engine from ldap. It's easy to read the user's storage engine from ldap. So there's no need to use perdition for that :-) But you can't read or proxy the storage engine from somebody who shared you his folders. That's my problem :-( Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From rostetter at mail.utexas.edu Tue Nov 8 21:50:56 2011 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Tue, 08 Nov 2011 13:50:56 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111082040.08183.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <201111081534.52068.p.heinlein@heinlein-support.de> <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> <201111082040.08183.p.heinlein@heinlein-support.de> Message-ID: <20111108135056.312858jv1jl7bhgk@mail.ph.utexas.edu> Quoting Peer Heinlein : > The problem is: You're running in problems with shared folders. You can't > read your neighbors storage-engine from ldap. Yes, but I didn't have any shared folders, so it worked. Your milage may vary, as I said... :) -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From tss at iki.fi Tue Nov 8 23:09:52 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 08 Nov 2011 23:09:52 +0200 Subject: [Dovecot] another sizeof tweak In-Reply-To: References: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Message-ID: <1320786593.21919.345.camel@hurina> On Tue, 2011-11-08 at 18:46 +0200, Timo Sirainen wrote: > On 8.11.2011, at 18.43, Mike Abbott wrote: > > > Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: > > memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { > > Should that be sizeof(*old_rec)? > > Yep, looks like was fixed in v2.1 already. I'll fix it for v2.0 too. Actually it doesn't matter in v2.0 since this is mailbox list indexing code, which is forcibly disabled (and broken) in v2.0. From tss at iki.fi Wed Nov 9 00:35:37 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 00:35:37 +0200 Subject: [Dovecot] v2.1.beta1 released Message-ID: <1320791738.21919.350.camel@hurina> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig Here's the first beta release of Dovecot v2.1. This version has already been tested quite a lot, so I'm not expecting any major bugs. So please upgrade and see if you can find any problems. I'm optimistic about getting rc1 released this year and perhaps even v2.1.0. Since alpha2 there have been a lot of fixes, especially to imapc backend. There are probably also some small new features, but nothing huge. As a reminder, here's the largest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now From stephan at rename-it.nl Wed Nov 9 01:37:05 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 09 Nov 2011 00:37:05 +0100 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320791738.21919.350.camel@hurina> References: <1320791738.21919.350.camel@hurina> Message-ID: <4EB9BD21.6060501@rename-it.nl> On 11/8/2011 11:35 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig > > Here's the first beta release of Dovecot v2.1. This version has already > been tested quite a lot, so I'm not expecting any major bugs. So please > upgrade and see if you can find any problems. I'm optimistic about > getting rc1 released this year and perhaps even v2.1.0. > Ok, I guess it is time for a Pigeonhole release for Dovecot v2.1. I'm a bit swamped this week, but next weekend I should be able to compose one. For now you can use the dovecot-2.0-pigeonhole hg and patch it with http://hg.rename-it.nl/pigeonhole-0.2-dovecot-2.1-patches/file/tip/pigeonhole-0.2-dovecot-2.1.patch to make it compile against v2.1. Regards, Stephan. From stan at hardwarefreak.com Wed Nov 9 09:34:04 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 01:34:04 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EBA2CEC.1040701@hardwarefreak.com> On 11/8/2011 7:50 AM, Peer Heinlein wrote: > I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. That's a lot of mail (likely a large user base--not given), on a filesystem not designed for such, on a decent SAN controller--LUN RAID configuration not given. > I always wanted to make some mesurements about several influences to the > performance (switch to ext4, switch to mdbox), but I never had enough time > to do that. If you're going to switch filesystems, for this size dataset and concurrent workload, you're moving in the wrong direction. > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). EXT3/4 are not designed, nor optimized, for high concurrency workloads. > I have to change my setup, maybe somebody else here have hard facts: > > *) Is ext4 faster? How much faster? Simulated maildir workload test on 2.6.35-rc5, 128 threads (No data published for newer kernels): http://btrfs.boxacle.net/repository/raid/2.6.35-rc5/2.6.35-rc5/2.6.35-rc5_Mail_server_simulation._num_threads=128.html As you can see EXT4 shows a small gain over EXT3, ~20%. If you really want high performance it's time to move to XFS, properly configured to match the underlying RAID characteristics of the LUN(s) you're mounting. You'll prefer kernel 2.6.39+, 2.6.36 at minimum, so you get the delayed logging feature (2.6.35 had delayed logging but had problems in other areas). I'll assume with a >10TB mail store that you're seeing greater than 128 concurrent user operations regularly. As you can see from the graph, XFS will give you ~50% greater ops/s than EXT4 and ~90% greater than EXT3--yes, almost double that of EXT3. As the concurrency increases, so will this performance gap, as XFS was designed from day 1 for high concurrency workloads. This is a simulated mail server benchmark. However you should see similar gains with Dovecot. The XFS delayed logging feature will dramatically reduce the number of physical IOs required for journal writes (i.e. metadata IO), as will delayed allocation, a feature of XFS since its inception in 1994. EXT4 was the first of its lineage to gain delayed allocation, some 10+ years later, after Ted T'so studied the XFS code. In short, if you want an 'enterprise caliber' production Linux filesystem tailor made for high IO concurrency, XFS is it. JFS yields similar performance, but hasn't been actively developed for 8 years or so. XFS has substantial ongoing feature and fix development. > *) Is it faster because of the ext4 kernel-module (which can be used on ext3 > to) or because of the ext4 filesystem layout? AIUI, the bulk of the EXT4 performance advantage over EXT3 is the delayed allocation logic. The new EXT4 extent based on disk layout yields little in the way of additional performance, but much in free space management, fragmentation mitigation, etc. > *) Is mdbox really faster? I'd like to have mdbox to have better performance > in running my backup-processes. But does it bring some performance boosts > to? mdbox will substantially decrease physical IOs to your storage back end due to dramatically less metadata operations compared to maildir. You've stated you currently have a storage IOPS bottleneck, so I'd have to assume mdbox will seriously increase your overall performance. Good old mbox will do so as well, but everyone shuns it for various reasons, some valid, some not so valid. If you have an appropriate LUN available (sufficient size and spindle speed/count of member disks), properly create an XFS filesystem on it (read much before creating it), and moved to mdbox atop that, I think you'll be really surprised by how much you gain from simply changing filesystems and mailbox storage formats. If you double the size of the LUN you could potentially carry twice as many users with, fewer IOPS than you're seeing now, on essentially the same hardware platform. -- Stan From ghandidrivesahumvee at rocketfish.com Wed Nov 9 09:40:14 2011 From: ghandidrivesahumvee at rocketfish.com (Dovecot-GDH) Date: Tue, 8 Nov 2011 23:40:14 -0800 Subject: [Dovecot] Multiple Patitions with with mdbox In-Reply-To: <201111081519.05867.p.heinlein@heinlein-support.de> References: <201111081519.05867.p.heinlein@heinlein-support.de> Message-ID: > How do you handle >> 10 TB mailstore? ZFS: no need to fsck. GlusterFS: "always-online". On Nov 8, 2011, at 6:19 AM, Peer Heinlein wrote: > > Having > 10 TByte mailstore filesystem-checks takes too much time. > > At the moment we have four different partitions, but I don't like to set > symlinks or LDAP-flags to sort customers and their domains to there > individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate > the path automatically. > > How do you handle >> 10 TB mailstore? > > I'm very interested in the feature "alternative mailstore" with mdbox, > because that makes it very easy to use at least TWO filesystems without any > tricky configuration. > > I think I'd love to have alternative mailstores. Why does dbox doesn't > look for its m.*-files in more then two directorys? Sure, looking in 4 > directorys would lead to four disc operations, but maybe it could be very > helpful. > > Peer > -- > > Heinlein Professional Linux Support GmbH > Linux: Akademie - Support - Hosting > http://www.heinlein-support.de > > Tel: 030/405051-42 > Fax: 030/405051-19 > > Zwangsangaben lt. ?35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From thomas at koch.ro Wed Nov 9 11:29:37 2011 From: thomas at koch.ro (Thomas Koch) Date: Wed, 9 Nov 2011 10:29:37 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT Message-ID: <201111091029.37963.thomas@koch.ro> Hi, somebody from the kolab groupware project recently explained me that the Bus- Factor[1] of around 1 would be one of their primary reasons not to use Dovecot and stick with Cyrus. What do you think about that? Is the bus factor much higher then 1? [1] http://en.wikipedia.org/wiki/Bus_factor Somehow related: Since the free software world[2] slowly converges towards GIT as the "one and only" VCS, have you thought about a switch? It's much more likely that somebody checks out your code, looks into it and provides patches if the VCS is already installed. [2] Android, Debian, Drupal, Eclipse, Fedora, Gnome, KDE, Linux Kernel, Perl, PHP (ongoing), PostgreSQL, Qt, Ruby on Rails, X.org (freedesktop.org) Best regards, Thomas Koch, http://www.koch.ro From sandro.tosi at register.it Wed Nov 9 11:45:12 2011 From: sandro.tosi at register.it (Sandro Tosi) Date: Wed, 09 Nov 2011 10:45:12 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <4EBA4BA8.1080403@register.it> On 11/09/2011 10:29 AM, Thomas Koch wrote: > Somehow related: Since the free software world[2] slowly converges towards GIT > as the "one and only" VCS, have you thought about a switch? It's much more > likely that somebody checks out your code, looks into it and provides patches > if the VCS is already installed. > > [2] Android, Debian, Drupal, Eclipse, Fedora, Gnome, KDE, Linux Kernel, Perl, > PHP (ongoing), PostgreSQL, Qt, Ruby on Rails, X.org (freedesktop.org) I don't know where you inferred that, but for sure Debian is not converging to git; we have that VCS as many others and none is the preferred/superior. Please try to balance what you say with actual facts: backing your reasoning with partial data is misleading for others not knowing the env you're talking about. Regards, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From p.heinlein at heinlein-support.de Wed Nov 9 11:47:04 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Wed, 9 Nov 2011 10:47:04 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT Message-ID: <201111091047.04570.p.heinlein@heinlein-support.de> Am Mittwoch, 9. November 2011, 10:29:37 schrieb Thomas Koch: Hi, > somebody from the kolab groupware project recently explained me that the > Bus- Factor[1] of around 1 would be one of their primary reasons not to > use Dovecot and stick with Cyrus. The problem is: Even it Timo will be hitten by a bus, everybody could stay on the actual Dovecot-version and everything's fine. Cyrus will IMO never reach the level, that Dovecot already has. :-) Peer P.S.: Timo! Don't leave your house. It's too dangerous for you. -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From david.ocana at bsc.es Wed Nov 9 11:54:28 2011 From: david.ocana at bsc.es (David Ocana) Date: Wed, 09 Nov 2011 10:54:28 +0100 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox Message-ID: <4EBA4DD4.4080505@bsc.es> Hello, I've been trying to set up dovecot 2.0.15, everything seems to work pretty well except for the quota feature. I would like to set a quota limit only for the Inbox folder. I configured two namespaces, according to some posts from Timo Sirainen (http://dovecot.org/list/dovecot/2006-July/014530.html) & (http://www.dovecot.org/list/dovecot/2011-January/056131.html) but I can't configure the quota plugin to act the way I pretend. Here's my namespace configuration: namespace { separator = / prefix = INBOX/ location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n inbox = yes hidden = yes } namespace { separator = / prefix = inbox = no location = mbox:/mail/%d/MAILBOXES/%n:INDEX=/var/dovecot/%d/%n } Quota config: plugin { quota = dirsize:User quota # I've tried with: quota_rule = INBOX:storage=819200K quota_rule = INBOX/*:storage=819200K quota_rule = INBOX/Inbox:storage=819200K # Works with: quota_rule = *:storage=819200K quota_rule = ?:storage=819200K (Same behavior as '*') } Using '*' and '?', is the only way I get it to work, but obviously, quota is calculated summing up all the mailbox folders. I don't know what else to change, any ideas? Is the two namespace configuration correct? Thanks in advance. David -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4439 bytes Desc: S/MIME Cryptographic Signature URL: From thomas at koch.ro Wed Nov 9 12:05:42 2011 From: thomas at koch.ro (Thomas Koch) Date: Wed, 9 Nov 2011 11:05:42 +0100 Subject: [Dovecot] GIT in Debian In-Reply-To: <4EBA4BA8.1080403@register.it> References: <201111091029.37963.thomas@koch.ro> <4EBA4BA8.1080403@register.it> Message-ID: <201111091105.42746.thomas@koch.ro> Hi Sandro, I've been asking myself whether I should include Debian in the list. You're right that there are also other VCS' used in Debian. Zack's statistics say: arch 22 bzr 271 cvs 31 darcs 382 git 5230 hg 63 mtn 13 svn 4843 http://upsilon.cc/~zack/stuff/vcs-usage/ The only open question is, towards which DVCS system the 4843 subversion packages will eventually migrate. I'd bet several rounds of beer on GIT. I believe that the recent migration of 2110 packages from the perl team from svn to git is not yet fully represented in the above numbers. But we shouldn't annoy the dovecot list with Debian details any longer. Regards, Thomas Koch, http://www.koch.ro From adrian.minta at gmail.com Wed Nov 9 14:59:17 2011 From: adrian.minta at gmail.com (Adrian M) Date: Wed, 9 Nov 2011 14:59:17 +0200 Subject: [Dovecot] Quota BUG ? Message-ID: Hello list, I believe I found a bug with quota. The dovecot quota replies with incorrect values when the mailbox is above 2GB. du -ks returns 2005764bytes dovecot IMAP on the same user/directory: 1 getquotaroot inbox * QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1096 2000000)* QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1096 2000000) 1 OK Getquotaroot completed. Is there any way to fix this ? --- linux system: dovecot 1.2.15-7 linux 2.6.32-5-amd64 dovecot.conf: dict { quotadict= mysql:/etc/dovecot/dovecot-dict-quota.conf } plugin { quota = dict:user::proxy::quotadict } dovecot-dict-quota.conf : user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' From maria_arrea at gmx.com Wed Nov 9 15:19:54 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Wed, 09 Nov 2011 14:19:54 +0100 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs Message-ID: <20111109131954.26480@gmx.com> Hello We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy cheap sata-based iscsi storage / sata-NFS. We want to evaluate how much space would be needed in alternate storage BEFORE buying it. I have been reading about doveadm altmove. We want to move ALL messages older than a year to alternate storage, what doveadm-script-fu could I do to identify how much space would go from a user to alternate storage? Regards Maria From tss at iki.fi Wed Nov 9 15:53:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 15:53:08 +0200 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <1320846789.21919.361.camel@hurina> On Wed, 2011-11-09 at 10:29 +0100, Thomas Koch wrote: > somebody from the kolab groupware project recently explained me that the Bus- > Factor[1] of around 1 would be one of their primary reasons not to use Dovecot > and stick with Cyrus. > > What do you think about that? Is the bus factor much higher then 1? I've created a company for Dovecot support, and if all goes well we should have at least one other coder in not too distant future (anyone want a job? :) Also besides me there's already at least Stephan Bosch who has written Dovecot's Sieve/ManageSieve implementations. I don't know if he'd take care of the whole Dovecot if I happened to die right now, but at least he knows the code pretty well. There are also a few big companies that have some people who have done some Dovecot coding. Also the Bus-Factor of Cyrus doesn't seem to be much higher than 1 to me. AFAIK there's only a single person currently developing it actively (plus I guess a few more not-very-active developers from CMU). > Somehow related: Since the free software world[2] slowly converges towards GIT > as the "one and only" VCS, have you thought about a switch? It's much more > likely that somebody checks out your code, looks into it and provides patches > if the VCS is already installed. I'm not as much against git anymore as I was when I switched to hg, but I don't see much benefits in switching to git either. I highly doubt I'd get even a single patch more if I used git instead of hg. The biggest problem with lack of patches is that few people are interested in coding a mail server. You can see the same with all open source IMAP/SMTP servers (and probably commercial ones too). Nearly always there's only a single guy who has written almost all of it. From achekalin at lazurit.com Wed Nov 9 15:57:11 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 09 Nov 2011 16:57:11 +0300 Subject: [Dovecot] Please advise on very fast search Message-ID: <4EBA86B7.2080201@lazurit.com> Hello, I try to create some kind of mail backup system. What I need is system that will store mail for the whole domain, and allow me to restore messages from/to specified email at that domain. The scheme is pretty simple: on our main mail server the SMTP server itself has a rule to send a copy of every message to 'backup at backupserver.host', and the backupserver.host domain is placed nearby on second server. The SMTP on second server do simple 'catchall' redirect of all messages to the single box. There is also a Dovecot that takes care for remote IMAP access to that box. And, finally, I've create some scripts to sort all messages in INBOX to folders named after message's date. So I have a lot of mailboxes inside the catchall box: INBOX 2011.11.03 2011.11.04 2011.11.05 2011.11.06 ...etc... and each folder holds messages for that day. Simply, and works perfectly. The problem is that when my archive become big (several years), it appears to be painful to find specified message(s). When someone suddenly needs to find his/her old message, it is mostly guesses like 'I think the message was between june and july of 2009, or maybe month or two before that', so I need to search all mailboxes (with 1000's messages in each). And it takes really long time. I tried to play with Dovecot indexes, but it won't help too much. The bad part is that I need to search for all emails in each message headers, not only for "From" or "To", since some messages are sent to maillists soe "To" = list address, not person's personal email. Then I tried to index messages on my own, storing info on emails into MySQL database ('email' -> 'mailbox', 'message filename'), but soon I find out that message files can be renamed by Dovecot. Could you please advice me how to speed up message search? Sorry for such a long question, hope you can help! Yours, Alexander Chekalin From tss at iki.fi Wed Nov 9 15:58:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 15:58:42 +0200 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs In-Reply-To: <20111109131954.26480@gmx.com> References: <20111109131954.26480@gmx.com> Message-ID: <1320847122.21919.365.camel@hurina> On Wed, 2011-11-09 at 14:19 +0100, Maria Arrea wrote: > We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy > cheap sata-based iscsi storage / sata-NFS. We want to evaluate how > much space would be needed in alternate storage BEFORE buying it. I > have been reading about doveadm altmove. We want to move ALL messages > older than a year to alternate storage, what doveadm-script-fu could I > do to identify how much space would go from a user to alternate > storage? doveadm -A -f tab fetch size.physical before 1y would print all the message sizes. You could then sum them up with some scripting to produce the total. Unfortunately they are the uncompressed sizes, but maybe simply dividing by 2 would give approximate size? :) (You could get the compressed sizes from dovecot.map.index files with doveadm dump and mapping their contents to the older than 1 year message UIDs, but this gets difficult.) From robert at schetterer.org Wed Nov 9 16:14:21 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 09 Nov 2011 15:14:21 +0100 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBA86B7.2080201@lazurit.com> References: <4EBA86B7.2080201@lazurit.com> Message-ID: <4EBA8ABD.8010804@schetterer.org> Am 09.11.2011 14:57, schrieb Alexander Chekalin: > Hello, > > I try to create some kind of mail backup system. What I need is system > that will store mail for the whole domain, and allow me to restore > messages from/to specified email at that domain. > > The scheme is pretty simple: on our main mail server the SMTP server > itself has a rule to send a copy of every message to > 'backup at backupserver.host', and the backupserver.host domain is placed > nearby on second server. > > The SMTP on second server do simple 'catchall' redirect of all messages > to the single box. There is also a Dovecot that takes care for remote > IMAP access to that box. And, finally, I've create some scripts to sort > all messages in INBOX to folders named after message's date. > > So I have a lot of mailboxes inside the catchall box: > INBOX > 2011.11.03 > 2011.11.04 > 2011.11.05 > 2011.11.06 > ...etc... > > and each folder holds messages for that day. Simply, and works perfectly. > > The problem is that when my archive become big (several years), it > appears to be painful to find specified message(s). When someone > suddenly needs to find his/her old message, it is mostly guesses like 'I > think the message was between june and july of 2009, or maybe month or > two before that', so I need to search all mailboxes (with 1000's > messages in each). And it takes really long time. > > > I tried to play with Dovecot indexes, but it won't help too much. The > bad part is that I need to search for all emails in each message > headers, not only for "From" or "To", since some messages are sent to > maillists soe "To" = list address, not person's personal email. > > Then I tried to index messages on my own, storing info on emails into > MySQL database ('email' -> 'mailbox', 'message filename'), but soon I > find out that message files can be renamed by Dovecot. > > Could you please advice me how to speed up message search? > > > Sorry for such a long question, hope you can help! > > Yours, > Alexander Chekalin > guess youre searching over imap ? perhaps compression will help for speed up, and many other speed related stuff, or you need some other idea of indexing at last if its maildir how fast is "grep" etc...and so on some ideas here http://wiki.dovecot.org/HowTo/ReadOnlyArchive etc anyway , i think you really need another kind of archive solution in Germany there is a law that you need to archive some kind of business mails up to 10 years for finance and other review, so there are a lot of "you can by" solutions now, these have solved the problems you discovered ( indexing etc ) i was shown i.e http://www.bytstormail.de which looked fine to me or perhaps you might have a look http://www.archiveopteryx.org/ here too -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From maria_arrea at gmx.com Wed Nov 9 16:24:14 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Wed, 09 Nov 2011 15:24:14 +0100 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs Message-ID: <20111109142414.26470@gmx.com> Timo, maybe you have mispelled the command? > We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy > cheap sata-based iscsi storage / sata-NFS. We want to evaluate how > much space would be needed in alternate storage BEFORE buying it. I > have been reading about doveadm altmove. We want to move ALL messages > older than a year to alternate storage, what doveadm-script-fu could I > do to identify how much space would go from a user to alternate > storage? doveadm -A -f tab fetch size.physical before 1y would print all the message sizes. You could then sum them up with some scripting to produce the total. Unfortunately they are the uncompressed sizes, but maybe simply dividing by 2 would give approximate size? :) (You could get the compressed sizes from dovecot.map.index files with doveadm dump and mapping their contents to the older than 1 year message UIDs, but this gets difficult.) From tss at iki.fi Wed Nov 9 17:15:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 17:15:23 +0200 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs In-Reply-To: <20111109142414.26470@gmx.com> References: <20111109142414.26470@gmx.com> Message-ID: <1320851723.21919.367.camel@hurina> On Wed, 2011-11-09 at 15:24 +0100, Maria Arrea wrote: > Timo, maybe you have mispelled the command? Minor changes, I tested that this actually works: doveadm -f tab fetch -A size.physical before 365d The -A parameter requires that you use a userdb that supports iteration. With SQL you need to provide the iteration_query. From tss at iki.fi Wed Nov 9 17:17:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 17:17:31 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBA86B7.2080201@lazurit.com> References: <4EBA86B7.2080201@lazurit.com> Message-ID: <1320851851.21919.369.camel@hurina> On Wed, 2011-11-09 at 16:57 +0300, Alexander Chekalin wrote: > The problem is that when my archive become big (several years), it > appears to be painful to find specified message(s). When someone > suddenly needs to find his/her old message, it is mostly guesses like 'I > think the message was between june and july of 2009, or maybe month or > two before that', so I need to search all mailboxes (with 1000's > messages in each). And it takes really long time. > > > I tried to play with Dovecot indexes, but it won't help too much. They'll help with the dates. > The > bad part is that I need to search for all emails in each message > headers, not only for "From" or "To", since some messages are sent to > maillists soe "To" = list address, not person's personal email. Headers only, not message body? Anyway, some of the full text search backends would support searching from both. I'd recommend using either Solr or with Dovecot v2.1 you can also use Lucene: http://wiki2.dovecot.org/Plugins/FTS From odhiambo at gmail.com Wed Nov 9 17:34:45 2011 From: odhiambo at gmail.com (Odhiambo Washington) Date: Wed, 9 Nov 2011 18:34:45 +0300 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320791738.21919.350.camel@hurina> References: <1320791738.21919.350.camel@hurina> Message-ID: On Wed, Nov 9, 2011 at 01:35, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig > > Here's the first beta release of Dovecot v2.1. This version has already > been tested quite a lot, so I'm not expecting any major bugs. So please > upgrade and see if you can find any problems. I'm optimistic about > getting rc1 released this year and perhaps even v2.1.0. > > Since alpha2 there have been a lot of fixes, especially to imapc > backend. There are probably also some small new features, but nothing > huge. > > As a reminder, here's the largest changes since v2.0: > > * Plugins now use UTF-8 mailbox names rather than mUTF-7: > acl, autocreate, expire, trash, virtual > * auth_username_format default changed to %Lu. If you really want > case sensitive usernames, set it back to empty. > * Solr full text search backend changed to use mailbox GUIDs > instead of > mailbox names, requiring reindexing everything. solr_old backend > can > be used with old indexes to avoid reindexing, but it doesn't > support > some newer features. > > + imapc (= IMAP client) storage allows using a remote IMAP server to > be used as storage. This allows using Dovecot as a smart (caching) > proxy or using dsync to do migration from remote IMAP server. > + Mailbox indexing via queuing indexer service (required for Lucene) > + Lucene full text search (FTS) backend rewritten with support for > different languages > + FTS finally supports "OR" search operation > + FTS supports indexing attachments via external programs > + IMAP FUZZY extension, supported by Lucene and Solr FTS backends > + Mailbox list indexes > + Statistics tracking via stats service. Exported via doveadm stats. > + Autocreate plugin creates/subscribes mailboxes physically only > when > the mailbox is opened for the first time. Mailbox listing shows > the > autocreated mailboxes even if they don't physically exist. > + Password and user databases now support default_fields and > override_fields settings to specify template defaults/overrides. > - listescape plugin works perfectly now > > I have been running the alpha for some time now. I have today tried compiling this -beta, but I saw so many warning about linking libraries, which I ignored, but it did fail with: Making all in wiki make: don't know how to make 2b. Stop *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1/doc. *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1. *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From achekalin at lazurit.com Wed Nov 9 18:16:39 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 09 Nov 2011 19:16:39 +0300 Subject: [Dovecot] Please advise on very fast search Message-ID: <4EBAA767.6060609@lazurit.com> Thanks, Robert, will take a look at. What I'm afraid for is how database storage should be planned (storage, CPU, RAM, scaling when will be over-filled). When dealing with files (I'm using maildir), it is much easy to understand and to fix just about everything. Adding database involves tune it up too, and I'll have more points of 'tune it a bit' In fact work with Dovecot is pretty nice, but I think I can tune it to work faster. I now run it on FreeBSD (on UFS2), maybe I should change OS + FS, but need to test (really hope ZFS disks on SAS drives will help; still find no benchmarks on such a setup). Will also try to use full text search, but afraid of index size (and I need no search on body, just on headers). Anyway thank your for pointing me in right directions! Yours, Alexander From tss at iki.fi Wed Nov 9 18:37:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 18:37:14 +0200 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: References: <1320791738.21919.350.camel@hurina> Message-ID: <1320856634.21919.371.camel@hurina> On Wed, 2011-11-09 at 18:34 +0300, Odhiambo Washington wrote: > I have today tried compiling this -beta, but I saw so many warning about > linking libraries, which I ignored, Probably plugin warnings which you can safely ignore. > Making all in wiki > make: don't know how to make 2b. Stop Looks like there's some weird file went into wiki docs, which some makes don't like. You could just find the line containing "2b" from doc/wiki/Makefile and delete it. From tss at iki.fi Wed Nov 9 18:40:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 18:40:04 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBAA767.6060609@lazurit.com> References: <4EBAA767.6060609@lazurit.com> Message-ID: <1320856804.21919.372.camel@hurina> On Wed, 2011-11-09 at 19:16 +0300, Alexander Chekalin wrote: > Will also try to use full text search, > but afraid of index size (and I need no search on body, just on headers). It wouldn't be difficult to patch Dovecot to skip indexing message bodies. Of course then you'd need to remember to keep applying the patch when updating. From tss at iki.fi Wed Nov 9 19:56:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 19:56:18 +0200 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <4EBA4DD4.4080505@bsc.es> References: <4EBA4DD4.4080505@bsc.es> Message-ID: <1320861378.21919.376.camel@hurina> On Wed, 2011-11-09 at 10:54 +0100, David Ocana wrote: > I've been trying to set up dovecot 2.0.15, everything seems to work > pretty well except for the quota feature. I would like to set a quota > limit only for the Inbox folder. I configured two namespaces, > according to some posts from Timo Sirainen > > namespace { > separator = / > prefix = INBOX/ > location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n > inbox = yes > hidden = yes > } > > plugin { > quota = dirsize:User quota quota = dirsize:User quota:ns=INBOX/ This limits the quota only to mailboxes in INBOX/ namespace. > # I've tried with: > quota_rule = INBOX:storage=819200K > quota_rule = INBOX/*:storage=819200K > quota_rule = INBOX/Inbox:storage=819200K Quota rules don't work in this way. There are no per-mailbox quotas really, at least in the way you're thinking about. From tss at iki.fi Wed Nov 9 19:59:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 19:59:42 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: References: Message-ID: <1320861585.21919.378.camel@hurina> On Wed, 2011-11-09 at 14:59 +0200, Adrian M wrote: > Hello list, > I believe I found a bug with quota. > The dovecot quota replies with incorrect values when the mailbox is above 2GB. > > du -ks returns 2005764bytes > > dovecot IMAP on the same user/directory: > 1 getquotaroot inbox > * QUOTAROOT "INBOX" "user" > * QUOTA "user" (STORAGE 1096 2000000)* QUOTAROOT "INBOX" "user" > * QUOTA "user" (STORAGE 1096 2000000) > 1 OK Getquotaroot completed. So it wraps. > dovecot.conf: > dict { > quotadict= mysql:/etc/dovecot/dovecot-dict-quota.conf > } > plugin { > quota = dict:user::proxy::quotadict > } > > dovecot-dict-quota.conf : > user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as > mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS > quota_rule FROM mailbox WHERE username = '%u' AND active = '1' This is dovecot-sql.conf, not dovecot-dict-quota.conf (or alternatively you've named them in a "wrong" way). Anyway, here's a guess: the quota is stored in mysql in "integer", which has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also use "integer". Replace that with "bigint" and recalculate quota and it should work. From adrian.minta at gmail.com Wed Nov 9 20:35:42 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 20:35:42 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320861585.21919.378.camel@hurina> References: <1320861585.21919.378.camel@hurina> Message-ID: <4EBAC7FE.5060207@gmail.com> On 11/09/11 19:59, Timo Sirainen wrote: > Anyway, here's a guess: the quota is stored in mysql in "integer", which > has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also > use "integer". Replace that with "bigint" and recalculate quota and it > should work. > Actually the mysql part is OK: quota | bigint(20) As you can see the value readed from mysql ( 2000000bytes) is also OK: * QUOTA "user" (STORAGE 1096 2000000) The problem is the 1096 number calculated internally by dovecot. By tweaking the SQL querry I coud put any value instead of 2000000, but I was unable to change 1096 part. -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From tss at iki.fi Wed Nov 9 20:41:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 20:41:23 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <4EBAC7FE.5060207@gmail.com> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> Message-ID: <1320864083.21919.379.camel@hurina> On Wed, 2011-11-09 at 20:35 +0200, Adrian Minta wrote: > On 11/09/11 19:59, Timo Sirainen wrote: > > Anyway, here's a guess: the quota is stored in mysql in "integer", which > > has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also > > use "integer". Replace that with "bigint" and recalculate quota and it > > should work. > > > Actually the mysql part is OK: > quota | bigint(20) > > As you can see the value readed from mysql ( 2000000bytes) is also OK: 2000000 kilobytes is the quota limit. > * QUOTA "user" (STORAGE 1096 2000000) > > The problem is the 1096 number calculated internally by dovecot. By > tweaking the SQL querry I coud put any value instead of 2000000, but I > was unable to change 1096 part. Is the quota in the database 1096 or >2GB? i.e. is the problem with reading it, or updating it? In general the quota code uses 64bit integers everywhere so this shouldn't be happening. From adrian.minta at gmail.com Wed Nov 9 20:44:31 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 20:44:31 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320864083.21919.379.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> Message-ID: <4EBACA0F.1010200@gmail.com> > Is the quota in the database 1096 or>2GB? i.e. is the problem with > reading it, or updating it? In general the quota code uses 64bit > integers everywhere so this shouldn't be happening. > > In the database quota field is 2048000000. -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From tss at iki.fi Wed Nov 9 20:46:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 20:46:02 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <4EBACA0F.1010200@gmail.com> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> Message-ID: <1320864362.21919.380.camel@hurina> On Wed, 2011-11-09 at 20:44 +0200, Adrian Minta wrote: > > Is the quota in the database 1096 or>2GB? i.e. is the problem with > > reading it, or updating it? In general the quota code uses 64bit > > integers everywhere so this shouldn't be happening. > > > > > In the database quota field is 2048000000. In the dict quota value? That looks more like the quota limit, not the current quota usage. From toddr at cpanel.net Wed Nov 9 20:57:39 2011 From: toddr at cpanel.net (Todd Rinaldo) Date: Wed, 9 Nov 2011 12:57:39 -0600 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <1320846789.21919.361.camel@hurina> References: <201111091029.37963.thomas@koch.ro> <1320846789.21919.361.camel@hurina> Message-ID: <629183B4-622D-4058-B706-D5FFF91FC366@cpanel.net> On Nov 9, 2011, at 7:53 AM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 10:29 +0100, Thomas Koch wrote: > >> Somehow related: Since the free software world[2] slowly converges towards GIT >> as the "one and only" VCS, have you thought about a switch? It's much more >> likely that somebody checks out your code, looks into it and provides patches >> if the VCS is already installed. > > I'm not as much against git anymore as I was when I switched to hg, but > I don't see much benefits in switching to git either. I highly doubt I'd > get even a single patch more if I used git instead of hg. > Hg / Git. Either seems fine to me. My big concern would be that the source history is in multiple accessible places around the world. This is the biggest limitation of SVN to my mind. Could I suggest a google code clone as an additional VCS backup location? From adrian.minta at gmail.com Wed Nov 9 21:01:43 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 21:01:43 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320864362.21919.380.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> <1320864362.21919.380.camel@hurina> Message-ID: <4EBACE17.4070508@gmail.com> On 11/09/11 20:46, Timo Sirainen wrote: > On Wed, 2011-11-09 at 20:44 +0200, Adrian Minta wrote: >>> Is the quota in the database 1096 or>2GB? i.e. is the problem with >>> reading it, or updating it? In general the quota code uses 64bit >>> integers everywhere so this shouldn't be happening. >>> >>> >> In the database quota field is 2048000000. > In the dict quota value? That looks more like the quota limit, not the > current quota usage. In the mailbox table. The 2000000 value commes from: user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' This value is easy to change by modifying the above mysql query, but the 1096 value I don't know how to change. Maybe to "instruct" dovecot to work internal with Kilobytes ? For instance by altering the query like this: user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', quota, 'K') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' ... I get ... * QUOTA "user" (STORAGE 1096 2048000000) -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From mrvjtod at gmail.com Wed Nov 9 21:04:12 2011 From: mrvjtod at gmail.com (Chris Young) Date: Wed, 9 Nov 2011 14:04:12 -0500 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: References: Message-ID: I am familiar with this article http://wiki.dovecot.org/LDA#multipleuids I want to disable setuid-root, but I don't know the default group and permissions. sudo chgrp root /usr/libexec/dovecot/dovecot-lda sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda but my delivery is still failing Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) Thoughts? Thanks Chris From e-frog at gmx.de Wed Nov 9 21:08:48 2011 From: e-frog at gmx.de (e-frog) Date: Wed, 09 Nov 2011 20:08:48 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <4EB6AFE3.3020001@gmx.de> References: <4EB6AFE3.3020001@gmx.de> Message-ID: <4EBACFC0.6060703@gmx.de> On 06.11.2011 17:03, wrote e-frog: > 1 lsub "" "virtual/*" > * LSUB () "/" "virtual" > * LSUB () "/" "virtual/Lists" > * LSUB () "/" "virtual/unread" > 1 OK Lsub completed. Just tested this again with 2.1beta1 with the same result. From apm at one.com Wed Nov 9 21:25:57 2011 From: apm at one.com (Peter Mogensen) Date: Wed, 09 Nov 2011 20:25:57 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EBAD3C5.7030006@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: >> Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted >> transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: >> indexid changed 1320419300 -> 1320419441 (sync_offset=0) > > Session A had TestMails open and created with index file whose ID was > 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 > 17:08:20 EET 2011). > > Session B came and recreated the index files 141 seconds later with ID > 1320419441. Either it didn't see A's original index files for some > reason or it simply decided to recreate them for some reason. Either way > this shouldn't have happened. Turns out this is expected to confuse Session A. The client in question sometimes start the session (B) with this command sequence: DELETE folder CREATE folder APPEND... Any Session A having opened "folder" of course would be surprised that there's a new index file (makes we wish for an IMAP5 where folderID!=displayname) This can be reproduced by hand speaking IMAP with two telnets. Only question left is, why does Dovecot end the log sequence by saying: "Disconnected: IMAP session state is inconsistent, please relogin." ... when it is capable of detecting this and returning "BYE folder deleted under us" and logging the same. /Peter From rpalmarin at yahoo.com Wed Nov 9 18:57:44 2011 From: rpalmarin at yahoo.com (rpalmarin) Date: Wed, 9 Nov 2011 16:57:44 +0000 (UTC) Subject: [Dovecot] LDAP expired password References: <116877.53915.qm@web29503.mail.ird.yahoo.com> <57o6m4h3miv8@mids.svenhartge.de> <4D95A998.9080701@noa.gr> <97o72ha3miv8@mids.svenhartge.de> Message-ID: Sven Hartge svenhartge.de> writes: > > Nikolaos Milas noa.gr> wrote: > > On 1/4/2011 11:09 ??, Sven Hartge wrote: > > >> Have a look at the ppolicy slapd.overlay. This will solve your > >> problem. hello all Sorry for the delay in the response I checked the ppolicy overlay but without success. This overlay does not have a single "password expired" attribute to put in the user_filter. currently my dovecot-ldap.conf contain: user_filter (&(|( = uid =% u) (employeeNumber =% u)) (objectClass = inetOrgPerson) (accountStatus = active)) > At my university we introduced our own attribute gifb-status which > contains a "1" if an account is valid, a "0" if it is not (and several > others for different purposes) and our ldap-filters all contain > something like "(&(ou=foobar)(gifb-status=1))". is possible that the only way to do this is to manage a new attribute? how can understand all the people that have configured the mail client to authenticate with imap-dovecot that their passoword has expired? thanks / roby From user+dovecot at localhost.localdomain.org Wed Nov 9 23:41:45 2011 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Wed, 09 Nov 2011 22:41:45 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <4EBAF399.7090706@localhost.localdomain.org> On 11/09/2011 10:29 AM Thomas Koch wrote: > Hi, > > somebody from the kolab groupware project recently explained me that the Bus- > Factor[1] of around 1 would be one of their primary reasons not to use Dovecot > and stick with Cyrus. And what is their primary reason for the lame excuse? Too many cooks spoil the broth. just my 2? Regards, Pascal -- The trapper recommends today: fabaceae.1131322 at localdomain.org From pch0317 at gmail.com Thu Nov 10 00:33:59 2011 From: pch0317 at gmail.com (pch0317) Date: Wed, 09 Nov 2011 22:33:59 +0000 Subject: [Dovecot] Can't delete sub-folder Message-ID: <4EBAFFD7.8020201@gmail.com> Hi list, I have problem with deleting sub-folder. I use thunderbird 8.0 as a client and dovecot 2.0.9 with imap protocol as a server. I use dovecot with tb-extra-mailbox-sep parameter. I use mbox mailboxes and thunderbird with *not checked* option 'server supports folders that contain sub-folders and messages' so when I create sub-folder I can choose between 'folders only' and 'messages only'. When I try to delete 'parent' folder which contain 'messages only' folder, error message appear: [ALREADYEXISTS] Mailbox has children, delete them first. Ok, so I delete 'message only' folder first and try to delete parent folder, but the same error appear: [ALREADYEXISTS] Mailbox has children, delete them first. When I list mailboxes from console I see inside 'parent' folder ".imap" folder. When I delete .imap folder from console and then delete parent folder from thunderbird, parent folder is deleted correctly. Which application create this ".imap" hidden folder and how solve this problem. Thanks From stan at hardwarefreak.com Thu Nov 10 02:42:33 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 18:42:33 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBAA767.6060609@lazurit.com> References: <4EBAA767.6060609@lazurit.com> Message-ID: <4EBB1DF9.4040208@hardwarefreak.com> On 11/9/2011 10:16 AM, Alexander Chekalin wrote: > Thanks, Robert, > > will take a look at. > > What I'm afraid for is how database storage should be planned (storage, > CPU, RAM, scaling when will be over-filled). When dealing with files > (I'm using maildir) Bingo. ^^^ Maildir is very likely a hug factor in your current slow search time. With a maildir search, every mail file must be opened and searched. How many total mail files are opened for each of your searches? Thousands? Tens of thousands? Maildir causes a massive disk IO bottleneck when searching so many files. Run iostat the next time you do one of these searches, and look at the %iowait value. It will likely be very high. If it is, this confirms maildir is a big part of the problem. mbox, and mdbox, would be many many times faster than maildir WRT searching as the total number of files is lower by orders of magnitude. Switching from maildir to mbox/mdbox shifts the workload burden from the disk subsystem to the processor/memory. And I'm sure as with everyone else on the planet today, you have massive spare CPU cycles, but extremely limited spindle throughput. And as Timo suggested, using one of the indexing search plugins would be much faster yet, as long as you keep the indexes updated. -- Stan From stan at hardwarefreak.com Thu Nov 10 02:50:07 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 18:50:07 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <1320856804.21919.372.camel@hurina> References: <4EBAA767.6060609@lazurit.com> <1320856804.21919.372.camel@hurina> Message-ID: <4EBB1FBF.9070401@hardwarefreak.com> On 11/9/2011 10:40 AM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 19:16 +0300, Alexander Chekalin wrote: >> Will also try to use full text search, >> but afraid of index size (and I need no search on body, just on headers). > > It wouldn't be difficult to patch Dovecot to skip indexing message > bodies. Of course then you'd need to remember to keep applying the patch > when updating. Also keep in mind that, in general, many/most message headers today are often as large, or larger than, the actual message body, especially for list mail. Just take a look at messages from this for evidence. Thus, I'd think that going out of your way to avoid indexing message bodies wouldn't be worth the effort/headaches involved. -- Stan From scott_the_musician at yahoo.com.au Thu Nov 10 06:09:02 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Wed, 9 Nov 2011 20:09:02 -0800 (PST) Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Message-ID: <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> ----- Forwarded Message ----- From: Scott Lewis To: "dovecot at dovecot.org" Sent: Thursday, 3 November 2011 4:31 PM Subject: patching dovecot for sieve/managesieve support, centos 5.6? Hi all, I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. Here's what happens: [root at mail ~]# whereis dovecot dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot ... checking whether to build static libraries... yes dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH to give path to compiled Dovecot sources or to a directory with the installed dovecot-config file. configure: error: dovecot-config not found -- I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: Avelsieve plugin details: backend = ManageSieve ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? thanks in advance! From achekalin at lazurit.com Thu Nov 10 06:37:05 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Thu, 10 Nov 2011 07:37:05 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB1DF9.4040208@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: Oh, that's the point to consider. But I must confess I'm in love with Maildir for maybe 10 years for that simple fact I can do anything with each and every single message even on disk (=much faster than via IMAP). If I would deal with mbox directly I'd need to parse huge files, brrrr. Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? 10.11.2011, ? 3:42, Stan Hoeppner ???????(?): > On 11/9/2011 10:16 AM, Alexander Chekalin wrote: >> Thanks, Robert, >> >> will take a look at. >> >> What I'm afraid for is how database storage should be planned (storage, >> CPU, RAM, scaling when will be over-filled). When dealing with files >> (I'm using maildir) > > Bingo. ^^^ > > Maildir is very likely a hug factor in your current slow search time. > With a maildir search, every mail file must be opened and searched. How > many total mail files are opened for each of your searches? Thousands? > Tens of thousands? Maildir causes a massive disk IO bottleneck when > searching so many files. Run iostat the next time you do one of these > searches, and look at the %iowait value. It will likely be very high. > If it is, this confirms maildir is a big part of the problem. > > mbox, and mdbox, would be many many times faster than maildir WRT > searching as the total number of files is lower by orders of magnitude. > Switching from maildir to mbox/mdbox shifts the workload burden from > the disk subsystem to the processor/memory. And I'm sure as with > everyone else on the planet today, you have massive spare CPU cycles, > but extremely limited spindle throughput. > > And as Timo suggested, using one of the indexing search plugins would be > much faster yet, as long as you keep the indexes updated. > > -- > Stan From achekalin at lazurit.com Thu Nov 10 07:35:11 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Thu, 10 Nov 2011 08:35:11 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB1DF9.4040208@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EBB628F.8060705@lazurit.com> Hello, Stan, in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway). You see, what I need is not only find message from|to someone on specified date, I also sometime need to restore that message back to user's original box. As far our mailserver and backup-mailserver are different machines, it is a bit tricky to copy messages between it fast enough. Say, if I need to find and restore all mails from user at domain.com within 2009 year, and search yields in some 1000's of messages, then use IMAP to copy it over to another server takes some time - and if you consider both search time and restore/copy time the whole process may take "ages". With maildir I can rsync/scp needed files to another host and that's fast way - that's why I stick with maildir. FTS in my case can help (I can search for user at domain.com, for example), but it also return messages that contains such a string in message body (and that takes index space, too), so I'll need to filter it later, but surely it'll be faster than checking every message in the archive. Yours, Alexander > Maildir is very likely a hug factor in your current slow search time. > With a maildir search, every mail file must be opened and searched. How > many total mail files are opened for each of your searches? Thousands? > Tens of thousands? Maildir causes a massive disk IO bottleneck when > searching so many files. Run iostat the next time you do one of these > searches, and look at the %iowait value. It will likely be very high. > If it is, this confirms maildir is a big part of the problem. > > mbox, and mdbox, would be many many times faster than maildir WRT > searching as the total number of files is lower by orders of magnitude. > Switching from maildir to mbox/mdbox shifts the workload burden from > the disk subsystem to the processor/memory. And I'm sure as with > everyone else on the planet today, you have massive spare CPU cycles, > but extremely limited spindle throughput. > > And as Timo suggested, using one of the indexing search plugins would be > much faster yet, as long as you keep the indexes updated. > -- ? ?????????, ????????? ??????? ??????? ??????????? +7 909 799 2549 achekalin at lazurit.com From research at cruzio.com Thu Nov 10 10:30:31 2011 From: research at cruzio.com (Mark Hanford) Date: Thu, 10 Nov 2011 00:30:31 -0800 Subject: [Dovecot] indexes permissions problem Message-ID: <4EBB8BA7.9070603@cruzio.com> Hey all, I did a search but didn't find the answer to my problem, so here goes. I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box with user home directories NFS mounted), my index files are setup to be in /u/indexes/ On the Dovecot 1.1.1 installation, the perms on the indexes directory is 777 with root:mail ownership. The same thing on the Dovecot 2 / Centos server results in a 'permission denied' error when Dovecot tries to create files. So, I guess my main question is, what perms and ownership should /u/indexes be set to? I've tried several different things before this cry for help... Thanks. Mark From ms at mike2k.de Thu Nov 10 11:19:34 2011 From: ms at mike2k.de (Michael Stilkerich) Date: Thu, 10 Nov 2011 10:19:34 +0100 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320442758.21919.222.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> <1320442758.21919.222.camel@hurina> Message-ID: Hi, On Nov 4, 2011, at 10:39 PM, Timo Sirainen wrote: >> Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking >> failed for file /home/dovecot/isa/dovecot.index.log: No such file or >> directory >> Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with >> file /home/dovecot/isa/dovecot.index.log: No such file or directory > > These simply shouldn't happen. I'd say it's a kernel bug. You're running > a default Ubuntu kernel? I wonder if other Ubuntu users have this > problem. It may be an apparmor issue. I noticed plenty of apparmor log entries on these accesses, though apparmor should only log but not disallow them. I have unloaded the dovecot apparmor profiles and not seen any of these errors since then. I got a new issue, however: I migrated from Maildir to mdbox. Since then, my shared mailboxes don't fully work anymore. I have given another user full rights to a shared mailbox (getacl returns akxeilprwtscd for that folder/user). The user sees the mailbox an can perform some operations including reading and deleting messages on it. If she tries to insert a new message, however, it fails and the error log shows: dovecot: imap(isa): Error: fcntl(write-lock) locking failed for file /home/dovecot/michael/storage/dovecot.map.index.log: Bad file descriptor dovecot: imap(isa): Error: mail_index_wait_lock_fd() failed with file /home/dovecot/michael/storage/dovecot.map.index.log: Bad file descriptor All my mail locations are owned by the respective system user and the mail group, and writeable by both. In particular, I checked that both the storage directory as well as the dovecot.map.index.log are writeable by the mail group. The users are not regular members of the mail group, but my dovecot config contains mail_access_groups = mail Any idea how to resolve this issue? -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4813 bytes Desc: not available URL: From alexdosh at gmail.com Thu Nov 10 11:27:36 2011 From: alexdosh at gmail.com (Alexander KIper) Date: Thu, 10 Nov 2011 13:27:36 +0400 Subject: [Dovecot] Exim thru through Dovecot deliver to spec IMAP-folder Message-ID: Hello All! How can I "post" some mails from Exim trnasport through Dovecot deliver to IMAP MailBox in specific folder (for example: "Junk")? -- With best regards, Alexander KIper! From usenet at schani.com Thu Nov 10 11:40:56 2011 From: usenet at schani.com (wwl) Date: Thu, 10 Nov 2011 10:40:56 +0100 Subject: [Dovecot] Limit the number of pop/imap requests Message-ID: Hello Group, is it possible to Limit the number of pop/imap requests to Dovecot? I read the Wiki and found max_mail_processes = 512 is this the way to limit this ? dovecot --version 1.2.15 Thanx for help Christian From david.ocana at bsc.es Thu Nov 10 13:14:50 2011 From: david.ocana at bsc.es (David Ocana) Date: Thu, 10 Nov 2011 12:14:50 +0100 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <1320861378.21919.376.camel@hurina> References: <4EBA4DD4.4080505@bsc.es> <1320861378.21919.376.camel@hurina> Message-ID: <4EBBB22A.90501@bsc.es> On 09/11/11 18:56, Timo Sirainen wrote: > On Wed, 2011-11-09 at 10:54 +0100, David Ocana wrote: > >> I've been trying to set up dovecot 2.0.15, everything seems to work >> pretty well except for the quota feature. I would like to set a quota >> limit only for the Inbox folder. I configured two namespaces, >> according to some posts from Timo Sirainen >> >> namespace { >> separator = / >> prefix = INBOX/ >> location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n >> inbox = yes >> hidden = yes >> } >> >> plugin { >> quota = dirsize:User quota > > quota = dirsize:User quota:ns=INBOX/ Actually I forgot to mention that I also tried that, but I got the following error: Error: Initialization failed: Failed to initialize quota: Quota root User quota: Unknown parameter for backend dirsize: ns=INBOX/ That's why I was trying to change quota settings by using the quota_rule directive. > > This limits the quota only to mailboxes in INBOX/ namespace. > >> # I've tried with: >> quota_rule = INBOX:storage=819200K >> quota_rule = INBOX/*:storage=819200K >> quota_rule = INBOX/Inbox:storage=819200K > > Quota rules don't work in this way. There are no per-mailbox quotas > really, at least in the way you're thinking about. > I see, I guess they're per-namespace quotas, right? I got the wrong idea after watching the following, which was exactly what I wanted to do :p quota_rule = : May be that, using dirsize backend lets you no other option than calculating quota for the whole user's mailbox? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4439 bytes Desc: S/MIME Cryptographic Signature URL: From stan at hardwarefreak.com Thu Nov 10 14:46:05 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 06:46:05 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EBBC78D.2060809@hardwarefreak.com> On 11/9/2011 10:37 PM, Alexander Chekalin wrote: > Oh, that's the point to consider. > > But I must confess I'm in love with Maildir for maybe 10 years This love affair may be coming to and end. >...for that simple fact I can do anything with each and every single message even on disk (=much faster than via IMAP). If I would deal with mbox directly I'd need to parse huge files, brrrr. Mbox is an excellent mailbox format for archived mail *because of* the fact that searching it is very fast and the disk subsystem overhead is low. For example, on my decade+ old 550MHz x86 SOHO server with only 384MB RAM and a single 7.2k SATA disk, after dropping caches, we'll search my debian-users mbox archive (my largest) for total message count by searching a known header of every message: -rw------- 1 stan stan 133M Nov 10 06:03 1-Debian-Users ~/mail$ time grep -c Content-Length 1-Debian-Users 22817 real 0m1.731s user 0m0.328s sys 0m0.852s Now let's search for posts from me (after dropping caches again): ~/mail$ time grep -c "From: Stan Hoeppner" 1-Debian-Users 536 real 0m1.657s user 0m0.216s sys 0m0.896s Nested greps will obviously take longer, as will those using perl expressions, but this gives some indication of the kind of speed we're talking about: less than seconds to search 22,000+ messages for a specific single header. So that's ~20 seconds for an mbox containing 220K+ messages, again on 10+ year old hardware. > Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? You should probably take a look at Enkive. I'm not sure what mail storage format it uses, and I've not used it personally, so I can't vouch for its speed, but it's pretty complete feature-wise. Take the test drive--nice search interface. http://www.enkive.org/ -- Stan From stan at hardwarefreak.com Thu Nov 10 15:17:42 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 07:17:42 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB628F.8060705@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EBB628F.8060705@lazurit.com> Message-ID: <4EBBCEF6.5040502@hardwarefreak.com> On 11/9/2011 11:35 PM, Alexander Chekalin wrote: > Hello, Stan, > > in fact the only thing I miss even with my current scheme is permanent > ID assigned to the message so I can easily find it despite the IMAP > mailbox it is now (so if someone moved the message from one > mailbox/folder to another, the ID allows to retrieve it fast anyway). > > You see, what I need is not only find message from|to someone on > specified date, I also sometime need to restore that message back to > user's original box. As far our mailserver and backup-mailserver are > different machines, it is a bit tricky to copy messages between it fast > enough. Say, if I need to find and restore all mails from > user at domain.com within 2009 year, and search yields in some 1000's of > messages, then use IMAP to copy it over to another server takes some > time - and if you consider both search time and restore/copy time the > whole process may take "ages". Apparently I didn't fully understand all of your requirements. Moving the archived mail to mbox/mdbox and/or getting a good indexing search engine installed will cut the search time down tremendously. Whether that would make up for the time consumed with an IMAP copy of many emails I don't know. If your servers aren't old and slow, and are not already overloaded, I would think the IMAP message copying over GbE would be pretty quick, even for the 1000 messages scenario. There may be some Dovecot tweaks that might make this copy process faster. Timo would need to chime in on that. Do you perform the IMAP transfers with a GUI IMAP client on your management PC? Or are you using imapsync or some other util directly on the servers? If the former you may be able to tweak your IMAP client to speed up the transfers as well. Try using IMAP and not IMAPS for the transfers. What is the network infrastructure between the servers and your management workstation? Is it all GbE with jumbo frames enabled? > With maildir I can rsync/scp needed files to another host and that's > fast way - that's why I stick with maildir. There is definitely some flexibility here. > FTS in my case can help (I can search for user at domain.com, for example), > but it also return messages that contains such a string in message body > (and that takes index space, too), so I'll need to filter it later, but > surely it'll be faster than checking every message in the archive. Sure. So you're concerned with your poor performance, but also with disk space. Unfortunately there's no free lunch to be had. You'll have to make sacrifices somewhere. You could go with mdbox and use compression, trading that saved space for search index files space. -- Stan From stan at hardwarefreak.com Thu Nov 10 15:19:57 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 07:19:57 -0600 Subject: [Dovecot] Exim thru through Dovecot deliver to spec IMAP-folder In-Reply-To: References: Message-ID: <4EBBCF7D.1080904@hardwarefreak.com> On 11/10/2011 3:27 AM, Alexander KIper wrote: > Hello All! > > How can I "post" some mails from Exim trnasport through Dovecot deliver to > IMAP MailBox in specific folder (for example: "Junk")? Dovecot 1.x: http://wiki.dovecot.org/LDA/Sieve Dovecot 2.x: http://wiki2.dovecot.org/Pigeonhole/Sieve -- Stan From tss at iki.fi Thu Nov 10 17:29:35 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 10 Nov 2011 17:29:35 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: On 10.11.2011, at 6.37, Alexander Chekalin wrote: > Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? See doveadm fetch / doveadm search. > in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway). Dovecot has message GUIDs (with maildir it's filename), but there's no quick lookup for them, even though doveadm can fetch them easily: doveadm fetch text guid 12312312 From adrian.minta at gmail.com Thu Nov 10 18:03:14 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Thu, 10 Nov 2011 18:03:14 +0200 Subject: [Dovecot] Quota BUG - fixed In-Reply-To: <1320864362.21919.380.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> <1320864362.21919.380.camel@hurina> Message-ID: <4EBBF5C2.7020403@gmail.com> After some deep investigations I manage to solve the problem. I was only reading quota in user_querry. Now I read it in user_querry and in password_query and all seems fine: --dovecot-sql.conf--- user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', quota) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' password_query = SELECT username as user, password, '/home/%d/%n' as userdb_home, 'maildir:/home/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid, CONCAT('*:bytes=', quota) as userdb_quota_rule FROM mailbox WHERE username = '%u' AND active = '1' --dovecot.conf--- plugin { quota = dict:user::proxy::quotadict quota_rule2 = Trash:storage=10%% quota_rule3 = SPAM:storage=10%% } the result is fine now: 2 getquotaroot inbox * QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1997999 2000000) 2 OK Getquotaroot completed. Only one "cosmetic" bug remains when an empty mailbox appear as a small negative number in quota2 table, but this is fixable in postfixadmin. -- Best regards, Adrian Minta MA3173-RIPE,www.minta.ro From micah at riseup.net Thu Nov 10 20:59:32 2011 From: micah at riseup.net (Micah Anderson) Date: Thu, 10 Nov 2011 13:59:32 -0500 Subject: [Dovecot] dovecot-lda quota rule Message-ID: <874nybztuz.fsf@algae.riseup.net> I really like the feature where you can define quota rules with percents which trigger off of the default values[0] (so you can set the Trash to allow for 10% more of the user's quota for example). What I would really love in dovecot would be for the ability to configure a quota rule for dovecot-lda. I would like to configure things so we don't bounce emails for users until they are well over quota, the IMAP quota plugin is a really great way to notify people that they are over quota because it fails to write to other folders.... that should be enough to get people's attention that they need to deal with things, but bouncing is harsh. Is there a way to do this now that I haven't seen? thanks! micah 0. http://wiki2.dovecot.org/Quota/Configuration -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From carloswill at gmail.com Thu Nov 10 21:17:15 2011 From: carloswill at gmail.com (Carlos Mennens) Date: Thu, 10 Nov 2011 14:17:15 -0500 Subject: [Dovecot] TLS Authentication Confusion Message-ID: I asked a user today to make sure his incoming and outgoing email was using TLS. He told me it wasn't possible because my Dovecot / Postfix daemons were only listening on TCP 25 & 143 according to a port scan he did. He told me the only way I could enable encrypted secure sessions between the client & server is to enable port 993 (IMAPs). I told him that TLS is supported on my mail server over the default ports TCP 25 / 143 and that many consider IMAPs to be legacy. I sent him a telnet session of my PC communicating with my server & it shows TLS is available. I just wanted to be sure I was correct with the information above or am I completely wrong and I do indeed need TCP port 993? I know this is the Dovecot mailing list but since Dovecot and Postfix both use and support TLS in their configuration files, I figured I would ask here for your help! carloss at pc1:~$ telnet mail.holyghost.org 25 Trying 192.168.4.100... Connected to mail.holyghost.org. Escape character is '^]'. 220 mail.holyghost.org ESMTP Postfix EHLO pc1.holyghost.org 250-mail.holyghost.org 250-PIPELINING 250-SIZE 20480000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN Below is a snip from my mail logs showing TLS: Nov 9 10:26:39 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=::ffff:192.168.4.100, lip=::ffff:192.168.4.100, TLS The above snip from my log means that I'm connecting to Dovecot via TLS, correct? From dick at fouter.net Thu Nov 10 21:28:55 2011 From: dick at fouter.net (Dick Middleton) Date: Thu, 10 Nov 2011 19:28:55 +0000 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: References: Message-ID: <4EBC25F7.1040700@fouter.net> On 11/10/11 19:17, Carlos Mennens wrote: > I asked a user today to make sure his incoming and outgoing email was > using TLS. He told me it wasn't possible because my Dovecot / Postfix > daemons were only listening on TCP 25 & 143 according to a port scan > he did. He told me the only way I could enable encrypted secure > sessions between the client & server is to enable port 993 (IMAPs). Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same port as plain. The difference between SSH and TLS is that with SSH the encryption is set up before any application communication takes place. i.e all application packets are contained in the encrypted payload. With TLS the application starts communication and then the application sets up encryption of its payload. Dick From stephan at rename-it.nl Thu Nov 10 21:37:56 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 10 Nov 2011 20:37:56 +0100 Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> Message-ID: <4EBC2814.2090200@rename-it.nl> This mail was answered before. Don't repost your question unless you have acted on the information provided, got new information or have additional questions. Re-posting the exact same message makes no sense. Regards, Stephan. On 11/10/2011 5:09 AM, Scott Lewis wrote: > > > ----- Forwarded Message ----- > From: Scott Lewis > To: "dovecot at dovecot.org" > Sent: Thursday, 3 November 2011 4:31 PM > Subject: patching dovecot for sieve/managesieve support, centos 5.6? > > > Hi all, > > I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. > > Here's what happens: > > [root at mail ~]# whereis dovecot > dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz > > [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot > > ... > > checking whether to build static libraries... yes > dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH > to give path to compiled Dovecot sources or to a directory with the > installed dovecot-config file. configure: error: > dovecot-config not found > > -- > > I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. > > I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: > > Avelsieve plugin details: backend = ManageSieve > ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? > > thanks in advance! From frank at moltke28.B.Shuttle.DE Thu Nov 10 21:56:50 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Thu, 10 Nov 2011 20:56:50 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC25F7.1040700@fouter.net> References: <4EBC25F7.1040700@fouter.net> Message-ID: (auto-added) On Thu, 10 Nov 2011 19:28:55 +0000 Dick Middleton wrote: > On 11/10/11 19:17, Carlos Mennens wrote: > > I asked a user today to make sure his incoming and outgoing email was > > using TLS. He told me it wasn't possible because my Dovecot / Postfix > > daemons were only listening on TCP 25 & 143 according to a port scan > > he did. He told me the only way I could enable encrypted secure > > sessions between the client & server is to enable port 993 (IMAPs). > > Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same > port as plain. > > The difference between SSH and TLS is that with SSH the encryption is set up > before any application communication takes place. i.e all application packets > are contained in the encrypted payload. With TLS the application starts > communication and then the application sets up encryption of its payload. :%s/SSH/SSL/g --Frank From tom at whyscream.net Thu Nov 10 22:11:01 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 10 Nov 2011 21:11:01 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC25F7.1040700@fouter.net> References: <4EBC25F7.1040700@fouter.net> Message-ID: <4EBC2FD5.2060602@whyscream.net> On 10-11-11 20:28, Dick Middleton wrote: > On 11/10/11 19:17, Carlos Mennens wrote: >> I asked a user today to make sure his incoming and outgoing email was >> using TLS. He told me it wasn't possible because my Dovecot / Postfix >> daemons were only listening on TCP 25 & 143 according to a port scan >> he did. He told me the only way I could enable encrypted secure >> sessions between the client & server is to enable port 993 (IMAPs). > > Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same > port as plain. > > The difference between SSH and TLS is that with SSH the encryption is set up > before any application communication takes place. i.e all application packets > are contained in the encrypted payload. With TLS the application starts > communication and then the application sets up encryption of its payload. > You're contributing to the confusion. SSL and TLS are practically the same, just another name for the same beast. The only difference is that SSL is the old name, and newer versions of the standard are labeled TLS. The term SSH is not in the scope of this question. There are 2 ways of using SSL/TLS to encrypt sessions: 1) Setup a dedicated port where a SSL/TLS session can be setup before the actual data is transferred. This is what happens for IMAPS/993 and SMTPS/465. 2) Extend an existing protocol to enable SSL/TLS during an open session. This is called STARTTLS in several protocols, SMTP and IMAP being among them. And this is what happens on SMTP/25, Submission/587 and IMAP/143. Note that although the second option is *named* STARTTLS, you probably could implement any server to *use* SSL 1.0 for the actual encryption (not recommended though). The OP is offering STARTTLS for both services, which is good. -- Regards, Tom From noeldude at gmail.com Thu Nov 10 22:21:49 2011 From: noeldude at gmail.com (Noel) Date: Thu, 10 Nov 2011 14:21:49 -0600 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC2FD5.2060602@whyscream.net> References: <4EBC25F7.1040700@fouter.net> <4EBC2FD5.2060602@whyscream.net> Message-ID: <4EBC325D.60404@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11/10/2011 2:11 PM, Tom Hendrikx wrote: > On 10-11-11 20:28, Dick Middleton wrote: >> On 11/10/11 19:17, Carlos Mennens wrote: >>> I asked a user today to make sure his incoming and outgoing email was >>> using TLS. He told me it wasn't possible because my Dovecot / Postfix >>> daemons were only listening on TCP 25 & 143 according to a port scan >>> he did. He told me the only way I could enable encrypted secure >>> sessions between the client & server is to enable port 993 (IMAPs). >> >> Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same >> port as plain. >> >> The difference between SSH and TLS is that with SSH the encryption is set up >> before any application communication takes place. i.e all application packets >> are contained in the encrypted payload. With TLS the application starts >> communication and then the application sets up encryption of its payload. >> > > You're contributing to the confusion. > > SSL and TLS are practically the same, just another name for the same > beast. The only difference is that SSL is the old name, and newer > versions of the standard are labeled TLS. The term SSH is not in the > scope of this question. > > There are 2 ways of using SSL/TLS to encrypt sessions: > > 1) Setup a dedicated port where a SSL/TLS session can be setup before > the actual data is transferred. This is what happens for IMAPS/993 and > SMTPS/465. > > 2) Extend an existing protocol to enable SSL/TLS during an open session. > This is called STARTTLS in several protocols, SMTP and IMAP being among > them. And this is what happens on SMTP/25, Submission/587 and IMAP/143. > > Note that although the second option is *named* STARTTLS, you probably > could implement any server to *use* SSL 1.0 for the actual encryption > (not recommended though). > > The OP is offering STARTTLS for both services, which is good. > > -- > Regards, > Tom The confusion is caused by the way some client software differentiate these services in their configuration, often referring to wrappermode smtps/imaps as "SSL", and STARTTLS as "TLS". -- Noel Jones -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOvDJcAAoJEHIluGOd3V4F6foH/16+xq91/j4hgXufdnAsxwW1 N2ZXf1fby7TjR4BpaYNdH6PsN5/UqFSZItVYkeDXWgGG/wYCTRC+LHdks/EeQKgR 1ondUL2iorQ7bGy25m3526DGShFmcEh7P+Z6WWwdFeOTLBS57LIgwvFHBg4niYHq 3ZbPOjzI+d7kbz8tT8ATb+Ju+uJlV2rpbZKHQ90qlOR9tRl6bUOEeW32yPf5hjpI gs89o66Ud+mb9kkH9vgrhnutxsWjVxWNWM1ba43S1bh4Jg9YneIdsHdQVQSPrFUz EPy5Tgz3b+LZC6lwe6czFrhYgv/GUiJutS34qRHLSMAQGY+fgOcZBSZQHKP7NC4= =TdNE -----END PGP SIGNATURE----- From sven at svenhartge.de Thu Nov 10 23:15:05 2011 From: sven at svenhartge.de (Sven Hartge) Date: Thu, 10 Nov 2011 22:15:05 +0100 Subject: [Dovecot] LDAP expired password References: <116877.53915.qm@web29503.mail.ird.yahoo.com> <57o6m4h3miv8@mids.svenhartge.de> <4D95A998.9080701@noa.gr> <97o72ha3miv8@mids.svenhartge.de> Message-ID: rpalmarin wrote: > Sven Hartge svenhartge.de> writes: >> Nikolaos Milas noa.gr> wrote: >>> On 1/4/2011 11:09 ??, Sven Hartge wrote: >>>> Have a look at the ppolicy slapd.overlay. This will solve your >>>> problem. > Sorry for the delay in the response I checked the ppolicy overlay but > without success. This overlay does not have a single "password > expired" attribute to put in the user_filter. I think you misunderstood the usage of the overlay. There is _no_ additional attribute to check. With ppolicy any authentication will fail if some previously defined conditions are met (or no longer met) like the max age of a password. Documentation is contained in "man slapo-ppolicy", which as bit hard to understand, I must admit. Also look at http://www.openldap.org/doc/admin24/overlays.html "12.10 Password Policies" has a nice example. With this overlay you don't need any additional attributes and no maintenance or houskeeping script to invalidate expired passwords. >> At my university we introduced our own attribute gifb-status which >> contains a "1" if an account is valid, a "0" if it is not (and >> several others for different purposes) and our ldap-filters all >> contain something like "(&(ou=foobar)(gifb-status=1))". > is possible that the only way to do this is to manage a new attribute? > how can understand all the people that have configured the mail > client to authenticate with imap-dovecot that their passoword has > expired? Well, either way (using ppolicy or an additional attribute): they will call the support desk, if they are unable to understand the message from their mail client. No way to fix _this_ problem, I am afraid ;) S? -- Sigmentation fault. Core dumped. From frank at moltke28.B.Shuttle.DE Fri Nov 11 13:01:05 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 12:01:05 +0100 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop Message-ID: (auto-added) Hi, running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA I've found a strange error in log which I've nerver observed before when startung dovecot: auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop Dovecot seems to work normal despite the error message. Any enlightment welcome. --Frank Elsner From tss at iki.fi Fri Nov 11 13:10:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 11 Nov 2011 13:10:57 +0200 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop In-Reply-To: (auto-added) References: (auto-added) Message-ID: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> On 11.11.2011, at 13.01, Frank Elsner wrote: > running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA .. > auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop authdb_imap is from Dovecot v2.1. It doesn't exist in v2.0.14.. From peter at newton.cx Fri Nov 11 09:41:21 2011 From: peter at newton.cx (Peter Williams) Date: Thu, 10 Nov 2011 23:41:21 -0800 Subject: [Dovecot] Invalid "doveadm search" queries not rejected Message-ID: <1320997281.2261.116.camel@iota.localdomain> Hello, I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS 6). According to the doveadm-search-query(7) manual page, a "mailbox" specification can only come at the beginning of a search query. However, if I put a mailbox specification in the middle of a query, not only do I not get an error message, I get surprising results: $ doveadm search mailbox A |wc -l 6979 $ doveadm search mailbox B |wc -l 206 $ doveadm search mailbox A OR mailbox B |wc -l 42492 If it became possible to do boolean logic on mailbox queries, that'd be cool. But barring that, these invalid queries should be rejected. Relatedly, if I can only do simple mailbox filtering, does anyone know of a clever way to efficiently emulate more complex boolean logic? Please CC me on any replies as I'm not subscribed to the list. Thanks, Peter -- Peter Williams / peter at newton.cx From frank at moltke28.B.Shuttle.DE Fri Nov 11 16:28:44 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 15:28:44 +0100 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop In-Reply-To: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> References: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> Message-ID: (auto-added) On Fri, 11 Nov 2011 13:10:57 +0200 Timo Sirainen wrote: > On 11.11.2011, at 13.01, Frank Elsner wrote: > > > running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA > .. > > auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop > > authdb_imap is from Dovecot v2.1. It doesn't exist in v2.0.14.. Nov 11 15:22:13 seymour dovecot: master: Dovecot v2.0.15 starting up (core dumps disabled) Nov 11 15:22:13 seymour dovecot: auth: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop This is definitly version 2.0.15 starting. But ... /usr/local/dovecot/lib/dovecot/auth has date Nov 9 10:17 usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so has date Nov 9 10:17 This was the time I've tested dovecot-2.1. --Frank Elsner From frank at moltke28.B.Shuttle.DE Fri Nov 11 16:37:17 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 15:37:17 +0100 Subject: [Dovecot] Re(2): undefined symbol: o_stream_switch_ioloop In-Reply-To: References: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> Message-ID: (auto-added) On Fri, 11 Nov 2011 15:28:44 +0100 Frank Elsner wrote: [ ... ] > But ... > > /usr/local/dovecot/lib/dovecot/auth has date Nov 9 10:17 > usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so has date Nov 9 10:17 > > This was the time I've tested dovecot-2.1. After removing all files from Nov 9 10:17 in /usr/local/dovecot/lib/dovecot version 2.0.15 start without the error message :-) --Frank Elsner From dovecot-20110531 at billmail.scconsult.com Fri Nov 11 18:00:47 2011 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Fri, 11 Nov 2011 11:00:47 -0500 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: References: Message-ID: <4EBD46AF.40507@billmail.scconsult.com> Carlos Mennens wrote, On 11/10/11 2:17 PM: > I asked a user today to make sure his incoming and outgoing email was > using TLS. He told me it wasn't possible because my Dovecot / Postfix > daemons were only listening on TCP 25& 143 according to a port scan > he did. He told me the only way I could enable encrypted secure > sessions between the client& server is to enable port 993 (IMAPs). I > told him that TLS is supported on my mail server over the default > ports TCP 25 / 143 and that many consider IMAPs to be legacy. I sent > him a telnet session of my PC communicating with my server& it shows > TLS is available. I just wanted to be sure I was correct with the > information above or am I completely wrong and I do indeed need TCP > port 993? You are correct. To convince your user, point him at http://en.wikipedia.org/wiki/STARTTLS which has links to the relevant RFC's. > > I know this is the Dovecot mailing list but since Dovecot and Postfix > both use and support TLS in their configuration files, I figured I > would ask here for your help! > > carloss at pc1:~$ telnet mail.holyghost.org 25 > Trying 192.168.4.100... > Connected to mail.holyghost.org. > Escape character is '^]'. > 220 mail.holyghost.org ESMTP Postfix > EHLO pc1.holyghost.org > 250-mail.holyghost.org > 250-PIPELINING > 250-SIZE 20480000 > 250-VRFY > 250-ETRN > 250-STARTTLS > 250-ENHANCEDSTATUSCODES > 250-8BITMIME > 250 DSN > > Below is a snip from my mail logs showing TLS: > > Nov 9 10:26:39 mail dovecot: imap-login: Login: user=, > method=PLAIN, rip=::ffff:192.168.4.100, lip=::ffff:192.168.4.100, TLS > > The above snip from my log means that I'm connecting to Dovecot via > TLS, correct? Yes From james.page at ubuntu.com Fri Nov 11 18:32:32 2011 From: james.page at ubuntu.com (James Page) Date: Fri, 11 Nov 2011 16:32:32 +0000 Subject: [Dovecot] Restarting dovecot periodically fails with "Socket already exists: /var/run/dovecot/anvil" In-Reply-To: <1320513622.21919.260.camel@hurina> References: <4E97000D.7030908@ubuntu.com> <1320513622.21919.260.camel@hurina> Message-ID: <4EBD4E20.50800@ubuntu.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 05/11/11 17:20, Timo Sirainen wrote: > http://hg.dovecot.org/dovecot-2.1/rev/6894298ae5fd should apply to > v2.0 as well, but since it's a bit large change I don't know if I > should backport it, at least without some more testing. Thanks for this pointer Timo I've patched this into dovecot in the latest Ubuntu development release - it appears to resolve the issue and we are a good 6 months from release so plenty of time to a) flush out any problems or b) upgrade to 2.1. Cheers James - -- James Page Ubuntu Core Developer -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJOvU4gAAoJEL/srsug59jD7fYQAIy+wnPiYuv58wazO23V4nyA ut2D6iaChsNG7XrySNtgY/RajOu9O8dIOnLuLTD9DmxbRZEaH2Tzd/7n6I0P3i7t h3PyI3tW7GPFyII/VOnAsTjfmncjF/ug7ohyUw2CYc1z2D7lQE0zTFlQeEWCqDCz mxKEHg37TvPlRZzfSvzgjn2hoTJ/vmhQPM11I8uEVYOnWWwahCZxkA2rgBmkjtEG mhCZfgoaJ6BZ5fH0BbjChN5Vr4zuQVEws8fkbGX/7/JPUxKHkz7jdxhVuZr7YN5o xy0HfGRr4sqa1v6qqTAFvFmZu6n3V6dVvn64SkjDLPECnOH80HxPzwMKN9+/cnYi d54/xIYH5N0pPhnSah+/caYCJp5FqDeAmim7xpMLExMzRdqvfbaI7ggGILecN3cH iJiWyj32jCArlAFRFKbxrin8VMRFhU4i+S7zigDp3jFuMotMnL5e1Wk+qrDjKRvt XhairFpT9po5rSvJp/J6qJxWrb+1vznBQkKgl91o570AjBBbRfmYvhxtnU5j2FZ9 MwA8pAD9+n7dc5aGY3o8ABa+5/0wqkAt8qGQ8HYuq+I86NwUbjTadvFLZcfy1Vy5 OOSoMPeruz2tQTCb0/PIHM35MFCmm45Lo0ZamqFmCi4iRKcKwx4tVesJHZY51lGu f29+i9iDlLmofPvhUQkF =hTb+ -----END PGP SIGNATURE----- From dovecot at knutejohnson.com Fri Nov 11 18:33:09 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Fri, 11 Nov 2011 08:33:09 -0800 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBD46AF.40507@billmail.scconsult.com> References: <4EBD46AF.40507@billmail.scconsult.com> Message-ID: <4EBD4E45.4080402@knutejohnson.com> So is a STARTTLS on port 143 session somehow superior to using SSL on port 993? Can you still limit connections on port 143 to require STARTTLS? Thanks, -- Knute Johnson From tkrah at fachschaft.imn.htwk-leipzig.de Fri Nov 11 20:43:22 2011 From: tkrah at fachschaft.imn.htwk-leipzig.de (Torsten Krah) Date: Fri, 11 Nov 2011 19:43:22 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBD4E45.4080402@knutejohnson.com> References: <4EBD46AF.40507@billmail.scconsult.com> <4EBD4E45.4080402@knutejohnson.com> Message-ID: <1321037002.2772.1.camel@sf045> Am Freitag, den 11.11.2011, 08:33 -0800 schrieb Knute Johnson: > So is a STARTTLS on port 143 session somehow superior to using SSL on > port 993? Can you still limit connections on port 143 to require STARTTLS? Yes you can disable authentication until starttls is issued. > > Thanks, > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5253 bytes Desc: not available URL: From bambero at gmail.com Sat Nov 12 06:32:31 2011 From: bambero at gmail.com (Bambero) Date: Sat, 12 Nov 2011 05:32:31 +0100 Subject: [Dovecot] submission_host problem Message-ID: Hello, I configured dovecot to use submission smtp host becouse of chroot. submission_host = 127.0.0.1 Unfortunatelly: Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next input="MAIL FROM:\\r\\n" Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next input="MAIL FROM:\\r\\n" Nov 12 05:11:15 myhost dovecot: lda(alias at apisoft.pl): Error: smtp(127.0.0.1): RCPT TO failed: 554 SMTP synchronization error Nov 12 05:11:15 myhost dovecot: lda(alias at apisoft.pl): Error: sieve: msgid=: failed to redirect message to (refer to server log for more information) Does anyone know how to fix it ? Regards, Bambero From rajeev at teamsg.in Sat Nov 12 09:28:42 2011 From: rajeev at teamsg.in (Rajeev Sharma) Date: Sat, 12 Nov 2011 12:58:42 +0530 Subject: [Dovecot] Migration From Courier Imap. Message-ID: <653a51f07131e61c52bf983359f4710c@192.168.0.251> Sir, I am using vpopmail,qmail along with courier-imap 4.0 i want to migrate from courier to dovecot but i have 100 shared folder under /home/vpopmail/domains/domain.com/Malidir and user shared them on the basis of rights assign. I am very much clear how to migrate from courier to dovecot but i am unable how to migrate shared folder and assign rights. Rajeev Sharma. -- Rajeev Sharma From mnbbrown at gmail.com Sun Nov 13 11:49:21 2011 From: mnbbrown at gmail.com (Matthew Brown) Date: Sun, 13 Nov 2011 19:49:21 +1000 Subject: [Dovecot] Dovecot v2.0.13 unix_listeners Message-ID: <604BA92C-C1D5-4694-B17E-E678C5FFFC0D@gmail.com> Could somebody be able to explain to me what the various unix_listeners do in the Dovecot 2.x configuration (specifically in 10-master.conf). Currently, for postfix to use for local delivery, I have: service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } and for auth I have: service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0666 user = vmail } } So what does each one specifically do? Also, does somebody know of a resource that can explain the mode setting? I couldn't seem to find anything on the wiki.. Sorry if these questions seem simple, Im still trying to get a grips with dovecots workings. MDA: Dovecot v2.0.13 MTA: SMTP User & Virtual Domain Management: MySQL Cheers, Matthew From scott_the_musician at yahoo.com.au Sun Nov 13 12:40:20 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Sun, 13 Nov 2011 02:40:20 -0800 (PST) Subject: [Dovecot] [re] patching dovecot for managesieve/sieve Message-ID: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> My apologies for the repost yesterday; it was not intentional. I was attempting to post a followup from my mobile, when something went awry with the editing options on my phone. Anyway.. ?I have managed to download and compile my own patched dovecot/dovecot-sieve/dovecot-managesieve modules from scratch, with Thanks to this page hardc0l2e.wordpress.com/2010/02/26/managesieve-with-dovecot-on-centos-5-4/ But when I run http://mydomain.com/src/configtest.php , I am still getting the following rather non-descriptive error: "I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=ManageSieve) is bad?" Where else can I look to resolve this? should managesieve show up as its own /etc/init.d service? Or does dovecot simply control it? How can I confirm that deliver is being used as my LDA? Thanks again, Scott From andreas at kado-web.de Sun Nov 13 16:48:49 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Sun, 13 Nov 2011 15:48:49 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive Message-ID: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> Hi List, i am trying to restore mails from an backup-archive which was made with rsnapshot. All the mails in the archive look similar like the following: -rw------- 2 1014 1015 3308 8. Aug 13:42 1312803768.V809I266a00aM505178.server:2,ST Now I copy the files from the archive into the Maildir/cur directory of the user. Then I set the permissions etc. and log in with the user account via Squirrelmail. The restored mails aren't in the INBOX, and when I go into the Maildir/cur dir of the user, the restored mails are gone. Possibly deleted after the login. Had anyone some similar experiences ? Any hints ? Kind regards Andreas From bambero at gmail.com Sun Nov 13 18:25:55 2011 From: bambero at gmail.com (Bambero) Date: Sun, 13 Nov 2011 17:25:55 +0100 Subject: [Dovecot] Chroot Message-ID: Hello, I have successfully setup dovecot with chroot but there are two things yet 1. submission_host problem (I posted message 2 days ago) 2. Problem with acl's and sharing: all maildirs are located in: /var/mail/jdoe at example.com/Maildir so: mail_chroot=/var/mail I setup acls with: 10-mail.conf ... namespace { type = private separator = / prefix =. inbox = yes hidden = no list = yes subscriptions = yes } namespace { type = shared separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = yes list = yes } ... 90-acl.conf ... plugin { acl = vfile:cache_secs=5 } plugin { acl_shared_dict = file:/%i.db } ... Setting acl is no problem there are dovecot-acl dovecot-acl-list in Maildir. and dictionary /var/mail/xxx.db contains list od shares I can also see shared folder in my IMAP client but when trying to subscribe i got error: Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No such file or directory Error: Namespace 'shared/': Could not lookup home for user jdoe at example.com /var/dovecot/auth-userdb is not accessible because chroot. But why userdb is necessary in this case ? I hope there is a solution without disabling chroot. Below my basic configuration. Regards, Bambero # 2.0.13: /etc/dovecot/dovecot.conf # OS: OpenBSD 5.0 i386 auth_mechanisms = plain login digest-md5 cram-md5 apop default_internal_user = _dovecot default_login_user = _dovenull first_valid_gid = 1000 first_valid_uid = 1000 last_valid_gid = 10000 last_valid_uid = 10000 mail_chroot = /var/mail mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date imapflags notify mbox_write_locks = fcntl mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir:User quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_global_path = /etc/sieve/default.sieve } protocols = imap pop3 sieve sieve service auth { unix_listener auth-client { mode = 0660 user = _exim } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/ca-bundle.crt ssl_cert = References: <604BA92C-C1D5-4694-B17E-E678C5FFFC0D@gmail.com> Message-ID: <20111113163741.Horde.s9ZFZpLnE6FOwDilUtzVjrA@kishi.patrickdk.com> Heh? they pretty much do exactly as described. the service lmtp is how you inject email into dovecot using the lmtp protocol. the service auth is how you can authenicate user/password against dovecot, normally used in postfix for sasl auth the mode setting is just that, the file mode to set for that file, 0666, all access, 0660 only user/group access. Quoting Matthew Brown : > Could somebody be able to explain to me what the various > unix_listeners do in the Dovecot 2.x configuration (specifically in > 10-master.conf). > > Currently, for postfix to use for local delivery, I have: > > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > > and for auth I have: > > service auth { > > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > > unix_listener auth-userdb { > mode = 0666 > user = vmail > } > } > > So what does each one specifically do? > > Also, does somebody know of a resource that can explain the mode > setting? I couldn't seem to find anything on the wiki.. > Sorry if these questions seem simple, Im still trying to get a grips > with dovecots workings. > > > MDA: Dovecot v2.0.13 > MTA: SMTP > User & Virtual Domain Management: MySQL > > Cheers, > Matthew From georg.martius at web.de Sun Nov 13 23:57:46 2011 From: georg.martius at web.de (Georg Martius) Date: Sun, 13 Nov 2011 22:57:46 +0100 Subject: [Dovecot] metadata plugin problems Message-ID: <3455528.juJhefTHFy@chugul> Hi, I have trouble with the metadata plugin. I compiled dovecot 2.0.15 from source and also the plugin [2] separately. I work on debian/lenny. I had some trouble compiling it (autoconf somehow didn't insert the DOVECOT_IMAP_INCLUDE variable, I have not idea why) Anyway, I did as described in [1]. Do I have to create: the file /var/lib/dovecot/shared-metadata ? The server starts without errors, but I get for instance the following error message: "Failed to write some annotations for 'Contacts' on the IMAP server. SetMetaData failed, server replied: A000076 NO Parsing entry failed" from my KMail. How can I test that the plugin is actually active and working? Any help would be appreciated. Regards, Georg [1]http://www.mail-archive.com/dovecot at dovecot.org/msg38887.html [2] http://hg.dovecot.org/dovecot-metadata-plugin From dovecot-20110531 at billmail.scconsult.com Mon Nov 14 00:27:02 2011 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Sun, 13 Nov 2011 17:27:02 -0500 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> References: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> Message-ID: <4EC04436.3090905@billmail.scconsult.com> Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From pw at wk-serv.de Mon Nov 14 01:31:24 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 14 Nov 2011 00:31:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EC0534C.8070409@wk-serv.de> Peer Heinlein schrieb: > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. You have 11 TB of mails on a non cluster filesystem? Is it only accessed from one server or how does it work? From andreas at kado-web.de Mon Nov 14 08:09:01 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Mon, 14 Nov 2011 07:09:01 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <4EC04436.3090905@billmail.scconsult.com> Message-ID: <0MHQzX-1RdoD93uYl-00Dcge@mrelayeu.kundenserver.de> Thanks for that information bill. The T was the point which was important for me, because the user didn't tell me that the messages were already marked for deletion. - Anyway, in outlook the mails were visible (marked for deletion). -----Urspr?ngliche Nachricht----- Von: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Bill Cole Gesendet: Sonntag, 13. November 2011 23:27 An: Dovecot Mailing List Betreff: Re: [Dovecot] Restoring Mails from Backup Archive Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From andreas at kado-web.de Mon Nov 14 08:09:20 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Mon, 14 Nov 2011 07:09:20 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <4EC04436.3090905@billmail.scconsult.com> Message-ID: <0M2TGj-1QaLdl2jLB-00sOsn@mrelayeu.kundenserver.de> Thanks for that information bill. The T was the point which was important for me, because the user didn't tell me that the messages were already marked for deletion. - Anyway, in outlook the mails were visible (marked for deletion). -----Urspr?ngliche Nachricht----- Von: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Bill Cole Gesendet: Sonntag, 13. November 2011 23:27 An: Dovecot Mailing List Betreff: Re: [Dovecot] Restoring Mails from Backup Archive Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From p.heinlein at heinlein-support.de Mon Nov 14 11:34:02 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Mon, 14 Nov 2011 10:34:02 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EC0534C.8070409@wk-serv.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> Message-ID: <201111141034.03060.p.heinlein@heinlein-support.de> Am Montag, 14. November 2011, 00:31:24 schrieb Patrick Westenberg: > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > HP EVA. > > You have 11 TB of mails on a non cluster filesystem? Yes. I don't believe a clustered filesystem would have more performance and would be more rock solid. I don't have a problem on my frontend server. Why should I have two or more of them? I have a problem in my backend. My SAN has too much to do. Why should a cluster filesystem be better for my SAN? > Is it only accessed from one server or how does it work? Yes. peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From arif at team.de.inter.net Mon Nov 14 11:51:20 2011 From: arif at team.de.inter.net (Kai Arif) Date: Mon, 14 Nov 2011 10:51:20 +0100 Subject: [Dovecot] patch for UID 0 problem Message-ID: <4EC0E498.7000900@team.de.inter.net> Hi all, attached you'll find a patch for cmd-thread.c which addresses a problem with certain clients in conjunction with the use of the UID THREAD REFS command: TAG5 UID THREAD REFS us-ascii SINCE 14-May-2011 * THREAD (0)(246)(247)(248)(249)(250)(252)(253)(254)(255)(256)(257)(258)(259)(260)(261)(262)(263)(264)(265)(266)(267)(268) TAG5 OK Thread completed. The first UID delivered (0) is invalid, some clients - @mail for instance - then try to fetch the invalid UID 0 subsequently and run into problems. The patch actually disables the return of (0) and seems to work for us. Please review and - if OK - incorporate it in the next version. Best Regards Kai -- Kai Arif - System Administrator Inter.net Germany GmbH Knesebeckstra?e 59-61 10719 Berlin Germany Fon +49 30 25430 0 Fax +49 30 25430 499 arif at team.de.inter.net www.de.inter.net Legal-Information: Inter.net Germany GmbH, HRB #79136, Amtsgericht Berlin Charlottenburg, UST-IdNr:: DE 813 165 159 FA f?r K?rperschaften I Berlin, Gesch?ftsf?hrer: J?rn Lubkoll Zust?ndige Regulierungsbeh?rde: Bundesnetzagentur Chemnitz, Liselotte-Herrmann-Str. 20a, 09127 Chemnitz, Registriernummer: 06/164 -------------- next part -------------- A non-text attachment was scrubbed... Name: cmd-thread.c.diff Type: text/x-diff Size: 644 bytes Desc: not available URL: From stephan at rename-it.nl Mon Nov 14 12:06:22 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 14 Nov 2011 11:06:22 +0100 Subject: [Dovecot] [re] patching dovecot for managesieve/sieve In-Reply-To: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> References: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> Message-ID: <4EC0E81E.2090401@rename-it.nl> On 13-11-2011 11:40, Scott Lewis wrote: > My apologies for the repost yesterday; it was not intentional. I was attempting to post a followup from my mobile, when something went awry with the editing options on my phone. > Anyway.. > > I have managed to download and compile my own patched dovecot/dovecot-sieve/dovecot-managesieve modules from scratch, with Thanks to this page > > hardc0l2e.wordpress.com/2010/02/26/managesieve-with-dovecot-on-centos-5-4/ > > But when I run http://mydomain.com/src/configtest.php , I am still getting the following rather non-descriptive error: > > "I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=ManageSieve) is bad?" You can test whether ManageSieve is working properly using this procedure: http://wiki.dovecot.org/ManageSieve/Troubleshooting > Where else can I look to resolve this? should managesieve show up as its own /etc/init.d service? Or does dovecot simply control it? > How can I confirm that deliver is being used as my LDA? The Dovecot LDA produces characteristic log lines in your system log (http://wiki1.dovecot.org/Logging). For instance, with Sieve enabled, the following is reported upon delivery to INBOX: Nov 14 10:53:41 host dovecot: deliver(stephan): sieve: msgid=: stored mail into mailbox 'INBOX' Something similar is logged without Sieve. All such messages are prefixed with 'deliver(): '. With mail_debug=yes, more verbose messages are logged as well. Regards, Stephan. From janfrode at tanso.net Mon Nov 14 12:27:19 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 14 Nov 2011 11:27:19 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111141034.03060.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> Message-ID: <20111114102719.GA2330@dibs.tanso.net> On Mon, Nov 14, 2011 at 10:34:02AM +0100, Peer Heinlein wrote: > > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > > HP EVA. > > > > You have 11 TB of mails on a non cluster filesystem? > > Yes. > > I don't believe a clustered filesystem would have more performance and > would be more rock solid. > > I don't have a problem on my frontend server. Why should I have two or > more of them? I have a problem in my backend. My SAN has too much to do. > Why should a cluster filesystem be better for my SAN? Agree. A non-clustered fs should give you better performance, and probably also be more reliable, if you can live with the SPoF and full downtime during patching/upgrades/maintenance. But I would expect xfs to be a better choice than ext*. We have about the same storage size as you (12TB/115M-inodes), with the backup-process almost biting itself in the tail every day, but I can't quite imagine running it all on a single local fs with no scale-out options if we should want/need more processing power for dovecot. I'm looking forward to moving to mdbox soonish.. to reduce the number of files and speed up the backup process. -jf From tlx at leuxner.net Mon Nov 14 13:30:08 2011 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 14 Nov 2011 12:30:08 +0100 Subject: [Dovecot] Managing public folder ACL files In-Reply-To: <1302268626.24579.79.camel@hurina> References: <20110406115434.3049ba70@kae.tiger-computing.wbp> <1302268626.24579.79.camel@hurina> Message-ID: <20111114113008.GA5126@nihlus.leuxner.net> On Fri, Apr 08, 2011 at 04:17:06PM +0300, Timo Sirainen wrote: > > doveadm mailbox list -u admin at domain "sales/CustA/*" | > while read mailbox; do > doveadm acl set -u user at domain "$mailbox" user at domain +lrw > done Have the +lrw modifiers been implemented at all? I can only get it to work with syntax like: $ doveadm acl set -u user at domain "Shops/ACME" owner insert lookup post read write write-seen This is a bit painful to type and does not allow to add or remove rights... Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From achekalin at lazurit.com Mon Nov 14 16:35:07 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 14 Nov 2011 17:35:07 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EC1271B.4060604@lazurit.com> Timo, Stan, I've just tested mdbox and find it pretty nice for me, but now I got some questions for you: 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if 2Mb. Looks like not even every message can fit into such storage container volume (nowadays we used to see messages of 20Mb and even more). Should I tune it (at least mdbox_rotate_size and mdbox_rotate_interval) or its size is on purpose? As for now I store each day's messages in separate IMAP folders (mailboxes), which gives me 2000-6000 messages and 2-5 Gb (on disk) per folder. 2. I can use no compression, gz and bz2 - which one will be better for storing archive messages? I've just tested mdbox by copying 5800+ msgs from maildir to compressed mdbox, and it took exactly the same size (2.8 G) in 100+ small m.* files. No good as far. 3. What if I use maildir as I do now but turn on compression, will this speed things up? I'd like to use mdbox as storage but for now it is very new for me and I simple afraid what should I do if I'll need to manually fix the storage (maildir is really good for that, surely). After all, I simple need to speed up the search and restore process in archive. Yours, Alexander From stan at hardwarefreak.com Mon Nov 14 20:23:14 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 14 Nov 2011 12:23:14 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC1271B.4060604@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> Message-ID: <4EC15C92.4000707@hardwarefreak.com> On 11/14/2011 8:35 AM, Alexander Chekalin wrote: > Timo, Stan, > > I've just tested mdbox and find it pretty nice for me, but now I got > some questions for you: > > 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if > 2Mb. Looks like not even every message can fit into such storage > container volume (nowadays we used to see messages of 20Mb and even > more). Should I tune it (at least mdbox_rotate_size and > mdbox_rotate_interval) or its size is on purpose? As for now I store > each day's messages in separate IMAP folders (mailboxes), which gives me > 2000-6000 messages and 2-5 Gb (on disk) per folder. mdbox_rotate_size of 2MB is too small for your needs. Test 32MB and 64MB. > 2. I can use no compression, gz and bz2 - which one will be better for > storing archive messages? I've just tested mdbox by copying 5800+ msgs > from maildir to compressed mdbox, and it took exactly the same size (2.8 > G) in 100+ small m.* files. No good as far. bzip2 may give you a little better compression but at the cost of much lower de/compression speed and higher CPU and memory consumption. gzip will be faster all around, between 4x-8x, with lower mem usage, but with less compression resulting in slightly larger file sizes than bzip2. > 3. What if I use maildir as I do now but turn on compression, will this > speed things up? No. Maildir performance is limited by the disk head actuator speed, which is between 150-300 seeks per second depending on your disk (7.2k vs 15k RPM). Compressing the files doesn't change the seek physics of the disk drives. You're still reading tens of thousands of files when doing your searches thus bouncing the heads tens of thousands of times. mbox uses a single file, so head speed isn't a factor, as it may only move a few times when reading an entire mailbox file. Thus, bandwidth becomes the potential bottleneck. Using compression with large mbox files can substantially increase search performance as effective bandwidth is increased by ~4x using gzip and 6x using bzip2. This assumes you have plenty of excess CPU power. mdbox should see similar compression speedups if you use file sizes much larger than the 2MB default. Doing so should keep your IOPS well below the drive's head saturation point as you're reading only a fraction of the file count compared to maildir. > I'd like to use mdbox as storage but for now it is very new for me and I > simple afraid what should I do if I'll need to manually fix the storage > (maildir is really good for that, surely). Doveadm handles such tasks pretty well. Just make sure you keep good backups of your mdbox files. > After all, I simple need to speed up the search and restore process in > archive. The only way to accomplish this with maildir is with much bigger, faster, more expensive storage hardware. And the gain will still be much less than simply switching to a larger file format such as mbox or mdbox. As with many things some computer technologies come full circle over time. One of the reasons the creators of the UNIX mbox mail file format decided upon a single file many decades ago was the horribly limited seek performance of the slow SCSI disks of that period. Doing something like the maildir format was simply impossible at that time. In the early days of the public internet, disk became faster than the average load and maildir was born to fix the locking and corruption shortcomings of mbox. Today many sites are hitting the seek problem of a few decades ago because boxes are oversubscribed with users, emails now frequently contain attachments, everyone is storing more email, and the total volume of email is a few orders of magnitude greater. IIRC, this is one of the reasons Timo created mdbox--to decrease the massive IOPS load, and thus slow performance, of large maildir stores. -- Stan From moseleymark at gmail.com Mon Nov 14 20:23:37 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 14 Nov 2011 10:23:37 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: Message-ID: On Mon, Sep 26, 2011 at 10:11 AM, Mark Moseley wrote: > On Fri, Sep 23, 2011 at 3:35 PM, Mark Moseley wrote: >> I've been goofing with this all day with 2.0.15 and I'm starting to >> realize that either a) I'm not that smart, b) it's been so long since >> I messed with namespaces that I'm going about it completely wrong, or >> c) it's just not possible. I haven't posted 'doveconf -n' and other >> details, because mainly I'm just looking for 'yes, this is possible' >> or 'no, you're smoking crack' before posting further details. At this >> point, it's all maildir and moving to mdbox, while highly desirable in >> the future, is not possible in the near- to medium-term. >> >> I'm trying to glue a namespace underneath INBOX: >> >> namespace INBOX { >> ? ? ? ?type = private >> ? ? ? ?separator = . >> ? ? ? ?prefix = INBOX. ? ?# Yes, this used to be on Courier >> ? ? ? ?inbox = yes >> ? ? ? ?list = yes >> ? ? ? ?hidden = no >> ? ? ? ?subscriptions = yes >> ? ? ? ?location = maildir:~/Maildir >> } >> namespace archive { >> ? ? ? ?type = private >> ? ? ? ?separator = . >> ? ? ? ?prefix = INBOX.Archives. >> ? ? ? ?inbox = no >> ? ? ? ?list = children >> ? ? ? ?subscriptions = yes >> ? ? ? ?location = maildir:~/Maildir-Archive >> } >> >> >> I've tried putting namespace archive's 'prefix' as just "Archives", >> but Tbird doesn't seem to see this namespace, regardless of how much I >> futz with the imap settings in tbird. >> >> With the above setup, it actually seems to work correctly (provided >> ~/Maildir-Archive exists), though I'm sure a big gotcha is waiting in >> the wings. I can move messages around, create subfolders, subscribe to >> folders in ~/Maildir-Archive). The only thing I can't seem to get >> working is quotas. With my password_query like: >> >> password_query = ... >> CONCAT( '*:bytes=', "1M" ) AS 'userdb_quota_rule', \ >> CONCAT( '*:messages=10' ) AS 'userdb_quota_rule2', \ >> CONCAT( 'INBOX.Archives:bytes=+4900M' ) AS 'userdb_quota_rule3', \ >> CONCAT( 'INBOX.Archives:messages=+3900' ) AS 'userdb_quota_rule4' >> ... >> >> only the default quota seems to be in place for any subfolder of >> INBOX.Archives and for INBOX.Archives itself, i.e. *:bytes still >> applies to INBOX.Archives. The debug log show that: >> >> Debug: Quota root: name=User quota backend=maildir args= >> Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=0 >> Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=10 >> Debug: Quota rule: root=User quota mailbox=INBOX.Archives >> bytes=+5138022400 messages=0 >> Debug: Quota rule: root=User quota mailbox=INBOX.Archives >> bytes=+5138022400 messages=+3900 >> >> These are wildly stupid quotas but they're just there to test. With >> INBOX already at capacity (byte-wise; only set to a meg), copying >> large messages inside INBOX.Archives fails (only copying a 800k >> message but the quota should be 5gig now). >> >> Again, before I post configs, I'm just curious if what I'm trying to >> do isn't remotely possible, or that I'm approaching this entirely >> wrongly. Thanks! >> > > Thanks to a fortuitously unrelated thread ("how to disable quota for > second namespace"), I got the quota part figured out and that seems to > be working: Add a second entry to plugin {}, e.g. "quota2 = > maildir:Archive quota:ns=INBOX.Archives." and add rules for > userdb_quota2_rule, userdb_quota2_rule2, etc. > > My real question now is: Are there any fatal gotchas in this that I'm > just not thinking of? > Haven't had a chance to try this large-scale yet. Anybody have any thoughts on it? From stan at hardwarefreak.com Mon Nov 14 20:42:30 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 14 Nov 2011 12:42:30 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111114102719.GA2330@dibs.tanso.net> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> Message-ID: <4EC16116.1000902@hardwarefreak.com> On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > Agree. A non-clustered fs should give you better performance, and > probably also be more reliable, if you can live with the SPoF and > full downtime during patching/upgrades/maintenance. But I would expect > xfs to be a better choice than ext*. Depends on the workload characteristics and how well the XFS filesystem is tuned to the storage hardware. If setup properly, using many allocation groups with fast spindles, a decent amount of BBWC, and a high concurrency maildir workload (dozens to hundreds of delivery and IMAP operations), XFS will runs circles around EXTx as it can create/write/read to every AG in parallel. Much of EXT4's operation is still serialized. This is why XFS outruns all other filesystems in the highly parallel mail workload benchmarks I posted previously, EXTx by a factor of 2-3. For smaller hosts that don't see parallelism, for example SOHO servers, XFS will likely be slower than EXTx as the workload will be serialized. -- Stan From achekalin at lazurit.com Mon Nov 14 23:16:53 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 15 Nov 2011 00:16:53 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC15C92.4000707@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> Message-ID: <43B341A1-9245-4863-932A-887E21970319@lazurit.com> Locking issues on mbox is the reason for my long-lasting love affair with maildir, and it's lasts long years. Ok, the life's lessons are like this, learn something and move on with it ;) even if it's "new old thing". Thank you for pointing that! What I was doubt about is default rotate size of 2M, since I used to see pretty reasonable default settings in all Dovecot config. 32 or 64 are much close to the ones I'd personally prefer. I also about to choose now is the OS and FS for the archive. I seriously think about ZFS with compression (in fact it will be stripes over couple of mirrors = software equivalent of RAID 10 on SATA drives, with compression on FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox itself. I see pros and contras for both, so that's the question to answer! Yours, Alexander > On 11/14/2011 8:35 AM, Alexander Chekalin wrote: >> Timo, Stan, >> >> I've just tested mdbox and find it pretty nice for me, but now I got >> some questions for you: >> >> 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if >> 2Mb. Looks like not even every message can fit into such storage >> container volume (nowadays we used to see messages of 20Mb and even >> more). Should I tune it (at least mdbox_rotate_size and >> mdbox_rotate_interval) or its size is on purpose? As for now I store >> each day's messages in separate IMAP folders (mailboxes), which gives me >> 2000-6000 messages and 2-5 Gb (on disk) per folder. > > mdbox_rotate_size of 2MB is too small for your needs. Test 32MB and 64MB. > >> 2. I can use no compression, gz and bz2 - which one will be better for >> storing archive messages? I've just tested mdbox by copying 5800+ msgs >> from maildir to compressed mdbox, and it took exactly the same size (2.8 >> G) in 100+ small m.* files. No good as far. > > bzip2 may give you a little better compression but at the cost of much > lower de/compression speed and higher CPU and memory consumption. gzip > will be faster all around, between 4x-8x, with lower mem usage, but with > less compression resulting in slightly larger file sizes than bzip2. > >> 3. What if I use maildir as I do now but turn on compression, will this >> speed things up? > > No. Maildir performance is limited by the disk head actuator speed, > which is between 150-300 seeks per second depending on your disk (7.2k > vs 15k RPM). Compressing the files doesn't change the seek physics of > the disk drives. You're still reading tens of thousands of files when > doing your searches thus bouncing the heads tens of thousands of times. > > mbox uses a single file, so head speed isn't a factor, as it may only > move a few times when reading an entire mailbox file. Thus, bandwidth > becomes the potential bottleneck. Using compression with large mbox > files can substantially increase search performance as effective > bandwidth is increased by ~4x using gzip and 6x using bzip2. This > assumes you have plenty of excess CPU power. mdbox should see similar > compression speedups if you use file sizes much larger than the 2MB > default. Doing so should keep your IOPS well below the drive's head > saturation point as you're reading only a fraction of the file count > compared to maildir. > >> I'd like to use mdbox as storage but for now it is very new for me and I >> simple afraid what should I do if I'll need to manually fix the storage >> (maildir is really good for that, surely). > > Doveadm handles such tasks pretty well. Just make sure you keep good > backups of your mdbox files. > >> After all, I simple need to speed up the search and restore process in >> archive. > > The only way to accomplish this with maildir is with much bigger, > faster, more expensive storage hardware. And the gain will still be > much less than simply switching to a larger file format such as mbox or > mdbox. > > As with many things some computer technologies come full circle over > time. One of the reasons the creators of the UNIX mbox mail file format > decided upon a single file many decades ago was the horribly limited > seek performance of the slow SCSI disks of that period. Doing something > like the maildir format was simply impossible at that time. In the > early days of the public internet, disk became faster than the average > load and maildir was born to fix the locking and corruption shortcomings > of mbox. > > Today many sites are hitting the seek problem of a few decades ago > because boxes are oversubscribed with users, emails now frequently > contain attachments, everyone is storing more email, and the total > volume of email is a few orders of magnitude greater. > > IIRC, this is one of the reasons Timo created mdbox--to decrease the > massive IOPS load, and thus slow performance, of large maildir stores. > > -- > Stan From steveh at itbasecamp.com.au Tue Nov 15 02:08:33 2011 From: steveh at itbasecamp.com.au (Steve H) Date: Tue, 15 Nov 2011 00:08:33 +0000 Subject: [Dovecot] XUIDL Migration problems Message-ID: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> Hello All, I am new to Dovecot and the Dovecot mailing list. I am trying to migrate an existing Exim + qpopper - mbox (pop3 only) installation over to Exim + Dovecot using mdir (pop3 and imap) And am running into problems with preserving the XUIDL headers. I am running Dovecot 2.0.14 I have used the mb2md.pl script : http://dovecot.org/tools/mb2md.pl from this page http://wiki2.dovecot.org/Migration/MailFormat to migrate and import the mboxes And have the following pop3 uidl settings in my configuration . pop3_reuse_xuidl = yes pop3_save_uidl = yes protocols = imap pop3 service imap-login { inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } protocol pop3 { pop3_reuse_xuidl = yes pop3_save_uidl = yes pop3_uidl_format = %08Xu%08Xv } The problem specifically is that when I test the migration and import a mailbox, I am able to connect via IMAP using Thunderbird and Outlook 2007 and the Read / Unread status is preserved. However when I connect via POP3 the Read/Unread status is not preserved. I am probably missing something simple, but have been unable to find anything that seems relevant on the Dovecot wiki or searching the mailing lists. Any suggestions would be appreciated. Regards, Steven Holloway IT Technical Consultant I.T. Basecamp P/L [http://www.itbasecamp.com.au/signature.gif] Address: Office 1 Level 1/50 Owen St Huskisson NSW 2540 Phone: +61 2 44418555 ext 308 Fax: +61 2 44418551 Web: http://www.itbasecamp.com.au Subscribe to our newsletter! Visit: IT Basecamp to manage your subscription. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 2602 bytes Desc: image001.gif URL: From tss at iki.fi Tue Nov 15 02:19:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:19:17 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC1271B.4060604@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> Message-ID: <23D65BA4-912A-43EE-8480-8586DEAD76FA@iki.fi> On 14.11.2011, at 16.35, Alexander Chekalin wrote: > 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if 2Mb. Looks like not even every message can fit into such storage container volume (nowadays we used to see messages of 20Mb and even more). The messages are never split into multiple files. So if you have a 20 MB message, it gets stored into its own m.* file. > Should I tune it (at least mdbox_rotate_size and mdbox_rotate_interval) or its size is on purpose? As for now I store each day's messages in separate IMAP folders (mailboxes), which gives me 2000-6000 messages and 2-5 Gb (on disk) per folder. The main problem with larger mdbox files is that if you expunge messages, there's more data to write when packing the data into a new file. I don't really know the "best" value for mdbox_rotate_size setting. But even a 2 MB mdbox file can contain thousands of small mails, so it's not too bad.. From tss at iki.fi Tue Nov 15 02:21:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:21:09 +0200 Subject: [Dovecot] XUIDL Migration problems In-Reply-To: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> References: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> Message-ID: <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> On 15.11.2011, at 2.08, Steve H wrote: > I am trying to migrate an existing Exim + qpopper - mbox (pop3 only) installation over to Exim + Dovecot using mdir (pop3 and imap) > And am running into problems with preserving the XUIDL headers. Are you sure the UIDLs are the problem? Did you connect to old and new POP3 server and compare their UIDL lists? > However when I connect via POP3 the Read/Unread status is not preserved. Another possibility would be simply that if you changed the hostname in the client config, it might have thought that it's a completely different server. From tss at iki.fi Tue Nov 15 02:38:15 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:38:15 +0200 Subject: [Dovecot] patch for UID 0 problem In-Reply-To: <4EC0E498.7000900@team.de.inter.net> References: <4EC0E498.7000900@team.de.inter.net> Message-ID: On 14.11.2011, at 11.51, Kai Arif wrote: > attached you'll find a patch for cmd-thread.c which addresses a problem with certain clients in > conjunction with the use of the UID THREAD REFS command: > > TAG5 UID THREAD REFS us-ascii SINCE 14-May-2011 > * THREAD > (0)(246)(247)(248)(249)(250)(252)(253)(254)(255)(256)(257)(258)(259)(260)(261)(262)(263)(264)(265)(266)(267)(268) > TAG5 OK Thread completed. > > The first UID delivered (0) is invalid, some clients - @mail for instance - then try to fetch the > invalid UID 0 subsequently and run into problems. Yes, it is. It should never happen. > The patch actually disables the return of (0) and seems to work for us. Please review and - if OK - > incorporate it in the next version. Could you instead send me such a mailbox where you can reproduce this problem? Probably sending dovecot.index, dovecot.index.log and dovecot.index.thread files would be enough. None of those contain any sensitive information. From steveh at itbasecamp.com.au Tue Nov 15 03:14:26 2011 From: steveh at itbasecamp.com.au (Steve H) Date: Tue, 15 Nov 2011 01:14:26 +0000 Subject: [Dovecot] XUIDL Migration problems In-Reply-To: <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> References: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> Message-ID: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8DB2@ITBEXCH.office.itb.net.au> Hi Timo, Thanks for the quick reply. > Are you sure the UIDLs are the problem? Did you connect to old and new POP3 server and compare their UIDL lists? No not 100% sure that the UIDLs are the issue; but running into dead ends with what I have tried. I will compare the UIDL lists from the old / new server and report back. > However when I connect via POP3 the Read/Unread status is not preserved. > Another possibility would be simply that if you changed the hostname in the client config, it might have thought that it's a completely different server. The new server is a completely different host and currently had a different IP, but it will end up with the same name and IP as the existing host when we cut over. At this time I have manually adjusted my lost hosts file to point towards the new host for testing. Would this be a problem even though the IMAP appears to be working were as the POP is not.?? I'll report back on the UIDLs Regards, Steven From mysqlstudent at gmail.com Tue Nov 15 04:30:35 2011 From: mysqlstudent at gmail.com (Alex) Date: Mon, 14 Nov 2011 21:30:35 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail Message-ID: Hi, I have a fedora15 system with dovecot-v2.0.15, dovecot-pigeonhole and and amavisd-new and would like to implement it on my system using postfix. I've read the basic wiki documentation, but I really can't figure out the proper starting point in my case. I've enabled "protocol lda" in 15-lda.conf and added "sieve" to the list of mail_plugins. I'm trying to adapt postfix to use deliver as its LDA for multiple virtual domains. I've added the following to master.cf: dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient} -a "${RECIPIENT}" Is that the proper format? Is it okay to use the dovecot user here, or should I create a new user? What permissions does this user need to possess? I've added the following to main.cf: dovecot_destination_recipient_limit = 1 virtual_transport = dovecot I have a number of procmail scripts and I've converted them to sieve, but they didn't convert very well. Can someone show me how to convert the following: :0 { :0 fhW | /home/alex/bin/process-mail.pl } The script skipped right over that. I'm also trying to convert the following: :0 * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) * !USER_IN_WHITELIST * !BAYES_00 * !URIBL_ * !KHOP_RCVD_UNTRUST * !RCVD_IN_DNSWL_MED * !SPF_SOFTFAIL * SPF_HELO_PASS * SPF_PASS { LOG="DNSWL: " :0: whitelist/dnswl } The script only produced the following: if true { if true { if true { if true { if true { if true { if true { if true { if true { if true { fileinto "{"; fileinto " LOG="DNSWL: ""; fileinto " :0:"; fileinto " whitelist/dnswl"; fileinto "}"; } Any ideas greatly appreciated. Thanks, Alex From JTRUTWIN at CSBSJU.EDU Tue Nov 15 05:34:55 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Tue, 15 Nov 2011 03:34:55 +0000 Subject: [Dovecot] getenv() in plugin not working Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> Hi - new to the list, can't find much on this using google. I'm trying to setup the dovecot DRAC plugin as described here: http://wiki.dovecot.org/HowTo/PopBSMTPAndDovecot#DRAC DRAC is installed and running using this startup command: /usr/local/sbin/rpc.dracd -i -e 5 /etc/postfix/dracd.db I downloaded the drac.c file linked on the above URL: http://www.dovecot.org/patches/1.1/drac.c and built it using this command: gcc -Wall -W -shared -fPIC -DHAVE_CONFIG_H drac.c -I/usr/include/dovecot -L/usr/lib64/dovecot -o drac.so -ldrac Then I copied drac.so to /usr/lib64/dovecot (this is an x86_64 SL 6.1 system - dovecot 2.0.9) and modified my /etc/dovecot/conf.d/20-imap.conf file: mail_plugin_dir = /usr/lib64/dovecot mail_plugins = drac Restart dovecot and try to login and getting these messages for ANY login attempts: dovecot: imap-login: Login: user=, method=PLAIN, rip=, lip=, mpid=4018, TLS server dovecot: imap(: Error: DRAC: IP environment not given I traced the "IP environment not given" to the drac.c plugin file, and the code is pretty simple: void drac_init(void) { const char *ip_str; struct ip_addr ip; ip_str = getenv("IP"); if (ip_str == NULL) i_error("DRAC: IP environment not given"); Is there anything special that needs to be done for getenv("IP") to report the login IP? I tried to put getenv("RIP") for remote ip and even getenv("USER") as well but always get NULL. Is this API only for dovecot 1.x? If so is there a 2.0 equivalent? The rest of the plugin is very basic... I think this is the last step in getting this setup running. I'd have to have this one last piece force me to run that icky perl program instead. Thank you, Let me know if anything else is needed. Josh From JTRUTWIN at CSBSJU.EDU Tue Nov 15 05:58:55 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Tue, 15 Nov 2011 03:58:55 +0000 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> > I'm trying to setup the dovecot DRAC plugin as described here: > Is there anything special that needs to be done for getenv("IP") to report the > login IP? I tried to put getenv("RIP") for remote ip and even getenv("USER") > as well but always get NULL. Is this API only for dovecot 1.x? If so is there > a 2.0 equivalent? The rest of the plugin is very basic... Appears this is the problem - I found this for dovecot v2 and it seems to be working fine: http://sourceforge.jp/projects/dovecot2-drac/ Might wanna make a note on the wiki page that the DRAC setup is only for dovecot 1.x? Thanks, Josh From stephan at rename-it.nl Tue Nov 15 09:41:37 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 15 Nov 2011 08:41:37 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: Message-ID: <4EC217B1.2040809@rename-it.nl> On 11/15/2011 3:30 AM, Alex wrote: > I have a number of procmail scripts and I've converted them to sieve, > but they didn't convert very well. Can someone show me how to convert > the following: > > :0 > { > :0 fhW > | /home/alex/bin/process-mail.pl > } > > The script skipped right over that. The Sieve language doesn't support invoking external programs. You can execute this program before Sieve, for instance by wrapping the LDA invocation. Also, there is a non-standard (an still pretty much experimental) language extension for this, as provided by the pipe plugin: http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe > I'm also trying to convert the following: > > :0 > * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) > * !USER_IN_WHITELIST > * !BAYES_00 > * !URIBL_ > * !KHOP_RCVD_UNTRUST > * !RCVD_IN_DNSWL_MED > * !SPF_SOFTFAIL > * SPF_HELO_PASS > * SPF_PASS > { > LOG="DNSWL: " > :0: > whitelist/dnswl > } > > The script only produced the following: [...] > Any ideas greatly appreciated. I'm not sure what these rules are supposed to mean (most conditions are obscured in variables), but by the looks of things these should also be executed outside Sieve as well. If all those conditions map to simple header and envelope tests, you can provide a Sieve alternative. However, I think USER_IN_WHITELIST queries some external list and that too is not (currently) possible from Pigeonhole Sieve. Also, it is not possible to change the log prefix from Sieve (as suggested by LOG=). Regards, Stephan From CMarcus at Media-Brokers.com Tue Nov 15 13:22:53 2011 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 15 Nov 2011 06:22:53 -0500 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> Message-ID: <4EC24B8D.9010401@Media-Brokers.com> On 2011-11-14 10:58 PM, Trutwin, Joshua wrote: > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? Why? You were on the wiki for v1, so obviously it is (or may be) only meant for v1. -- Best regards, Charles From bind at enas.net Tue Nov 15 17:02:22 2011 From: bind at enas.net (Urban Loesch) Date: Tue, 15 Nov 2011 16:02:22 +0100 Subject: [Dovecot] Question about pop3_reuse_xuidl Message-ID: <4EC27EFE.1060504@enas.net> Hi, we are in migration progress from Communigate Pro 5.0.x to Dovecot 2.0.15 with mdbox. We still migrated about 25.000 IMAP accounts from CGP to Dovecot. Also there were still added about 2000 new POP3 Accounts on Dovecot. Now we must migrate about 10.000 POP3 accounts from CGP to Dovecot. At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, or does Dovecot only change the uidl value for new received and still not downloaded mails? Many thanks and regards Urban Loesch From mysqlstudent at gmail.com Tue Nov 15 17:38:44 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 10:38:44 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <4EC217B1.2040809@rename-it.nl> References: <4EC217B1.2040809@rename-it.nl> Message-ID: Hi, >> I have a number of procmail scripts and I've converted them to sieve, >> but they didn't convert very well. Can someone show me how to convert >> the following: >> >> :0 >> { >> ? ? :0 fhW >> ? ? | /home/alex/bin/process-mail.pl >> } >> >> The script skipped right over that. > > The Sieve language doesn't support invoking external programs. You can > execute this program before Sieve, for instance by wrapping the LDA > invocation. Can you describe how I might do this? > Also, there is a non-standard (an still pretty much > experimental) language extension for this, as provided by the pipe plugin: Is there a way to just continue to use procmail? >> I'm also trying to convert the following: >> >> :0 >> * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) >> * !USER_IN_WHITELIST >> * !BAYES_00 >> * !URIBL_ >> * !KHOP_RCVD_UNTRUST >> * !RCVD_IN_DNSWL_MED >> * !SPF_SOFTFAIL >> * SPF_HELO_PASS >> * SPF_PASS >> { >> ? ? ? ? LOG="DNSWL: " >> ? ? ? ? :0: >> ? ? ? ? whitelist/dnswl >> } >> >> The script only produced the following: > > [...] >> >> Any ideas greatly appreciated. > > I'm not sure what these rules are supposed to mean (most conditions are > obscured in variables), but by the looks of things these should also be > executed outside Sieve as well. If all those conditions map to simple header > and envelope tests, you can provide a Sieve alternative. However, I think > USER_IN_WHITELIST queries some external list and that too is not (currently) > possible from Pigeonhole Sieve. Also, it is not possible to change the log > prefix from Sieve (as suggested by LOG=). No, these are all simple header matches. There's no external program being executed here. The LOG= is also a procmail built-in, but I can do without that. Thanks again, Alex From stephan at rename-it.nl Tue Nov 15 18:49:00 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 15 Nov 2011 17:49:00 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> Message-ID: <4EC297FC.8050405@rename-it.nl> Op 15-11-2011 16:38, Alex schreef: > Hi, > >> The Sieve language doesn't support invoking external programs. You can >> execute this program before Sieve, for instance by wrapping the LDA >> invocation. > Can you describe how I might do this? You can write a shell script that is executed from your MTA instead of the Dovecot LDA. Perform the process-mail task from within that script and then call the Dovecot LDA from the shell script (and pipe the message to it). >> Also, there is a non-standard (an still pretty much >> experimental) language extension for this, as provided by the pipe plugin: > Is there a way to just continue to use procmail? Yes, you can pipe the message to the Dovecot LDA from procmail. This would be similar to the solution described above, only using Procmail instead of a shell script. Regards, Stephan. From mysqlstudent at gmail.com Tue Nov 15 20:09:39 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 13:09:39 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <4EC297FC.8050405@rename-it.nl> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> Message-ID: Hi, >>> The Sieve language doesn't support invoking external programs. You can >>> execute this program before Sieve, for instance by wrapping the LDA >>> invocation. >> >> Can you describe how I might do this? > > You can write a shell script that is executed from your MTA instead of the > Dovecot LDA. Perform the process-mail task from within that script and then > call the Dovecot LDA from the shell script (and pipe the message to it). So you mean from master.cf, in the same way as deliver is being executed? dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient} -a "${RECIPIENT}" Replacing deliver here with a shell script? Then execute deliver from within the shell script, or just do the delivery from within a shell script altogether? >> Is there a way to just continue to use procmail? > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > be similar to the solution described above, only using Procmail instead of a > shell script. Are you referring to doing something like this: http://wiki2.dovecot.org/procmail I just don't understand how to invoke procmail from deliver. My main motivation for trying to continue to use procmail is that the sieve language seems far more complicated and involved than even a procmail recipe, and isn't as fully developed. I don't want to make this even more complicated, however. Perhaps you can explain how to filter a basic message based on something in the body or a header and sort it into a file? Thanks so much for your help. Best, Alex From stan at hardwarefreak.com Tue Nov 15 20:26:38 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 15 Nov 2011 12:26:38 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <43B341A1-9245-4863-932A-887E21970319@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> Message-ID: <4EC2AEDE.2080306@hardwarefreak.com> On 11/14/2011 3:16 PM, Alexander Chekalin wrote: > Locking issues on mbox is the reason for my long-lasting love affair with maildir, Same reason most others fell in love with it. Many now want to divorce maildir, as the cost of the storage to maintain acceptable performance is now too high. > and it's lasts long years. Ok, the life's lessons are like this, learn something and move on with it ;) even if it's "new old thing". Thank you for pointing that! Many old UNIX gurus still use mbox, not maildir, and never will. If you ask them why they'll likely say "you don't use a screwdriver to drive a nail do you?" > What I was doubt about is default rotate size of 2M, since I used to see pretty reasonable default settings in all Dovecot config. 32 or 64 are much close to the ones I'd personally prefer. Given the fact that we're talking about an archive server, you'd be better off using a very large mdbox file size, say 1GB. You're never deleting individual messages from this archive correct? No expunges? This is why I recommended mbox in the first place. If your only writes to these mailbox files are appends of new messages, mbox is the best format by far. It's faster at appending than any other format, and it's faster for searching than any other. > I also about to choose now is the OS and FS for the archive. I seriously think about ZFS with compression (in fact it will be stripes over couple of mirrors = software equivalent of RAID 10 on SATA drives, with compression on FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox itself. I see pros and contras for both, so that's the question to answer! It's an archive. You're not going to use maildir so you don't need random IOPS performance. Thus RAID5/6 are a much better fit for an archive as you get better read performance, with more than adequate write performance, and you use less disks. And as this is an archive, you don't need real time automatic/transparent compression. Thus I recommend something like: 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled 2.6.39 or later kernel 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 Specify a strip size of 256KB for the array Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading ahead 1024 sectors at a time instead of the default of 256. This will speed up your searches quite a bit. 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults 4. mbox w/zlib plugin. Compress daily files each night with a script 5. You don't need LVM with a good RAID card (or with mdraid). This controller can expand the RAID5 up to 8 drives (up to 32 drives max using SAS expanders) -- Stan From tss at iki.fi Tue Nov 15 21:02:43 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:02:43 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC2AEDE.2080306@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> Message-ID: <1321383763.21919.484.camel@hurina> On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. Just as long as you're not simultaneously trying to read and write the mbox file (or just write in 2+ sessions). Then there's a lot waiting on locks. (mdbox has no read locks, and its write locks are very short lived.) From micah at riseup.net Tue Nov 15 21:24:18 2011 From: micah at riseup.net (Micah Anderson) Date: Tue, 15 Nov 2011 14:24:18 -0500 Subject: [Dovecot] dsync problems Message-ID: <871ut9i3z1.fsf@algae.riseup.net> When a user renames their username, I am using dsync to copy their mail over to the new username's mail location[0]. Some of the dsyncs are failing with errors that I dont know how to work with, for example: dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=42fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=43fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=44fcd40303c8a64e432300007ef44c7a ERROR: dsync failed, exit code: 256 The errors cause a non-zero exit code from dsync, which causes my rename script to bail out. What are these errors, and how can I fix them? Thanks, micah 0. Why use dsync instead of a simple mv operation? This seems to be necessary for two corner cases: 1. dovecot creates the new mailbox automatically when the user logs in or receives a mail, so if the user changes their mail and logs in or receives an email before the move has been done, then the mailbox is created and then a move command will fail. 2. If there has been new mail created under the new name, we can't just simply remove the stuff that is automatically created there and replace it with the old things because we could potentially be removing mail that has been delivered in the mean time. I'd be really interested if people had suggestions for a better mechanism, or perhaps a way to have dovecot not create the new mail location automatically. -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Tue Nov 15 21:25:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:25:55 +0200 Subject: [Dovecot] Question about pop3_reuse_xuidl In-Reply-To: <4EC27EFE.1060504@enas.net> References: <4EC27EFE.1060504@enas.net> Message-ID: <1321385157.21919.497.camel@hurina> On Tue, 2011-11-15 at 16:02 +0100, Urban Loesch wrote: > At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. > Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? > > Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, Yes. > or does Dovecot only change the uidl value for new received and still not downloaded mails? Nope.. There is pop3_save_uidl=yes setting, which would do this for Maildir, assuming the POP3 clients had logged in while this option was enabled, but this isn't implemented for dbox. The biggest problem with implementing this for dbox would be that it has no variable length permanent per-mail storage (Maildir's dovecot-uidlist equivalent). One way to solve this would be by patching dsync a little bit, and then dsyncing existing mdbox to mdbox-new, which adds existing POP3 UIDLs as dbox metadata. Those will override X-UIDL: headers. The patching would be basically by making dsync/dsync-local-worker.c local_worker_msg_get_next() always set data.pop3_uidl variable to the correct UIDL (which would depend on your pop3_uidl_format setting - pop3/pop3-commands.c:pop3_get_uid() shows how). From tss at iki.fi Tue Nov 15 21:30:37 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:30:37 +0200 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> Message-ID: <1321385437.21919.498.camel@hurina> On Tue, 2011-11-15 at 03:58 +0000, Trutwin, Joshua wrote: > Appears this is the problem - I found this for dovecot v2 and it seems to be > working fine: > > http://sourceforge.jp/projects/dovecot2-drac/ > > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? I changed the wiki2 link to this URL. From mgoldey at goldey.net Tue Nov 15 21:31:50 2011 From: mgoldey at goldey.net (Mark Goldey) Date: Tue, 15 Nov 2011 14:31:50 -0500 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 Message-ID: <4EC2BE26.3030807@goldey.net> Folks: I've been running Dovecot 2.0.7 on a Qmail / Vpopmail installation built on Debian Squeeze, with no problems for about a year. Dovecot is used for IMAP/S auth only. Dovecot is built from source. I just upgraded to 2.0.15 but it segfaults whenever an IMAP auth is attempted, regardless of the user who tries to login, whether by webmail (Roundmail) Android phone (K-9 Mail) or iPad (Roundmail) I downgraded to 2.0.13 but the segfault persisted. Re-installed 2.0.7 and that cleared it right up. Here's the pertinent details from 2.0.15. Anything else I can provide? root at XXX:/# uname -a Linux XXX 2.6.32-5-686 #1 SMP Wed Jan 12 04:01:41 UTC 2011 i686 GNU/Linux root at XXX:/# dovecot -n # 2.0.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.3 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no first_valid_uid = XXXX last_valid_uid = XXXX log_path = /var/log/dovecot.log mail_debug = yes passdb { driver = vpopmail } protocols = imap service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 128 vsz_limit = 64 M } ssl_cert = , method=PLAIN, rip=174.79.xxx.xxx, lip=192.xxx.xxx.xxx, mpid=691, TLS Nov 15 14:08:43 imap-login: Warning: SSL alert: where=0x4008, ret=256: warning close notify [174.79.xxx.xxx] Nov 15 14:08:43 auth-worker: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth I ought to mention that /usr/local/lib/dovecot/auth is empty. Here's the backtrace: root at XXX:/# gdb /usr/local/libexec/dovecot/auth /usr/local/var/run/dovecot/core GNU gdb (GDB) 7.0.1-debian [snip] Reading symbols from /usr/local/libexec/dovecot/auth...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/i686/cmov/libcrypt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/libcrypt-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libcrypt.so.1 Reading symbols from /home/vpopmail/lib/libvpopmail.so...(no debugging symbols found)...done. Loaded symbols for /home/vpopmail/lib/libvpopmail.so Reading symbols from /lib/i686/cmov/libdl.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libdl-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libdl.so.2 Reading symbols from /lib/i686/cmov/librt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/librt-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/librt.so.1 Reading symbols from /lib/i686/cmov/libc.so.6...Reading symbols from /usr/lib/debug/lib/i686/cmov/libc-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libc.so.6 Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/i686/cmov/libpthread.so.0...Reading symbols from /usr/lib/debug/lib/i686/cmov/libpthread-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libpthread.so.0 Reading symbols from /home/vpopmail/etc/vpopmail.authmodule...(no debugging symbols found)...done. Loaded symbols for /home/vpopmail/etc/vpopmail.authmodule Core was generated by `dovecot/auth -w'. Program terminated with signal 11, Segmentation fault. #0 0x00000000 in ?? () (gdb) bt full #0 0x00000000 in ?? () No symbol table info available. #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at auth-worker-client.c:405 auth_request = 0x82b56e8 userdb_id = 1 #2 auth_worker_handle_line (client=0x82b34d0) at auth-worker-client.c:534 id = 2 #3 auth_worker_input (client=0x82b34d0) at auth-worker-client.c:614 _data_stack_cur_id = 3 line = ret = 232 #4 0xb785f021 in io_loop_call_io (io=0x82b3688) at ioloop.c:384 ioloop = 0x82a6398 t_id = 2 #5 0xb78600d3 in io_loop_handler_run (ioloop=0x82a6398) at ioloop-epoll.c:213 ctx = 0x82b1f38 event = 0x82b1f78 list = 0x82b36b0 io = 0x829e2b0 tv = {tv_sec = 2147483, tv_usec = 0} msecs = 1 ret = 1 i = 0 j = 0 call = 192 #6 0xb785efa0 in io_loop_run (ioloop=0x82a6398) at ioloop.c:405 No locals. #7 0xb784abaa in master_service_run (service=0x82a62e8, callback=0x805c470 ) at master-service.c:481 No locals. #8 0x0805c828 in main (argc=2, argv=0x82a61c0) at main.c:298 c = (gdb) --mhg From tss at iki.fi Tue Nov 15 21:35:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:35:03 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> Message-ID: <1321385703.21919.501.camel@hurina> On Tue, 2011-11-15 at 13:09 -0500, Alex wrote: > > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > > be similar to the solution described above, only using Procmail instead of a > > shell script. > > Are you referring to doing something like this: > > http://wiki2.dovecot.org/procmail > > I just don't understand how to invoke procmail from deliver. The idea was to invoke deliver from procmail. I still do this myself since I haven't bothered changing my procmail scripts to Sieve: DELIVER=/usr/local/libexec/dovecot/deliver :0 * ^List-Id.*dovecot.dovecot.org | $DELIVER -m dovecot :0 | $DELIVER etc. From tss at iki.fi Tue Nov 15 21:39:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:39:50 +0200 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: Message-ID: <1321385991.21919.503.camel@hurina> On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote: > > Thanks to a fortuitously unrelated thread ("how to disable quota for > > second namespace"), I got the quota part figured out and that seems to > > be working: Add a second entry to plugin {}, e.g. "quota2 = > > maildir:Archive quota:ns=INBOX.Archives." and add rules for > > userdb_quota2_rule, userdb_quota2_rule2, etc. > > > > My real question now is: Are there any fatal gotchas in this that I'm > > just not thinking of? > > > > Haven't had a chance to try this large-scale yet. Anybody have any > thoughts on it? The gotcha is that you have two completely independent quotas with independent usage/limits for the INBOX and Archive namespaces. If that is what you want, it should all be fine. From tss at iki.fi Tue Nov 15 21:44:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:44:20 +0200 Subject: [Dovecot] Chroot In-Reply-To: References: Message-ID: <1321386260.21919.505.camel@hurina> On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote: > all maildirs are located in: /var/mail/jdoe at example.com/Maildir So this is /var/mail/%u/Maildir template. > namespace { > type = shared > separator = / > prefix = shared/%%u/ > location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u You can replace this with: location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u Although if this is used outside chroot (e.g. lmtp, doveadm) it won't work correctly.. > Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No > such file or directory This is done because you used %%h, to look up home directory from userdb. If you don't use %%h, it's not done. From tss at iki.fi Tue Nov 15 21:49:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:49:20 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: Message-ID: <1321386560.21919.509.camel@hurina> On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > I configured dovecot to use submission smtp host becouse of chroot. > > submission_host = 127.0.0.1 > > Unfortunatelly: > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > synchronization error (next input sent too soon: pipelining was not > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > input="MAIL FROM:\\r\\n" This basically says that Dovecot's SMTP client is being noncompliant, because Exim doesn't advertise PIPELINING extension. But I'd rather not add more code to fix this, since PIPELINING gives you a little bit better performance anyway and you can most likely fix this by modifying Exim's configs in some way (I can't believe Exim wouldn't support PIPENING..). From tss at iki.fi Tue Nov 15 22:16:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:16:14 +0200 Subject: [Dovecot] Migration From Courier Imap. In-Reply-To: <653a51f07131e61c52bf983359f4710c@192.168.0.251> References: <653a51f07131e61c52bf983359f4710c@192.168.0.251> Message-ID: <1321388180.21919.515.camel@hurina> On Sat, 2011-11-12 at 12:58 +0530, Rajeev Sharma wrote: > I am using vpopmail,qmail along with courier-imap 4.0 i want to > migrate from courier to dovecot but i have 100 shared folder > under /home/vpopmail/domains/domain.com/Malidir and user shared them > on the basis of rights assign. > > I am very much clear how to migrate from courier to dovecot but i am > unable how to migrate shared folder and assign rights. I don't think there is any existing documentation about this. I've also not used Courier shared folders myself much. I think the ACL files' contents are compatible, although Dovecot names them differently. IIRC Courier used some kind of symlinked maildir files for shared folders to provide per-user flags? Migrating those to Dovecot would be a bit tricky, especially because Dovecot currently supports only per-user \Seen flags (maybe Courier does too? I'm not sure) and Dovecot stores the per-user \Seen flags only in per-user index files. Anyway, I don't really have time to look into details of this, so you'll just need to figure out the specifics yourself (or alternatively pay me to do that via my company). From tss at iki.fi Tue Nov 15 22:24:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:24:57 +0200 Subject: [Dovecot] Invalid "doveadm search" queries not rejected In-Reply-To: <1320997281.2261.116.camel@iota.localdomain> References: <1320997281.2261.116.camel@iota.localdomain> Message-ID: <1321388697.21919.518.camel@hurina> On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > 6). > > According to the doveadm-search-query(7) manual page, a "mailbox" > specification can only come at the beginning of a search query. It doesn't exactly say that, but yeah, it doesn't also include MAILBOX and MAILBOX-GUID as search keys. Added now to future versions. > However, > if I put a mailbox specification in the middle of a query, not only do I > not get an error message, I get surprising results: > > $ doveadm search mailbox A |wc -l > 6979 > $ doveadm search mailbox B |wc -l > 206 > $ doveadm search mailbox A OR mailbox B |wc -l > 42492 I'm pretty sure this was fixed in v2.0.14. > If it became possible to do boolean logic on mailbox queries, that'd be > cool. It is. From tss at iki.fi Tue Nov 15 22:27:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:27:20 +0200 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <874nybztuz.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> Message-ID: <1321388840.21919.520.camel@hurina> On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > I really like the feature where you can define quota rules with percents > which trigger off of the default values[0] (so you can set the Trash to > allow for 10% more of the user's quota for example). > > What I would really love in dovecot would be for the ability to > configure a quota rule for dovecot-lda. I would like to configure things > so we don't bounce emails for users until they are well over quota, the > IMAP quota plugin is a really great way to notify people that they are > over quota because it fails to write to other folders.... that should be > enough to get people's attention that they need to deal with things, but > bouncing is harsh. You can give LDA-specific quota settings. Like maybe: protocol lda { plugin { quota_rule2 = INBOX:storage=200% } } or even not enforcing quota at all: protocol lda { plugin { quota = maildir:User quota:noenforcing } } From tss at iki.fi Tue Nov 15 22:28:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:28:57 +0200 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <4EBBB22A.90501@bsc.es> References: <4EBA4DD4.4080505@bsc.es> <1320861378.21919.376.camel@hurina> <4EBBB22A.90501@bsc.es> Message-ID: <1321388937.21919.521.camel@hurina> On Thu, 2011-11-10 at 12:14 +0100, David Ocana wrote: > >> namespace { > >> separator = / > >> prefix = INBOX/ > >> location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n > >> inbox = yes > >> hidden = yes > >> } > >> > >> plugin { > >> quota = dirsize:User quota > > > > quota = dirsize:User quota:ns=INBOX/ > > Actually I forgot to mention that I also tried that, but I got the > following error: > > Error: Initialization failed: Failed to initialize quota: Quota root > User quota: Unknown parameter for backend dirsize: ns=INBOX/ Oh. Yeah, dirsize backend doesn't have all the features that other backends have, because it's so little used. You could switch to dict file quota.. From tss at iki.fi Tue Nov 15 22:30:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:30:16 +0200 Subject: [Dovecot] Limit the number of pop/imap requests In-Reply-To: References: Message-ID: <1321389016.21919.522.camel@hurina> On Thu, 2011-11-10 at 10:40 +0100, wwl wrote: > Hello Group, > > is it possible to Limit the number of pop/imap requests to Dovecot? > > I read the Wiki and found > > max_mail_processes = 512 > > is this the way to limit this ? This limits the number of simultaneous IMAP and POP3 connections. Is that what you mean with "requests"? If so, yeah, you can reduce this value. I think the counts are tracked separately for IMAP and POP3 (so you can have 512 IMAP and 512 POP3 connections). From tss at iki.fi Tue Nov 15 22:32:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:32:08 +0200 Subject: [Dovecot] indexes permissions problem In-Reply-To: <4EBB8BA7.9070603@cruzio.com> References: <4EBB8BA7.9070603@cruzio.com> Message-ID: <1321389128.21919.523.camel@hurina> On Thu, 2011-11-10 at 00:30 -0800, Mark Hanford wrote: > I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). > For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box > with user home directories NFS mounted), my index files are setup to be > in /u/indexes/ > > On the Dovecot 1.1.1 installation, the perms on the indexes directory is > 777 with root:mail ownership. > > The same thing on the Dovecot 2 / Centos server results in a 'permission > denied' error when Dovecot tries to create files. What exactly is the error message? Copy&paste it entirely. > So, I guess my main question is, what perms and ownership should > /u/indexes be set to? I've tried several different things before this > cry for help... Those permissions look ok to me. It's possible that this was a bug in 2.0.beta6 and upgrading might help (but might not). From tss at iki.fi Tue Nov 15 22:34:21 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:34:21 +0200 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: References: Message-ID: <1321389261.21919.524.camel@hurina> On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > I am familiar with this article > http://wiki.dovecot.org/LDA#multipleuids > > I want to disable setuid-root, but I don't know the default group and > permissions. > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > but my delivery is still failing > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, > orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) dovecot-lda's group should be the same as you specify to the pipe in Postfix's master.cf. From tss at iki.fi Tue Nov 15 22:35:36 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:35:36 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <20111105194350.GA3782@state-of-mind.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <20111105194350.GA3782@state-of-mind.de> Message-ID: <1321389336.21919.525.camel@hurina> On Sat, 2011-11-05 at 20:43 +0100, Patrick Ben Koetter wrote: > * Timo Sirainen : > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > > > is there a way to use Dovecot SASL Authentication for submission on a > > > different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > > > > Dovecot machine <-> postfix machine(for submission) > > > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. > > Document it and Wietse will support it on the mailing list. If its not > documented support will be declined. You mean document in Dovecot wiki or for Postfix? > Should I? Please do :) From tss at iki.fi Tue Nov 15 22:39:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:39:13 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC2BE26.3030807@goldey.net> References: <4EC2BE26.3030807@goldey.net> Message-ID: <1321389553.21919.527.camel@hurina> On Tue, 2011-11-15 at 14:31 -0500, Mark Goldey wrote: Could you do a bit more with gdb and this core: > #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at > auth-worker-client.c:405 > auth_request = 0x82b56e8 > userdb_id = 1 fr 1 p *auth_request->userdb->userdb p *auth_request->userdb->userdb->iface From tss at iki.fi Tue Nov 15 22:43:24 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:43:24 +0200 Subject: [Dovecot] dsync problems In-Reply-To: <871ut9i3z1.fsf@algae.riseup.net> References: <871ut9i3z1.fsf@algae.riseup.net> Message-ID: <1321389804.21919.530.camel@hurina> On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > When a user renames their username, I am using dsync to copy their mail > over to the new username's mail location[0]. > > Some of the dsyncs are failing with errors that I dont know how to work > with, for example: > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a These shouldn't really happen. Something's going internally wrong with dsync. Can you reproduce this reliably somehow? > The errors cause a non-zero exit code from dsync, which causes my rename > script to bail out. What are these errors, and how can I fix them? Does a second dsync on error succeed? :) > 0. Why use dsync instead of a simple mv operation? This seems to be > necessary for two corner cases: > > 1. dovecot creates the new mailbox automatically when the user logs in > or receives a mail, so if the user changes their mail and logs in or > receives an email before the move has been done, then the mailbox is > created and then a move command will fail. > > 2. If there has been new mail created under the new name, we can't just > simply remove the stuff that is automatically created there and replace > it with the old things because we could potentially be removing mail > that has been delivered in the mean time. You could temporarily change the permissions for the home directory so that no new mailboxes/mails could be created during the move (e.g. 0700 root). From mrvjtod at gmail.com Tue Nov 15 23:13:13 2011 From: mrvjtod at gmail.com (Chris Young) Date: Tue, 15 Nov 2011 16:13:13 -0500 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: <1321389261.21919.524.camel@hurina> References: <1321389261.21919.524.camel@hurina> Message-ID: Thanks for the response. I was able to get it working with chmod 755 On Tue, Nov 15, 2011 at 3:34 PM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > > I am familiar with this article > > http://wiki.dovecot.org/LDA#multipleuids > > > > I want to disable setuid-root, but I don't know the default group and > > permissions. > > > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > > > but my delivery is still failing > > > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=< > user at domain.com>, > > orig_to=, relay=dovecot, delay=0.05, > delays=0.04/0/0/0.01, > > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: > fatal: > > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied > ) > > dovecot-lda's group should be the same as you specify to the pipe in > Postfix's master.cf. > > > From mysqlstudent at gmail.com Wed Nov 16 00:14:50 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 17:14:50 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321385703.21919.501.camel@hurina> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> I just don't understand how to invoke procmail from deliver. > > The idea was to invoke deliver from procmail. I still do this myself > since I haven't bothered changing my procmail scripts to Sieve: > > DELIVER=/usr/local/libexec/dovecot/deliver > > :0 > * ^List-Id.*dovecot.dovecot.org > | $DELIVER -m dovecot > > :0 > | $DELIVER How do I tell dovecot to use procmail so that it may invoke deliver? Thanks again, Alex From tss at iki.fi Wed Nov 16 00:27:12 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 00:27:12 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 0.14, Alex wrote: >>> I just don't understand how to invoke procmail from deliver. >> >> The idea was to invoke deliver from procmail. I still do this myself >> since I haven't bothered changing my procmail scripts to Sieve: >> >> DELIVER=/usr/local/libexec/dovecot/deliver >> >> :0 >> * ^List-Id.*dovecot.dovecot.org >> | $DELIVER -m dovecot >> >> :0 >> | $DELIVER > > How do I tell dovecot to use procmail so that it may invoke deliver? You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc. From nick.z.edwards at gmail.com Wed Nov 16 00:48:19 2011 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 16 Nov 2011 08:48:19 +1000 Subject: [Dovecot] Excessive imap-logins, never die Message-ID: Hiya, Today I note there are 38 imap-logins that seem to be stale on our staff imap server, they do not seem to die off, new imap-logins are created with new logins and exit accordingly. Now, these are from our monitoring software that runs on another server on our LAN. Our corporate link was taken down for 2 hours for urgent relocation of fibre (demolishing old adjoining building, and just in case, they needed to move it which we were glad about), so, this begs the question, why would imap-logins be there I guess in some sort of wait state, even with a still working internal DNS (internal view as well) the only difference is no actual internet connection in this time. A week ago when they first started to working and we lost internet for 45 minutes I also saw a few extra stale imap-logins, but after todays events, I decided this is a bigger issue that needs to be looked into, as dovecot required a restart to clear it all (I wasnt going to sit there killing individual PIDs all evening). Dovecot version 1.2.17, built from source From tss at iki.fi Wed Nov 16 00:50:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 00:50:25 +0200 Subject: [Dovecot] Excessive imap-logins, never die In-Reply-To: References: Message-ID: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> On 16.11.2011, at 0.48, Nick Edwards wrote: > Today I note there are 38 imap-logins that seem to be stale on our staff > imap server, > they do not seem to die off, new imap-logins are created with new logins > and exit accordingly. .. > Dovecot version 1.2.17, built from source All this code has been rewritten in v2.0 and there's not much point in spending time on figuring out why things aren't working perfectly in v1.x. But you could most likely avoid this problem by switching to "high-performance mode": http://wiki.dovecot.org/LoginProcess From nick.z.edwards at gmail.com Wed Nov 16 01:28:07 2011 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 16 Nov 2011 09:28:07 +1000 Subject: [Dovecot] Excessive imap-logins, never die In-Reply-To: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> References: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> Message-ID: On Wed, Nov 16, 2011 at 8:50 AM, Timo Sirainen wrote: > On 16.11.2011, at 0.48, Nick Edwards wrote: > > > Today I note there are 38 imap-logins that seem to be stale on our staff > > imap server, > > they do not seem to die off, new imap-logins are created with new logins > > and exit accordingly. > .. > > Dovecot version 1.2.17, built from source > > All this code has been rewritten in v2.0 and there's not much point in > spending time on figuring out why things aren't working perfectly in v1.x. > But you could most likely avoid this problem by switching to > "high-performance mode": http://wiki.dovecot.org/LoginProcess > > Okay, we do use high security mode specifically, and we require SSL on that server, I might just leave it as is, if there is no timeout value for these which I couldn't see when googling earlier today. I want to look at v2, but need to wait until all staff on are Christmas holidays, so if upgrade goes pair shaped, I can restore and not get bombarded with phone calls from management who can't get their mail. From mysqlstudent at gmail.com Wed Nov 16 02:00:13 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 19:00:13 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >>>> I just don't understand how to invoke procmail from deliver. >>> >>> The idea was to invoke deliver from procmail. I still do this myself >>> since I haven't bothered changing my procmail scripts to Sieve: >>> >>> DELIVER=/usr/local/libexec/dovecot/deliver >>> >>> :0 >>> * ^List-Id.*dovecot.dovecot.org >>> | $DELIVER -m dovecot >>> >>> :0 >>> | $DELIVER >> >> How do I tell dovecot to use procmail so that it may invoke deliver? > > You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc. Okay, I'm making progress. I disabled the virtual_domain stuff in main.cf for postfix and just enabled mail_command=procmail. I think there's something wrong with the deliver command. It's now delivering, but my deliver command must be wrong. This is my ~/.procmailrc: # Log an abstract of all messages LOGABSTRACT=all PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/lib:/usr/local/bin LOGFILE=$HOME/.maillog MAILDIR=$HOME/mail PROCHOME=$HOME/.procmail SENDMAIL=/usr/sbin/sendmail VERBOSE=yes DELIVER=/usr/libexec/dovecot/deliver :0 fhW | $DELIVER -m xspamtest :0 fhw | $DELIVER When a mail is received, it prints this to .maillog: procmail: [23382] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: [23383] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Assigning "LASTFOLDER=/var/spool/mail/alexuser" procmail: Opening "/var/spool/mail/alexuser" procmail: Acquiring kernel-lock procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Unlocking "/var/spool/mail/alexuser.lock" >From prvs=293a46ea3=mm at mydomain.com Tue Nov 15 18:56:04 2011 Subject: Grand Manor, Corvallis, OR, USA, AUTOMATED Extrane Folder: /var/spool/mail/bcc-user 17025 It delivers to the mail spool, but executing deliver as its printed in the procmailrc: # # /usr/libexec/dovecot/deliver -m spamtest lda(root): Fatal: destination user parameter (-d user) not given Am I doing something wrong? Thanks, Alex From tss at iki.fi Wed Nov 16 02:03:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 02:03:16 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 2.00, Alex wrote: > It's now delivering, but my deliver command must be wrong. This is my > ~/.procmailrc: .. > # # /usr/libexec/dovecot/deliver -m spamtest > lda(root): Fatal: destination user parameter (-d user) not given > > Am I doing something wrong? Is it trying to deliver mails to root user? That's not supported. From moseleymark at gmail.com Wed Nov 16 02:04:46 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 15 Nov 2011 16:04:46 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: <1321385991.21919.503.camel@hurina> References: <1321385991.21919.503.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 11:39 AM, Timo Sirainen wrote: > On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote: > >> > Thanks to a fortuitously unrelated thread ("how to disable quota for >> > second namespace"), I got the quota part figured out and that seems to >> > be working: Add a second entry to plugin {}, e.g. "quota2 = >> > maildir:Archive quota:ns=INBOX.Archives." and add rules for >> > userdb_quota2_rule, userdb_quota2_rule2, etc. >> > >> > My real question now is: Are there any fatal gotchas in this that I'm >> > just not thinking of? >> > >> >> Haven't had a chance to try this large-scale yet. Anybody have any >> thoughts on it? > > The gotcha is that you have two completely independent quotas with > independent usage/limits for the INBOX and Archive namespaces. If that > is what you want, it should all be fine. Nope, that's totally fine. The idea is to put Archive on cheaper (slower) storage and then grant more generous quotas there to make it worth their while to use, without slowing down their Inbox. Another application would be to put their Spam in another namespace (for people who choose to have it put in a separate folder) with a lower quota, again to offload it onto cheaper storage, since hardly anyone actually looks at it. Or is this something that I could be doing more transparently in 2.1 with imapc? From mysqlstudent at gmail.com Wed Nov 16 02:08:34 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 19:08:34 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> It's now delivering, but my deliver command must be wrong. This is my >> ~/.procmailrc: > .. >> # # /usr/libexec/dovecot/deliver -m spamtest >> lda(root): Fatal: destination user parameter (-d user) not given >> >> Am I doing something wrong? > > Is it trying to deliver mails to root user? That's not supported. No, I was just running as root from the command-line for an example. The maillog shows error 75: procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded This was for a test user named "alexuser" I created. Thanks again, Alex From tss at iki.fi Wed Nov 16 02:16:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 02:16:33 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 2.08, Alex wrote: > No, I was just running as root from the command-line for an example. > The maillog shows error 75: > > procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" > procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" > procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > procmail: Executing "/usr/libexec/dovecot/deliver" > procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > > This was for a test user named "alexuser" I created. deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging From micah at riseup.net Wed Nov 16 03:04:42 2011 From: micah at riseup.net (Micah Anderson) Date: Tue, 15 Nov 2011 20:04:42 -0500 Subject: [Dovecot] Deleting a mailbox and all its children Message-ID: <87pqgsho7p.fsf@algae.riseup.net> I'm looking at scripting a mechanism to delete a mailbox, which is easy with 'doveadm mailbox delete' -- however it gets complicated when there are children of arbitrary depth, for example, I may have this: restored.daily1.INBOX restored.daily1.Sent restored.daily1.Mystuff restored.daily1.Mystuff.foo I would like to delete the 'restored' mailbox and all its children (and unsubscribe the folders), but the 'doveadm mailbox delete' command wont delete any children. I'm surprised that there isn't a -r (recursive) option to 'doveadm mailbox delete', but perhaps I am missing the easier way to handle this? thanks for any ideas or suggestions! micah -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Wed Nov 16 03:15:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 03:15:02 +0200 Subject: [Dovecot] Deleting a mailbox and all its children In-Reply-To: <87pqgsho7p.fsf@algae.riseup.net> References: <87pqgsho7p.fsf@algae.riseup.net> Message-ID: <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> On 16.11.2011, at 3.04, Micah Anderson wrote: > I'm looking at scripting a mechanism to delete a mailbox, which is easy > with 'doveadm mailbox delete' -- however it gets complicated when there > are children of arbitrary depth, for example, I may have this: > > restored.daily1.INBOX > restored.daily1.Sent > restored.daily1.Mystuff > restored.daily1.Mystuff.foo > > I would like to delete the 'restored' mailbox and all its children (and > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > delete any children. > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > mailbox delete', but perhaps I am missing the easier way to handle this? Well, how about something like: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm mailbox delete restored That's what it would have to do internally anyway.. From mysqlstudent at gmail.com Wed Nov 16 03:50:56 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 20:50:56 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" >> procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" >> procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> procmail: Executing "/usr/libexec/dovecot/deliver" >> procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> >> This was for a test user named "alexuser" I created. > > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging Yes, looks like permissions. I have the following now in syslog: Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): Operation not permitted Where is it trying to write that it can't? Does alexuser need to be a member of group mail? Thanks, Alex From micah at riseup.net Wed Nov 16 04:23:47 2011 From: micah at riseup.net (micah anderson) Date: Tue, 15 Nov 2011 21:23:47 -0500 Subject: [Dovecot] Deleting a mailbox and all its children In-Reply-To: <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> References: <87pqgsho7p.fsf@algae.riseup.net> <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> Message-ID: <87lirghkjw.fsf@algae.riseup.net> On Wed, 16 Nov 2011 03:15:02 +0200, Timo Sirainen wrote: > On 16.11.2011, at 3.04, Micah Anderson wrote: > > > I'm looking at scripting a mechanism to delete a mailbox, which is easy > > with 'doveadm mailbox delete' -- however it gets complicated when there > > are children of arbitrary depth, for example, I may have this: > > > > restored.daily1.INBOX > > restored.daily1.Sent > > restored.daily1.Mystuff > > restored.daily1.Mystuff.foo > > > > I would like to delete the 'restored' mailbox and all its children (and > > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > > delete any children. > > > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > > mailbox delete', but perhaps I am missing the easier way to handle this? > > Well, how about something like: > > doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete > doveadm mailbox delete restored That wont work because 'restored.daily1.Mystuff.foo' wont get removed before 'restored.daily1.Mystuff' is attempted, resulting in the following: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm(micah): Error: Can't delete mailbox restored.daily1.Mystuff: Mailbox has children, delete them first The problem is the subfolders need to be removed first and then walk up the tree. perhaps I can do: doveadm mailbox list 'restored.*' | sort -r | xargs -n1 doveadm mailbox delete doveadm mailbox delete restored although I'm not sure if the reverse sort is going to work in all cases. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From achekalin at lazurit.com Wed Nov 16 08:16:04 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 16 Nov 2011 09:16:04 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC2AEDE.2080306@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> Message-ID: <4EC35524.30803@lazurit.com> Hello, Stan, > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. I now seriously consider to use mdbox due to its nice self-regulation. After all it I believe mdbox should do file compression on its own, no cron scripts required. > It's an archive. You're not going to use maildir so you don't need > random IOPS performance. Thus RAID5/6 are a much better fit for an > archive as you get better read performance, with more than adequate > write performance, and you use less disks. And as this is an archive, > you don't need real time automatic/transparent compression. Thus I > recommend something like: > > 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled > 2.6.39 or later kernel > 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors > e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 > Specify a strip size of 256KB for the array > Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading > ahead 1024 sectors at a time instead of the default of 256. This > will speed up your searches quite a bit. > 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults > 4. mbox w/zlib plugin. Compress daily files each night with a script > 5. You don't need LVM with a good RAID card (or with mdraid). This > controller can expand the RAID5 up to 8 drives (up to 32 drives max > using SAS expanders) We are considering to get HP DL180G6 server for 8 or 14 drives bays (base model price is somewhat equal, but additional drives adds up cost) with HP Smart Array P410 RAID controller (some servers are equipped with this controller by default) with 256 Mb battery-backed cache, but I'll check your suggestions! What memory size should I plan in the server? You're talking about AMD64 OS image, and 64-bit software are like to consume more memory that 32-bit, so looks like your talking about pretty huge RAM, and I don't believe it's necessary, or maybe I'm wrong? Problem is I have no experience with XFS and not sure I can tune it in the best way, so I'll go with mkfs.xfs defaults, I think. Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins and 2.1 handle that much better, but I don't like the idea of use unstable in production. Thank you for taking your time on my case, yours, Alexander From peter at newton.cx Wed Nov 16 11:00:50 2011 From: peter at newton.cx (Peter Williams) Date: Wed, 16 Nov 2011 01:00:50 -0800 Subject: [Dovecot] Invalid "doveadm search" queries not rejected In-Reply-To: <1321388697.21919.518.camel@hurina> References: <1320997281.2261.116.camel@iota.localdomain> <1321388697.21919.518.camel@hurina> Message-ID: <1321434050.2493.5.camel@iota.localdomain> On Tue, 2011-11-15 at 22:24 +0200, Timo Sirainen wrote: > On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > > > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > > 6). > > > > According to the doveadm-search-query(7) manual page, a "mailbox" > > specification can only come at the beginning of a search query. > > It doesn't exactly say that, but yeah, it doesn't also include MAILBOX > and MAILBOX-GUID as search keys. Added now to future versions. Thanks for explaining the situation. As a small improvement I would argue that the manpage was, and still is, a little bit misleading -- here's the bit of text that I was focusing on: Use either the keyword _mailbox_ and the name of the mailbox or the keyword _mailbox-guid_ and the mailbox's globally unique identifier ***at the beginning of the search_query.*** Peter -- Peter Williams / peter at newton.cx From odhiambo at gmail.com Wed Nov 16 11:26:54 2011 From: odhiambo at gmail.com (Odhiambo Washington) Date: Wed, 16 Nov 2011 12:26:54 +0300 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320856634.21919.371.camel@hurina> References: <1320791738.21919.350.camel@hurina> <1320856634.21919.371.camel@hurina> Message-ID: On Wed, Nov 9, 2011 at 19:37, Timo Sirainen wrote: > On Wed, 2011-11-09 at 18:34 +0300, Odhiambo Washington wrote: > > I have today tried compiling this -beta, but I saw so many warning about > > linking libraries, which I ignored, > > Probably plugin warnings which you can safely ignore. > > > Making all in wiki > > make: don't know how to make 2b. Stop > > Looks like there's some weird file went into wiki docs, which some makes > don't like. You could just find the line containing "2b" from > doc/wiki/Makefile and delete it. > > That did it. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 16:00:38 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 15:00:38 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321386560.21919.509.camel@hurina> References: <1321386560.21919.509.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 8:49 PM, Timo Sirainen wrote: > On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > >> I configured dovecot to use submission smtp host becouse of chroot. >> >> submission_host = 127.0.0.1 >> >> Unfortunatelly: >> >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol >> synchronization error (next input sent too soon: pipelining was not >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next >> input="MAIL FROM:\\r\\n" > > This basically says that Dovecot's SMTP client is being noncompliant, > because Exim doesn't advertise PIPELINING extension. But I'd rather not > add more code to fix this, since PIPELINING gives you a little bit > better performance anyway and you can most likely fix this by modifying > Exim's configs in some way (I can't believe Exim wouldn't support > PIPENING..). > > Thanks for your reply. Indeed exim supports PIPELINING by default: # telnet mail.apisoft.pl 25 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. 220 zeus.apisoft.pl ESMTP Exim 4.76 Wed, 16 Nov 2011 14:55:25 +0100 EHLO test 250-zeus.apisoft.pl Hello localhost [127.0.0.1] 250-SIZE 52428800 250-PIPELINING 250-AUTH LOGIN PLAIN CRAM-MD5 DIGEST-MD5 250-STARTTLS 250 HELP and PIPELINING seems to work fine. But maybe there is a problem with smtp dialog between dovecot and exim. From jester at arcor.de Wed Nov 16 16:02:51 2011 From: jester at arcor.de (Oliver Eales) Date: Wed, 16 Nov 2011 15:02:51 +0100 Subject: [Dovecot] Usage of director without LMTP Message-ID: <4EC3C28B.7040700@arcor.de> Hello, we want to migrate our dovecot 1.2 installation to verison 2.0 and want to use the director feature of version 2. Currently we have a two tier archtiecture, with dovecot proxys and a mailstore togehter with indexes on NFS. (With the common problems of this setup...) For the final setup we want to utilize the director for all protocols (IMAP,POP,LMTP,SIEVE). During the transition phase we will not be able to use LMTP for mail delivery, but we are still stuck with deliver as LDA. So for mail delivery we will not have the fixed user/server assignment, but rather a random assignment. What NFS- and locking settings should be used on the backend servers in this case. And should i use different settings for the host the deliver is executed than the IMAP/POP3 hosts which are accessed through a director proxy. Thanks in advance, Oliver Eales From mh+dovecot at zugschlus.de Wed Nov 16 16:11:25 2011 From: mh+dovecot at zugschlus.de (Marc Haber) Date: Wed, 16 Nov 2011 15:11:25 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321386560.21919.509.camel@hurina> References: <1321386560.21919.509.camel@hurina> Message-ID: <20111116141125.GC4966@torres.zugschlus.de> On Tue, Nov 15, 2011 at 09:49:20PM +0200, Timo Sirainen wrote: > On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > > I configured dovecot to use submission smtp host becouse of chroot. > > > > submission_host = 127.0.0.1 > > > > Unfortunatelly: > > > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > > synchronization error (next input sent too soon: pipelining was not > > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > > input="MAIL FROM:\\r\\n" > > This basically says that Dovecot's SMTP client is being noncompliant, > because Exim doesn't advertise PIPELINING extension. But I'd rather not > add more code to fix this, since PIPELINING gives you a little bit > better performance anyway and you can most likely fix this by modifying > Exim's configs in some way (I can't believe Exim wouldn't support > PIPENING..). I have always interpreted the standard in the way that a client MUST NOT assume that the server supports pipelining before it has advertised PIPELINING. Since PIPELINING is only advertised after the client has identified itself as being ESMTP compliant by saying EHLO instead of HELO, I believe that the client MUST wait with his EHLO until the server has shown its banner. Forcing synchronization is a very effective means of spam protection since most spam bots just blast away with EHLO, MAIL FROM without bothering to wait for the server's banner. Greetings Marc -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 31958061 Nordisch by Nature | How to make an American Quilt | Fax: *49 621 31958062 From llonchj at gmail.com Wed Nov 16 16:13:44 2011 From: llonchj at gmail.com (Jordi Llonch) Date: Thu, 17 Nov 2011 01:13:44 +1100 Subject: [Dovecot] Creating a password_scheme Message-ID: Hello, I am trying to develop a password_scheme for the Django passwords. After building and installing the module in $module_dir/auth/password/djangopass.so, also tried libdjangopass.so and libdjangopass_plugin.so, Dovecot does not recognize the scheme: doveadm pw -l CRYPT MD5 MD5-CRYPT SHA SHA1 SHA256 SHA512 SMD5 SSHA SSHA256 SSHA512 PLAIN CLEARTEXT CRAM-MD5 HMAC-MD5 DIGEST-MD5 PLAIN-MD4 PLAIN-MD5 LDAP-MD5 LANMAN NTLM OTP SKEY RPA SHA256-CRYPT SHA512-CRYPT Please can you help me? Thanks -- Jordi Llonch # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.40.3-0.fc15.x86_64 x86_64 Fedora release 15 (Lovelock) ext4 ----------password-scheme-django.c-------------------------- #include #include #include #include #include #include #include #define uchar unsigned char static bool djangopass_verify(const char *plaintext, const char *user, const unsigned char *raw_password, size_t size) { return TRUE; } void djangopass_generate(const char *plaintext, const char *user, const unsigned char **raw_password_r, size_t *size_r) { } struct password_scheme djangopass_scheme = { "DJANGO", PW_ENCODING_BASE64, 0, djangopass_verify, djangopass_generate }; struct module; void djangopass_init(struct module *module) { password_scheme_register(&djangopass_scheme); } void djangopass_deinit(void) { password_scheme_unregister(&djangopass_scheme); } --------------------------------------------------------------------------- From david at blue-labs.org Wed Nov 16 16:14:54 2011 From: david at blue-labs.org (David Ford) Date: Wed, 16 Nov 2011 09:14:54 -0500 Subject: [Dovecot] submission_host problem In-Reply-To: <20111116141125.GC4966@torres.zugschlus.de> References: <1321386560.21919.509.camel@hurina> <20111116141125.GC4966@torres.zugschlus.de> Message-ID: <4EC3C55E.8040401@blue-labs.org> this and several other features are tools i use with tremendous success at battling spam. every MTA connection that violates protocol by making an assumption or posts invalid data for the SMTP phase, gets kicked off with a 421. -david On 11/16/2011 09:11 AM, Marc Haber wrote: > I have always interpreted the standard in the way that a client MUST > NOT assume that the server supports pipelining before it has > advertised PIPELINING. Since PIPELINING is only advertised after the > client has identified itself as being ESMTP compliant by saying EHLO > instead of HELO, I believe that the client MUST wait with his EHLO > until the server has shown its banner. Forcing synchronization is a > very effective means of spam protection since most spam bots just > blast away with EHLO, MAIL FROM without bothering to wait for the > server's banner. Greetings Marc From tss at iki.fi Wed Nov 16 16:37:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 16:37:46 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> Message-ID: <1321454266.21919.545.camel@hurina> On Wed, 2011-11-16 at 15:00 +0100, Bambero wrote: > >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > >> synchronization error (next input sent too soon: pipelining was not > >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > >> input="MAIL FROM:\\r\\n" > > > > This basically says that Dovecot's SMTP client is being noncompliant, > > because Exim doesn't advertise PIPELINING extension. But I'd rather not > > add more code to fix this, since PIPELINING gives you a little bit > > better performance anyway and you can most likely fix this by modifying > > Exim's configs in some way (I can't believe Exim wouldn't support > > PIPENING..). > > > > > Thanks for your reply. > Indeed exim supports PIPELINING by default: Oh. Well, see if the attached patch helps? If it does, I'll add it to next version. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 1493 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 16:54:43 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 15:54:43 +0100 Subject: [Dovecot] Chroot In-Reply-To: <1321386260.21919.505.camel@hurina> References: <1321386260.21919.505.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 8:44 PM, Timo Sirainen wrote: > On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote: > >> all maildirs are located in: /var/mail/jdoe at example.com/Maildir > > So this is /var/mail/%u/Maildir template. > >> namespace { >> ? type = shared >> ? separator = / >> ? prefix = shared/%%u/ >> ? location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u > > You can replace this with: > > location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u > > Although if this is used outside chroot (e.g. lmtp, doveadm) it won't > work correctly.. > >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >> such file or directory > > This is done because you used %%h, to look up home directory from > userdb. If you don't use %%h, it's not done. > Thanks but my db backend returns home dir in chroot like /test at example.com while the realpath with no chroot is: /var/mail/test at example.com So both locations in this case are the same location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u From tss at iki.fi Wed Nov 16 16:59:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 16:59:06 +0200 Subject: [Dovecot] Chroot In-Reply-To: References: <1321386260.21919.505.camel@hurina> Message-ID: <1321455546.21919.553.camel@hurina> On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: > >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No > >> such file or directory > > > > This is done because you used %%h, to look up home directory from > > userdb. If you don't use %%h, it's not done. > > > > Thanks but my db backend returns home dir in chroot like > /test at example.com > > while the realpath with no chroot is: > /var/mail/test at example.com > > So both locations in this case are the same > location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u > location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u Yes, they are the same, but the difference is that with the second one you shouldn't fail with the userdb lookup error. Right? From tss at iki.fi Wed Nov 16 17:02:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 17:02:50 +0200 Subject: [Dovecot] Usage of director without LMTP In-Reply-To: <4EC3C28B.7040700@arcor.de> References: <4EC3C28B.7040700@arcor.de> Message-ID: <1321455770.21919.555.camel@hurina> On Wed, 2011-11-16 at 15:02 +0100, Oliver Eales wrote: > For the final setup we want to utilize the director for all protocols > (IMAP,POP,LMTP,SIEVE). > During the transition phase we will not be able to use LMTP for mail > delivery, but we are still stuck with deliver as LDA. > So for mail delivery we will not have the fixed user/server assignment, > but rather a random assignment. > > What NFS- and locking settings should be used on the backend servers in > this case. > And should i use different settings for the host the deliver is executed > than the IMAP/POP3 hosts which are accessed through a director proxy. All the LDA/IMAP/POP3 servers should use these settings to minimize problems: mail_nfs_storage = yes mail_nfs_index = yes Once you're using LMTP proxying also you can remove those. Otherwise the settings are the same. From leo at strike.wu.ac.at Wed Nov 16 17:04:20 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Wed, 16 Nov 2011 16:04:20 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <1320441536.21919.206.camel@hurina> References: <4EAED444.70608@strike.wu.ac.at> <1320441536.21919.206.camel@hurina> Message-ID: <4EC3D0F4.7020204@strike.wu.ac.at> On 11/04/2011 10:18 PM, Timo Sirainen wrote: > On Mon, 2011-10-31 at 18:00 +0100, Alexander 'Leo' Bergolth wrote: >> NAMESPACE_2='maildir:/var/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep' >> NAMESPACE_2_PREFIX=Shared/spamrep/ > > The settings are now named. > http://wiki2.dovecot.org/MailLocation#Custom_namespace_location shows > the basics. So for example if you have: > > namespace foo { > .. > } > > You can change it via: > > NAMESPACE/FOO/PREFIX=Shared/spamrep/ > NAMESPACE/FOO/SEP=/ > ..etc.. > > The '/' character is a bit problematic with shells scripts though, see > the wiki link how to do it via "env" at least. > > Another problem is if you want to add more namespaces than you have > configured. IIRC it's possible to simply add: > > NAMESPACE=foo Thanks a lot, Timo! Changing to named, /-separated environment variables did the job. NAMESPACE/S-SPAMREP/LIST=yes NAMESPACE/S-SPAMREP/LOCATION=maildir:/var/spool/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep NAMESPACE/S-SPAMREP/PREFIX=Shared/spamrep/ NAMESPACE/S-SPAMREP/SEPARATOR=/ NAMESPACE/S-SPAMREP/TYPE=public NAMESPACE/S-TESTSHARED/LIST=yes NAMESPACE/S-TESTSHARED/LOCATION=maildir:/var/spool/mail/Shared/testshared:INDEX=~/Maildir/index/Shared/testshared NAMESPACE/S-TESTSHARED/PREFIX=Shared/testshared/ NAMESPACE/S-TESTSHARED/SEPARATOR=/ NAMESPACE=s-testshared s-spamrep NAMESPACE/S-TESTSHARED/TYPE=public USERDB_KEYS=SYSTEM_GROUPS_USER UID GID HOME NAMESPACE/S-SPAMREP/LIST NAMESPACE NAMESPACE/S-TESTSHARED/SEPARATOR NAMESPACE/S-TESTSHARED/TYPE NAMESPACE/S-TESTSHARED/PREFIX NAMESPACE/S-TESTSHARED/LIST NAMESPACE/S-TESTSHARED/LOCATION NAMESPACE/S-SPAMREP/SEPARATOR NAMESPACE/S-SPAMREP/TYPE NAMESPACE/S-SPAMREP/PREFIX NAMESPACE/S-SPAMREP/LOCATION > And more than one is probably either "foo bar" or "foo,bar". If you get > something to work, please add example to > http://wiki2.dovecot.org/PostLoginScripting :) I'll add my script to the Wiki. Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From tss at iki.fi Wed Nov 16 17:04:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 17:04:42 +0200 Subject: [Dovecot] Creating a password_scheme In-Reply-To: References: Message-ID: <1321455883.21919.556.camel@hurina> On Thu, 2011-11-17 at 01:13 +1100, Jordi Llonch wrote: > I am trying to develop a password_scheme for the Django passwords. > > After building and installing the module in > $module_dir/auth/password/djangopass.so, also tried libdjangopass.so and > libdjangopass_plugin.so, Dovecot does not recognize the scheme: There is no longer a separate password/ directory. Just put it to $module_dir/auth/. Where do you see the password/ directory documented? It should be changed.. From achekalin at lazurit.com Wed Nov 16 17:16:35 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 16 Nov 2011 18:16:35 +0300 Subject: [Dovecot] How to do SEARCH with doveadm Message-ID: <4EC3D3D3.7000601@lazurit.com> Hello, I'm still playing with search messages from/to specified email address. In fact I need to do that to take such messages and copy it to another mailbox (so that box will contain 'Search results' and will be available to examine by any IMAP client). I've put my mailings in mdbox storage format, and use latest 2.1 beta version of Dovecot to be sure I use the freshest approach in search (and plan to use Solr FTS as well). The setup is each, I use passdb { driver = passwd-file args = /etc/dovecot/auth/%Lu } and userdb { driver = static args = uid=dovecot gid=dovecot home=/MAIL/mail_storage/%Lu } and put user:password pair in file named after my mailbox in /etc/dovecot/auth/ (let's name it test at domain.com), and the messages are at /MAIL/mail_storage/test at domain.com/... While I use it from IMAP client it works nicely. Ok, I put somewhat 5800 messages in INBOX, and IMAP pretty fast in working with it. But as I try to use doveadm to search for messages I need I simple lost my way (doing it under root): # doveadm search ALL doveadm(root): Error: chdir(/root/) failed: Permission denied (euid=104(dovecot) egid=106(dovecot) missing +x perm: /root, dir owned by 0:0 mode=0700) doveadm(root): Error: chdir(/root) failed: Permission denied Whatever cli command I try doveadv try to find messages in /root dispute it is at /MAIL/mail_storage/%Lu. How can I make doveadv use right mail path? In fact, I don't want to use system users to work with mail, it'll be simple virtual mail user, so it is pointless to set up special system user to do that search. I think I lost myself in general concept of how Dovecot handles users and credentials, so please please please push me in right direction! Thank you, Alexander From micah at riseup.net Wed Nov 16 18:09:12 2011 From: micah at riseup.net (micah anderson) Date: Wed, 16 Nov 2011 11:09:12 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <1321388840.21919.520.camel@hurina> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> Message-ID: <87ehx8gic7.fsf@algae.riseup.net> On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen wrote: > On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > > I really like the feature where you can define quota rules with percents > > which trigger off of the default values[0] (so you can set the Trash to > > allow for 10% more of the user's quota for example). > > > > What I would really love in dovecot would be for the ability to > > configure a quota rule for dovecot-lda. I would like to configure things > > so we don't bounce emails for users until they are well over quota, the > > IMAP quota plugin is a really great way to notify people that they are > > over quota because it fails to write to other folders.... that should be > > enough to get people's attention that they need to deal with things, but > > bouncing is harsh. > > You can give LDA-specific quota settings. Like maybe: > > protocol lda { > plugin { > quota_rule2 = INBOX:storage=200% > } > } Aha! Just what I was looking for. I presume the number following 'quota_rule' must be an unique number across all quota_rules in my config? > or even not enforcing quota at all: > > protocol lda { > plugin { > quota = maildir:User quota:noenforcing > } > } I was not aware of the noenforcing option, that answered another question I had not asked yet, so thanks for that! :) micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 18:23:02 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:23:02 +0100 Subject: [Dovecot] Chroot In-Reply-To: <1321455546.21919.553.camel@hurina> References: <1321386260.21919.505.camel@hurina> <1321455546.21919.553.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 3:59 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: >> >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >> >> such file or directory >> > >> > This is done because you used %%h, to look up home directory from >> > userdb. If you don't use %%h, it's not done. >> > >> >> Thanks but my db backend returns home dir in chroot like >> /test at example.com >> >> while the realpath with no chroot is: >> /var/mail/test at example.com >> >> So both locations in this case are the same >> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u >> location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u > > Yes, they are the same, but the difference is that with the second one > you shouldn't fail with the userdb lookup error. Right? > Thanks a lot it almost helped. But now there is another problem: Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX ** test at apisoft.pl R=localuser T=local_delivery: Child process of local_delivery transport (running command "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain") was terminated by signal 11 (Segmentation fault) But it anly appear if user has .sieve filter with mail redirection (uses submission_host) From bambero at gmail.com Wed Nov 16 18:27:13 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:27:13 +0100 Subject: [Dovecot] Chroot In-Reply-To: References: <1321386260.21919.505.camel@hurina> <1321455546.21919.553.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 5:23 PM, Bambero wrote: > On Wed, Nov 16, 2011 at 3:59 PM, Timo Sirainen wrote: >> On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: >>> >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >>> >> such file or directory >>> > >>> > This is done because you used %%h, to look up home directory from >>> > userdb. If you don't use %%h, it's not done. >>> > >>> >>> Thanks but my db backend returns home dir in chroot like >>> /test at example.com >>> >>> while the realpath with no chroot is: >>> /var/mail/test at example.com >>> >>> So both locations in this case are the same >>> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u >>> location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u >> >> Yes, they are the same, but the difference is that with the second one >> you shouldn't fail with the userdb lookup error. Right? >> > > Thanks a lot it almost helped. But now there is another problem: > Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX > ** test at apisoft.pl R=localuser T=local_delivery: Child process of > local_delivery transport (running command > "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f > $sender_address -a $original_local_part@$original_domain") was > terminated by signal 11 (Segmentation fault) > > But it anly appear if user has .sieve filter with mail redirection > (uses submission_host) > Sorry wrong thread. I this case you were right now everything work fine. From tss at iki.fi Wed Nov 16 18:27:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 18:27:20 +0200 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <87ehx8gic7.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> Message-ID: <1321460847.21919.562.camel@hurina> On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote: > > protocol lda { > > plugin { > > quota_rule2 = INBOX:storage=200% > > } > > } > > Aha! Just what I was looking for. I presume the number following > 'quota_rule' must be an unique number across all quota_rules in my > config? No only unique, but all of the numbers must exist. For example quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10 isn't used, unless quota_rule2..9 also exist. From bambero at gmail.com Wed Nov 16 18:28:16 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:28:16 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321454266.21919.545.camel@hurina> References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 3:37 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 15:00 +0100, Bambero wrote: >> >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol >> >> synchronization error (next input sent too soon: pipelining was not >> >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next >> >> input="MAIL FROM:\\r\\n" >> > >> > This basically says that Dovecot's SMTP client is being noncompliant, >> > because Exim doesn't advertise PIPELINING extension. But I'd rather not >> > add more code to fix this, since PIPELINING gives you a little bit >> > better performance anyway and you can most likely fix this by modifying >> > Exim's configs in some way (I can't believe Exim wouldn't support >> > PIPENING..). >> > >> > >> Thanks for your reply. >> Indeed exim supports PIPELINING by default: > > Oh. Well, see if the attached patch helps? If it does, I'll add it to > next version. > > Thanks a lot it almost helped. But now there is another problem: Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX ** test at apisoft.pl R=localuser T=local_delivery: Child process of local_delivery transport (running command "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain") was terminated by signal 11 (Segmentation fault) But it anly appear if user has .sieve filter with mail redirection (uses submission_host) From micah at riseup.net Wed Nov 16 18:31:01 2011 From: micah at riseup.net (micah anderson) Date: Wed, 16 Nov 2011 11:31:01 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <1321460847.21919.562.camel@hurina> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> <1321460847.21919.562.camel@hurina> Message-ID: <87ty64f2re.fsf@algae.riseup.net> On Wed, 16 Nov 2011 18:27:20 +0200, Timo Sirainen wrote: > On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote: > > > protocol lda { > > > plugin { > > > quota_rule2 = INBOX:storage=200% > > > } > > > } > > > > Aha! Just what I was looking for. I presume the number following > > 'quota_rule' must be an unique number across all quota_rules in my > > config? > > No only unique, but all of the numbers must exist. For example > quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10 > isn't used, unless quota_rule2..9 also exist. Ah, interesting, good to know. I presume that 'quota_rule' stands in for 'quota_rule1' and one does not need to specify a quota_rule1 if one has a 'quota_rule'. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From stan at hardwarefreak.com Wed Nov 16 18:36:12 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 16 Nov 2011 10:36:12 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC35501.6050601@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> <4EC35501.6050601@lazurit.com> Message-ID: <4EC3E67C.4050107@hardwarefreak.com> On 11/16/2011 12:15 AM, Alexander Chekalin wrote: > Hello, Stan, > >> This is why I recommended mbox in the first place. If your only writes >> to these mailbox files are appends of new messages, mbox is the best >> format by far. It's faster at appending than any other format, and it's >> faster for searching than any other. > > I now seriously consider to use mdbox due to its nice self-regulation. > After all it I believe mdbox should do file compression on its own, no > cron scripts required. mbox and mdbox each has strengths and weaknesses. mbox will compress with a higher ratio than mdbox. You already have a nightly script that moves all mail from the day into a new file. Piping that through gzip or bzip2 is a no brainer. It'll add one line to your existing script, if that. Dovecot will decompress the file transparently when you access it via IMAP. And again since it's a single file searching it is much faster. With mbox you will have a single file for each day of emails. This seems ideal for archive purposes, one file per day. mdbox does fully transparent de/compression which is nice. The downside is that Dovecot does dbox compression on a per email basis, not a per file basis. So your compression ratio will be much less than with mbox, especially with bzip2 which works best on files over 900KB in size. Most emails are less than 8KB. Using mdbox will yield multiple files per day of emails instead of just one. Either format is much better than maildir for archiving. >> It's an archive. You're not going to use maildir so you don't need >> random IOPS performance. Thus RAID5/6 are a much better fit for an >> archive as you get better read performance, with more than adequate >> write performance, and you use less disks. And as this is an archive, >> you don't need real time automatic/transparent compression. Thus I >> recommend something like: >> >> 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled >> 2.6.39 or later kernel >> 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors >> e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 >> Specify a strip size of 256KB for the array >> Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading >> ahead 1024 sectors at a time instead of the default of 256. This >> will speed up your searches quite a bit. >> 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults >> 4. mbox w/zlib plugin. Compress daily files each night with a script >> 5. You don't need LVM with a good RAID card (or with mdraid). This >> controller can expand the RAID5 up to 8 drives (up to 32 drives max >> using SAS expanders) > > We are considering to get HP DL180G6 server for 8 or 14 drives bays The P410 tops out at 8 drives, so get the 8 drive model. Start with 4 x 2TB drives in RAID5. Add 4 more drives when you need the capacity, and when drive prices are back down to normal (see below). http://h18004.www1.hp.com/products/quickspecs/13248_na/13248_na.html > (base model price is somewhat equal, but additional drives adds up cost) Especially right now in 2011. Flooding in Thailand, where 25% of the world's drives are produced, has doubled the cost of all hard drives worldwide. Now is a horrible time to buy spinning drives. I've read it may be 12 months before prices start coming back down... > with HP Smart Array P410 RAID controller (some servers are equipped with > this controller by default) with 256 Mb battery-backed cache, but I'll > check your suggestions! The P410 should be fine for a dedicated archive server. > What memory size should I plan in the server? You're talking about AMD64 > OS image, and 64-bit software are like to consume more memory that > 32-bit, so looks like your talking about pretty huge RAM, and I don't > believe it's necessary, or maybe I'm wrong? The memory footprint of 64bit binaries is nothing to worry about. The additional amount consumed is more than offset by the performance gained with direct access to RAM above 4GB compared to the performance of PAE. Keep in mind that 90% of your memory will be eaten by Linux buffer cache. Your binaries will account for less than 5% of your RAM consumption. If I understand correctly how you will use this archive server, then 8GB should be plenty. 8GB is standard on the 8 drive DL180 G6. http://h18004.www1.hp.com/products/quickspecs/13248_na/13248_na.html > Problem is I have no experience with XFS and not sure I can tune it in > the best way, so I'll go with mkfs.xfs defaults, I think. With only 4 drives and using a P410 w/cache and RAID5, doing manual XFS tuning isn't necessary for good performance, especially for an archive application which is data heavy, not metadata heavy. Setting sunit/swidth to match the RAID5 layout may increase performance slightly due to stripe aligned writes, but not enough that I'd worry about it. Just use the mkfs.xfs defaults. If you get the BBWC for the P410, enable the controller write cache, and mount XFS with 'nobarrier'. This will increase write performance quite a bit as fsyncs will complete instantly. > Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins > and 2.1 handle that much better, but I don't like the idea of use > unstable in production. Me neither. Speaking of archive/search, did you take look at Enkive yet? http://www.enkive.org/ > Thank you for taking your time on my case, You're welcome Alexander. -- Stan P.S. You may wish to implement dnswl.org ;) From tss at iki.fi Wed Nov 16 18:38:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 18:38:07 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> Message-ID: <1321461498.21919.566.camel@hurina> On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: > Thanks a lot it almost helped. But now there is another problem: > Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX > ** test at apisoft.pl R=localuser T=local_delivery: Child process of > local_delivery transport (running command > "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f > $sender_address -a $original_local_part@$original_domain") was > terminated by signal 11 (Segmentation fault) > > But it anly appear if user has .sieve filter with mail redirection > (uses submission_host) It's difficult to debug crashes without gdb backtrace. But since you're running in a strange chrooted environment, it may be difficult to get a core file.. Probably the easiest way would be if you apply the attached patch, then try to deliver a mail that causes this crash (and no other mails), and attach gdb into the process during the 10 second wait: gdb -p `pidof dovecot-lda` cont bt full -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 286 bytes Desc: not available URL: From tss at iki.fi Wed Nov 16 20:02:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:02:18 +0200 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <4EBACFC0.6060703@gmx.de> References: <4EB6AFE3.3020001@gmx.de> <4EBACFC0.6060703@gmx.de> Message-ID: <1321466541.21919.576.camel@hurina> On Wed, 2011-11-09 at 20:08 +0100, e-frog wrote: > On 06.11.2011 17:03, wrote e-frog: > > 1 lsub "" "virtual/*" > > * LSUB () "/" "virtual" > > * LSUB () "/" "virtual/Lists" > > * LSUB () "/" "virtual/unread" > > 1 OK Lsub completed. > > Just tested this again with 2.1beta1 with the same result. Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/16024d03f66e From tss at iki.fi Wed Nov 16 20:15:44 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:15:44 +0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB72177.8090801@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> Message-ID: <1321467345.21919.580.camel@hurina> On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: > Ed W schrieb: > > > See the "sticky" in my reply. You use one of several techniques to > > ensure that users always end up on the server with the indexes on. That > > way much of the IO is served from that local machine and you only access > > the SAN for the (in theory much less frequent) access to the mail files > > themselves. > > I know you can afford that (IMAP) users always end up on one particular > server but afaik this only works for incoming IMAP connections. > > My mail exchangers use dovecot-lda and I think indexes will be written > from these servers too or am I wrong with this? You can use LMTP and LMTP proxying. From tss at iki.fi Wed Nov 16 20:18:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:18:55 +0200 Subject: [Dovecot] Missing public folder In-Reply-To: <201111071110.20143.komodo@uvt.cz> References: <201111071110.20143.komodo@uvt.cz> Message-ID: <1321467535.21919.582.camel@hurina> On Mon, 2011-11-07 at 11:10 +0100, komodo wrote: > Hi > > i am using dovecot 2.0.13 and i can't see public folder in my mail client. But what is strange, it > was working after instalation, and i don't know when it dissapeared. Maybe after some upgrade ? Try talking IMAP protocol directly: telnet localhost 143 a login user pass b list "" * c select spam_learner > namespace { > location = maildir:/home/shared/Maildir Is this the "spam_learner" mailbox itself alone, or does it have children? ("spam_learner/spam"?) From tss at iki.fi Wed Nov 16 20:25:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:25:29 +0200 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <4EB7BF9A.9020900@webfusion.com> References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> <4EB7BF9A.9020900@webfusion.com> Message-ID: <1321467929.21919.583.camel@hurina> On Mon, 2011-11-07 at 13:23 +0200, Mark Zealey wrote: > 04-11-2011 23:56, Timo Sirainen yazm??: > From my reading of the code in 2.0.15, you pass the local & remote IP > in lmtp/commands.c:427 but the struct mail_storage_service_input only > has remote_ip and local_ip fields, not port fields (unlike the > auth_user_info struct which has both ip's and port entries as well). I > think that the mail_storage_service_input struct needs the uint > local_port, remote_port fields adding in & appropriate code changes to > pass these through? Yeah, you're right, I missed that one. Added to hg now. From tss at iki.fi Wed Nov 16 20:31:15 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:31:15 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC3FDB5.7090301@goldey.net> References: <4EC2BE26.3030807@goldey.net> <4EC3FDB5.7090301@goldey.net> Message-ID: <1321468275.21919.586.camel@hurina> Your message was discarded by mailman, probably because it was html-only. Anyway: On Wed, 2011-11-16 at 13:15 -0500, Mark Goldey wrote: > (gdb) p *auth_request->userdb->userdb->iface > $2 = {name = 0x807023e "vpopmail", preinit = 0x8065d60 > , init = 0, deinit = 0, > lookup = 0x8066010 , iterate_init = 0, iterate_next > = 0, iterate_deinit = 0} OK, so "auth_request->userdb->userdb->iface->lookup" should call vpopmail_lookup(), it's not NULL as this says: > > (gdb) bt full > > #0 0x00000000 in ?? () So something's going strangely wrong. Maybe gcc optimizations cause this and the real problem happens later. Try compiling auth process with optimizations: - Remove -O2 from src/auth/Makefile - cd src/auth ; make clean ; make ; make install Is the backtrace different? From tss at iki.fi Wed Nov 16 20:33:27 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:33:27 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: <1321468407.21919.588.camel@hurina> On Tue, 2011-11-15 at 20:50 -0500, Alex wrote: > > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging > > Yes, looks like permissions. I have the following now in syslog: > > Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: > setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): > Operation not permitted > Where is it trying to write that it can't? It's trying to change the process's UID/GID to mail/mail for some reason. Have you set "mail_uid=mail" or something? > Does alexuser need to be a member of group mail? No. From tss at iki.fi Wed Nov 16 20:34:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:34:14 +0200 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: <1321385991.21919.503.camel@hurina> Message-ID: <1321468455.21919.589.camel@hurina> On Tue, 2011-11-15 at 16:04 -0800, Mark Moseley wrote: > > The gotcha is that you have two completely independent quotas with > > independent usage/limits for the INBOX and Archive namespaces. If that > > is what you want, it should all be fine. > > Nope, that's totally fine. The idea is to put Archive on cheaper > (slower) storage and then grant more generous quotas there to make it > worth their while to use, without slowing down their Inbox. Another > application would be to put their Spam in another namespace (for > people who choose to have it put in a separate folder) with a lower > quota, again to offload it onto cheaper storage, since hardly anyone > actually looks at it. Should be fine then. > Or is this something that I could be doing more transparently in 2.1 with imapc? I don't really see how that could help. From moseleymark at gmail.com Wed Nov 16 20:40:45 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Wed, 16 Nov 2011 10:40:45 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: <1321468455.21919.589.camel@hurina> References: <1321385991.21919.503.camel@hurina> <1321468455.21919.589.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 10:34 AM, Timo Sirainen wrote: > On Tue, 2011-11-15 at 16:04 -0800, Mark Moseley wrote: >> > The gotcha is that you have two completely independent quotas with >> > independent usage/limits for the INBOX and Archive namespaces. If that >> > is what you want, it should all be fine. >> >> Nope, that's totally fine. The idea is to put Archive on cheaper >> (slower) storage and then grant more generous quotas there to make it >> worth their while to use, without slowing down their Inbox. Another >> application would be to put their Spam in another namespace (for >> people who choose to have it put in a separate folder) with a lower >> quota, again to offload it onto cheaper storage, since hardly anyone >> actually looks at it. > > Should be fine then. > >> Or is this something that I could be doing more transparently in 2.1 with imapc? > > I don't really see how that could help. Ah, bummer. I thought maybe 2.1 could proxy to a separate folder or namespace (but I've also barely had a chance to look at it), like so certain folders would be grabbed from a proxy. Haven't really thought that through though :) From pw at wk-serv.de Wed Nov 16 20:40:50 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 16 Nov 2011 19:40:50 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <1321467345.21919.580.camel@hurina> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> Message-ID: <4EC403B2.60005@wk-serv.de> Timo Sirainen schrieb: > On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: >> >> My mail exchangers use dovecot-lda and I think indexes will be written >> from these servers too or am I wrong with this? > > You can use LMTP and LMTP proxying. I already use lmtp:unix:private/dovecot-lmtp as transport but where is the link to the indexes? From bambero at gmail.com Wed Nov 16 21:25:10 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 20:25:10 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321461498.21919.566.camel@hurina> References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 5:38 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: >> Thanks a lot it almost helped. But now there is another problem: >> Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX >> ** test at apisoft.pl R=localuser T=local_delivery: Child process of >> local_delivery transport (running command >> "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f >> $sender_address -a $original_local_part@$original_domain") was >> terminated by signal 11 (Segmentation fault) >> >> ?But it anly appear if user has .sieve filter with mail redirection >> (uses submission_host) > > It's difficult to debug crashes without gdb backtrace. But since you're > running in a strange chrooted environment, it may be difficult to get a > core file.. > > Probably the easiest way would be if you apply the attached patch, then > try to deliver a mail that causes this crash (and no other mails), and > attach gdb into the process during the 10 second wait: > > gdb -p `pidof dovecot-lda` > cont > > bt full > > Can you see something here: gdb -p `pgrep dovecot-lda` GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-openbsd5.0". Attaching to process 4118 0x030e6719 in ?? () (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x0c96c825 in ?? () (gdb) bt full #0 0x0c96c825 in ?? () No symbol table info available. #1 0x7dfa61c0 in ?? () No symbol table info available. #2 0x00000001 in ?? () No symbol table info available. #3 0x00000001 in ?? () No symbol table info available. #4 0x00000002 in ?? () No symbol table info available. #5 0x2665d31c in ?? () No symbol table info available. #6 0x2665d31c in ?? () No symbol table info available. #7 0xcfbe13e8 in ?? () No symbol table info available. #8 0x0665fd79 in ?? () No symbol table info available. #9 0x00000000 in ?? () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: , process 4118 From bambero at gmail.com Wed Nov 16 21:37:32 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 20:37:32 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 8:25 PM, Bambero wrote: > On Wed, Nov 16, 2011 at 5:38 PM, Timo Sirainen wrote: >> On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: >>> Thanks a lot it almost helped. But now there is another problem: >>> Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX >>> ** test at apisoft.pl R=localuser T=local_delivery: Child process of >>> local_delivery transport (running command >>> "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f >>> $sender_address -a $original_local_part@$original_domain") was >>> terminated by signal 11 (Segmentation fault) >>> >>> ?But it anly appear if user has .sieve filter with mail redirection >>> (uses submission_host) >> >> It's difficult to debug crashes without gdb backtrace. But since you're >> running in a strange chrooted environment, it may be difficult to get a >> core file.. >> >> Probably the easiest way would be if you apply the attached patch, then >> try to deliver a mail that causes this crash (and no other mails), and >> attach gdb into the process during the 10 second wait: >> >> gdb -p `pidof dovecot-lda` >> cont >> >> bt full >> >> > > Can you see something here: > > gdb -p `pgrep dovecot-lda` > GNU gdb 6.3 > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. ?Type "show warranty" for details. > This GDB was configured as "i386-unknown-openbsd5.0". > Attaching to process 4118 > 0x030e6719 in ?? () > (gdb) cont > Continuing. > > Program received signal SIGSEGV, Segmentation fault. > 0x0c96c825 in ?? () > (gdb) bt full > #0 ?0x0c96c825 in ?? () > No symbol table info available. > #1 ?0x7dfa61c0 in ?? () > No symbol table info available. > #2 ?0x00000001 in ?? () > No symbol table info available. > #3 ?0x00000001 in ?? () > No symbol table info available. > #4 ?0x00000002 in ?? () > No symbol table info available. > #5 ?0x2665d31c in ?? () > No symbol table info available. > #6 ?0x2665d31c in ?? () > No symbol table info available. > #7 ?0xcfbe13e8 in ?? () > No symbol table info available. > #8 ?0x0665fd79 in ?? () > No symbol table info available. > #9 ?0x00000000 in ?? () > No symbol table info available. > (gdb) quit > The program is running. ?Quit anyway (and detach it)? (y or n) y > Detaching from program: , process 4118 > I forgot to say that mails are delivered using SMTP so submission_host works, mails are strored i destination mailbox too. Only this Segmentation Fault ... From tss at iki.fi Wed Nov 16 21:45:34 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 21:45:34 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: <1321472737.21919.596.camel@hurina> On Wed, 2011-11-16 at 20:25 +0100, Bambero wrote: > Can you see something here: No, the "??" isn't very helpful. > gdb -p `pgrep dovecot-lda` > This GDB was configured as "i386-unknown-openbsd5.0". Oh, this is OpenBSD. Try if this works instead: gdb /usr/local/libexec/dovecot/dovecot-lda `pgrep dovecot-lda` If it still gives only "??" output, that didn't work either.. From tss at iki.fi Wed Nov 16 21:47:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 21:47:17 +0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EC403B2.60005@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> Message-ID: <1321472837.21919.598.camel@hurina> On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: > Timo Sirainen schrieb: > > On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: > >> > >> My mail exchangers use dovecot-lda and I think indexes will be written > >> from these servers too or am I wrong with this? > > > > You can use LMTP and LMTP proxying. > > I already use lmtp:unix:private/dovecot-lmtp as transport but where is > the link to the indexes? You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot LMTP proxy, which would forward the connection to the backend server which handles that user's IMAP/POP3/LMTP connections. From guenther at rudersport.de Wed Nov 16 22:08:40 2011 From: guenther at rudersport.de (Karsten =?ISO-8859-1?Q?Br=E4ckelmann?=) Date: Wed, 16 Nov 2011 21:08:40 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: <1321474120.4854.13.camel@monkey> On Tue, 2011-11-15 at 19:00 -0500, Alex wrote: > DELIVER=/usr/libexec/dovecot/deliver > > :0 fhW > | $DELIVER -m xspamtest Do you really want the pipe to be a (f)ilter? What do you expect deliver to pass back? And you're feeding deliver the mail (h)eaders only, dropping the body. -- char *t="\10pse\0r\0dtu\0. at ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} From sb at dod.no Thu Nov 17 00:45:28 2011 From: sb at dod.no (Steinar Bang) Date: Wed, 16 Nov 2011 23:45:28 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT References: <201111091029.37963.thomas@koch.ro> <1320846789.21919.361.camel@hurina> Message-ID: <87sjlnr8jb.fsf@dod.no> >>>>> Timo Sirainen : > I'm not as much against git anymore as I was when I switched to hg, > but I don't see much benefits in switching to git either. I highly > doubt I'd get even a single patch more if I used git instead of hg. FWIW the distingushing feature of git isn't that it is a ditributed VCS. It's the ease with which it does branching and merging. What put me off git for a long time, was Linus' arrogance wrt. to existing version control systems. But after using git, I have to admit that it is... quite clever. Especially the branching and merging stuff. From tss at iki.fi Thu Nov 17 01:30:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 01:30:42 +0200 Subject: [Dovecot] v2.0.16 released Message-ID: <1321486243.21919.620.camel@hurina> http://dovecot.org/releases/2.0/dovecot-2.0.16.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.16.tar.gz.sig I only now noticed that the VSZ limits weren't being enforced with earlier v2.0.x releases (or they were set 1024 times too high). So if a Dovecot process was leaking memory, it wasn't being killed by kernel. Now that this enforcing is done, some installations will probably start seeing errors about reaching these limits in normal operation. The default_vsz_limit is 256 MB. You may want to increase it in larger installations to 1 GB just in case. Besides the changes listed below, a lot of smaller fixes were done. v2.0.16 2011-11-17 Timo Sirainen * VSZ limits weren't being enforced for any processes. On server with large mailboxes you may now see errors about it if the limits aren't high enough. To fix them, either increase individual service { vsz_limit } values or simply increase the default_vsz_limit setting. * Proxying: If using ssl=yes or starttls=yes with a hostname (not IP) as proxy destination, require that the certificate matches the given hostname. * LMTP: Changed default client_limit to 1. This should improve LMTP throughput with default settings. * dsync: Quota is no longer enforced (i.e. dsync can't fail because user is over quota). + Added "auto" mail storage driver, which can be used to autodetect mailbox location and format. This behavior is already the default for empty mail_location setting, so this change is mainly useful for shared namespace's location setting. + checkpassword: Export all auth %variables to AUTH_* environment. From mysqlstudent at gmail.com Thu Nov 17 02:29:36 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:29:36 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321468407.21919.588.camel@hurina> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> Message-ID: Hi, >> > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging >> >> Yes, looks like permissions. I have the following now in syslog: >> >> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >> Operation not permitted > >> Where is it trying to write that it can't? > > It's trying to change the process's UID/GID to mail/mail for some > reason. Have you set "mail_uid=mail" or something? mail_uid isn't set, but I have the following in dovecot.conf: mail_gid = mail mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_privileged_group = mail Is that not correct? Thanks, Alex From tss at iki.fi Thu Nov 17 02:34:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 02:34:31 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> Message-ID: <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> On 17.11.2011, at 2.29, Alex wrote: >>> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >>> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >>> Operation not permitted >> >>> Where is it trying to write that it can't? >> >> It's trying to change the process's UID/GID to mail/mail for some >> reason. Have you set "mail_uid=mail" or something? > > mail_uid isn't set, but I have the following in dovecot.conf: > > mail_gid = mail The error most likely happens because of this. Is there a reason why you want this? > mail_privileged_group = mail This should be enough for creating lock files to /var/spool/mail/. Although dovecot-lda can't use this setting, since it's not started as root.. From mysqlstudent at gmail.com Thu Nov 17 02:49:53 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:49:53 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: Hi, >>>> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >>>> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >>>> Operation not permitted >>> >>>> Where is it trying to write that it can't? >>> >>> It's trying to change the process's UID/GID to mail/mail for some >>> reason. Have you set "mail_uid=mail" or something? >> >> mail_uid isn't set, but I have the following in dovecot.conf: >> >> mail_gid = mail > > The error most likely happens because of this. Is there a reason why you want this? > >> mail_privileged_group = mail > > This should be enough for creating lock files to /var/spool/mail/. Although dovecot-lda can't use this setting, since it's not started as root.. Okay, that fixed it. It's now delivering mail properly, thanks. Just to be clear I understand, if I use the following .procmailrc, it should save all messages with the X-Spam-Status header in xspamstatus, and everything else gets delivered to /var/spool/mail/user, correct? DELIVER=/usr/libexec/dovecot/deliver :0 * ^X-Spam-Status | $DELIVER -m xspamstatus :0 | $DELIVER This is just a simple test which appears to work the way I outlined, but wanted to be sure. Thanks again for sticking with me. Best, Alex From mysqlstudent at gmail.com Thu Nov 17 02:56:37 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:56:37 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321474120.4854.13.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321474120.4854.13.camel@monkey> Message-ID: Hi, >> DELIVER=/usr/libexec/dovecot/deliver >> >> :0 fhW >> | $DELIVER -m xspamtest > > Do you really want the pipe to be a (f)ilter? What do you expect deliver > to pass back? And you're feeding deliver the mail (h)eaders only, > dropping the body. Yes, thanks. I pasted from another filter we were using, and forgot. procmail still seems to be much better than the deliver sieve language, despite how long it's been since I've used it :-) Thanks, Alex From mysqlstudent at gmail.com Thu Nov 17 02:57:12 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:57:12 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321474120.4854.13.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321474120.4854.13.camel@monkey> Message-ID: Hi, >> DELIVER=/usr/libexec/dovecot/deliver >> >> :0 fhW >> | $DELIVER -m xspamtest > > Do you really want the pipe to be a (f)ilter? What do you expect deliver > to pass back? And you're feeding deliver the mail (h)eaders only, > dropping the body. Yes, thanks. I pasted from another filter we were using, and forgot. procmail still seems to be much better than the deliver sieve language, despite how long it's been since I've used it :-) Thanks, Alex From mysqlstudent at gmail.com Thu Nov 17 03:42:04 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 20:42:04 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: Hi, My apologies if a previous message appeared twice. Not sure how that happened... > Okay, that fixed it. It's now delivering mail properly, thanks. > > Just to be clear I understand, if I use the following .procmailrc, it > should save all messages with the X-Spam-Status header in xspamstatus, > and everything else gets delivered to /var/spool/mail/user, correct? > > DELIVER=/usr/libexec/dovecot/deliver > > :0 > * ^X-Spam-Status > | $DELIVER -m xspamstatus I thought this was working, but it doesn't create or populate a folder named xspamstatus although it executes deliver: procmail: [30333] Wed Nov 16 20:39:36 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Match on "^X-Spam-Status" procmail: Assigning "LASTFOLDER=/usr/libexec/dovecot/deliver -m xspamstatus" >From smarchant at mydomain.com.au Wed Nov 16 20:39:36 2011 procmail: Executing "/usr/libexec/dovecot/deliver,-m,mymailbox" Subject: Riverside (97293) Folder: /usr/libexec/dovecot/deliver -m mymailbox 28400 I've set lda_mailbox_autocreate to yes, but it still doesn't create it. Is this folder relative to ~user/mail/ ? Is there an index file that dovecot-lda updates for imap? Can I just eliminate it entirely and just have procmail do all the delivery? Thanks again, Alex From sf.rique at gmail.com Thu Nov 17 03:53:29 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Wed, 16 Nov 2011 23:53:29 -0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EC16116.1000902@hardwarefreak.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> Message-ID: Jan-Frode Wich clustered filesytem do you have? I used to have ocfs2 but had problems with performance. So had to get back to ext4 and it solve the performance problem... My ocfs2 setup had some problems... but still.. Some numbers: OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 15 hours Ext4 1TB of maildir files. Full backup 16 Hours Incremental 1 hour Same LUN on storage. []'sf.rique On Mon, Nov 14, 2011 at 4:42 PM, Stan Hoeppner wrote: > On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > > > Agree. A non-clustered fs should give you better performance, and > > probably also be more reliable, if you can live with the SPoF and > > full downtime during patching/upgrades/maintenance. But I would expect > > xfs to be a better choice than ext*. > > Depends on the workload characteristics and how well the XFS filesystem > is tuned to the storage hardware. If setup properly, using many > allocation groups with fast spindles, a decent amount of BBWC, and a > high concurrency maildir workload (dozens to hundreds of delivery and > IMAP operations), XFS will runs circles around EXTx as it can > create/write/read to every AG in parallel. Much of EXT4's operation is > still serialized. This is why XFS outruns all other filesystems in the > highly parallel mail workload benchmarks I posted previously, EXTx by a > factor of 2-3. > > For smaller hosts that don't see parallelism, for example SOHO servers, > XFS will likely be slower than EXTx as the workload will be serialized. > > -- > Stan > From guenther at rudersport.de Thu Nov 17 04:18:38 2011 From: guenther at rudersport.de (Karsten =?ISO-8859-1?Q?Br=E4ckelmann?=) Date: Thu, 17 Nov 2011 03:18:38 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: <1321496318.4854.62.camel@monkey> On Wed, 2011-11-16 at 20:42 -0500, Alex wrote: [...] Unrelated to other dovecot specific questions... > Is there an index file that dovecot-lda updates for imap? Yes. Which would be the advantage of using dovecot deliver, directly or called from procmail. > Can I just eliminate it entirely and just have procmail do all the > delivery? Yes, you can. In fact, that's what I usually still use. Procmail can just do much more than sieve. And procmail doesn't scare me as much as sieve. But then again, I like Perl... And I've never yet encountered a problem with dovecot IMAP updating indexes on the fly -- which it does, unless deliver does it incrementally. However, wasn't your original question about converting procmail recipes to sieve? (Yes, it was.) So what would hold you back of just not converting? -- char *t="\10pse\0r\0dtu\0. at ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} From mysqlstudent at gmail.com Thu Nov 17 04:50:52 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 21:50:52 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321496318.4854.62.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> <1321496318.4854.62.camel@monkey> Message-ID: > Unrelated to other dovecot specific questions... > >> Is there an index file that dovecot-lda updates for imap? > > Yes. Which would be the advantage of using dovecot deliver, directly or > called from procmail. > >> Can I just eliminate it entirely and just have procmail do all the >> delivery? > > Yes, you can. > > In fact, that's what I usually still use. Procmail can just do much more > than sieve. And procmail doesn't scare me as much as sieve. But then > again, I like Perl... And I've never yet encountered a problem with > dovecot IMAP updating indexes on the fly -- which it does, unless > deliver does it incrementally. > > However, wasn't your original question about converting procmail recipes > to sieve? (Yes, it was.) So what would hold you back of just not > converting? Yes, because I hadn't realized it was possible to still use procmail at all. I had said previously, although possibly not clearly, that I'd prefer to use continue to use procmail -- sieve looks entirely too convoluted, and I just don't need to do mail filtering frequently enough to have the time to figure out sieve. It also didn't support a few other things I'm currently doing with procmail. I can't figure out why "deliver -m" doesn't work as expected. It just doesn't create the mailbox. Should I just use 'c' with procmail to make a copy to forward on to deliver? DELIVER=/usr/libexec/dovecot/deliver :0 { :0c * ^X-Spam-Status xspamstatus :0 | $DELIVER } There has to be a better way... Thanks, Alex From mgoldey at goldey.net Thu Nov 17 06:48:43 2011 From: mgoldey at goldey.net (Mark Goldey) Date: Wed, 16 Nov 2011 23:48:43 -0500 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 Message-ID: <4EC4922B.5050000@goldey.net> Sorry about the HTML. I removed the string "-O2" from the /auth Makefile and, for good measure, did make clean, make and make install for all of Dovecot. Still, it crashed when trying to log in via IMAP (webmail). Core dumped, here's the backtrace. [snip] Core was generated by `dovecot/auth -w'. Program terminated with signal 11, Segmentation fault. #0 0x00000000 in ?? () (gdb) bt full #0 0x00000000 in ?? () No symbol table info available. #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, args=0x82601c8) at auth-worker-client.c:405 auth_request = 0x82776e8 userdb_id = 1 #2 0x0805b630 in auth_worker_handle_line (client=0x82754d0, line=0x82757da "2\tUSER\t1\tuser=mgoldXX at goldey.net\tservice=imap\toriginal_username=mgoldXX at goldey.net\trequested_login_user\tlip=127.0.0.1\trip=127.0.0.1\tlport=143\trport=47574\tsecured=1\tsuccessful=1\tmech=PLAIN") at auth-worker-client.c:534 args = 0x82601c0 id = 2 ret = false #3 0x0805b890 in auth_worker_input (client=0x82754d0) at auth-worker-client.c:614 _data_stack_cur_id = 3 line = 0x82757da "2\tUSER\t1\tuser=mgoldXX at goldey.net\tservice=imap\toriginal_username=mgoldXX at goldey.net\trequested_login_user\tlip=127.0.0.1\trip=127.0.0.1\tlport=143\trport=47574\tsecured=1\tsuccessful=1\tmech=PLAIN" ret = 183 #4 0xb7726021 in io_loop_call_io (io=0x8275688) at ioloop.c:384 ioloop = 0x8268398 t_id = 2 #5 0xb77270d3 in io_loop_handler_run (ioloop=0x8268398) at ioloop-epoll.c:213 ctx = 0x8273f38 event = 0x8273f78 list = 0x82756b0 io = 0xbf80049f tv = {tv_sec = 2147483, tv_usec = 0} msecs = 1 ret = 1 i = 0 j = 0 call = 192 #6 0xb7725fa0 in io_loop_run (ioloop=0x8268398) at ioloop.c:405 No locals. #7 0xb7711baa in master_service_run (service=0x82682e8, callback=0x805e466 ) at master-service.c:481 No locals. #8 0x0805e7aa in main (argc=2, argv=0x82681c0) at main.c:298 c = -1 And, for good measure: (gdb) fr 1 #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, args=0x82601c8) at auth-worker-client.c:405 405 auth_request->userdb->userdb->iface-> (gdb) p *auth_request->userdb->userdb $1 = {args = 0x8268d10 "", cache_key = 0x0, blocking = true, id = 1, init_refcount = 1, iface = 0x807b320} (gdb) p *auth_request->userdb->userdb->iface $2 = {name = 0x8074f3d "vpopmail", preinit = 0x8069a27 , init = 0, deinit = 0, lookup = 0x806975f , iterate_init = 0, iterate_next = 0, iterate_deinit = 0} This is all Greek to me, but I'm happy to do more troubleshooting. --mhg From janfrode at tanso.net Thu Nov 17 11:34:49 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 17 Nov 2011 10:34:49 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> Message-ID: <20111117093449.GA18157@dibs.tanso.net> On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > Wich clustered filesytem do you have? We use IBM GPFS, with currently 7 servers working against shared LUNs from an IBM DS4800. > > My ocfs2 setup had some problems... but still.. > Some numbers: > > OCFS2 > 1TB of maildir files. > Full backup 36 Hours > Incremental 15 hours > > Ext4 > 1TB of maildir files. > Full backup 16 Hours > Incremental 1 hour Wow, local fs's are fast! We have split the backup process up to run on 6 of the servers, with each server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). The backup processing time varies quite a bit every day, but are mostly somewhere between 14-24 hours on each server. This sounds like something between 1.5x to 2x the incremental-performance you're seeing: 15 hours/incremental of 1TB ocfs2 = 15h/TB 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB 6x 20 hours for incremental of 12 TB GPFS = 10h/TB All our backups are incremental. -jf From stan at hardwarefreak.com Thu Nov 17 12:27:50 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 17 Nov 2011 04:27:50 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <1321383763.21919.484.camel@hurina> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> <1321383763.21919.484.camel@hurina> Message-ID: <4EC4E1A6.7060101@hardwarefreak.com> On 11/15/2011 1:02 PM, Timo Sirainen wrote: > On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > >> This is why I recommended mbox in the first place. If your only writes >> to these mailbox files are appends of new messages, mbox is the best >> format by far. It's faster at appending than any other format, and it's >> faster for searching than any other. > > Just as long as you're not simultaneously trying to read and write the > mbox file (or just write in 2+ sessions). Then there's a lot waiting on > locks. (mdbox has no read locks, and its write locks are very short > lived.) Of course. My understanding of Alexander's workflow is that copies of all daily new mail are written to an IMAP mailbox via some MTA bcc rule or sieve rule. A nightly script moves the daily mail to another mailbox created and named by date. These named mailboxes are then used for backup and the search function, but are never written to again. So I assume there is no simultaneous read/write of the archive mailboxes he performs searches on. It's possible I don't fully understand Alexander's work flow yet. -- Stan From Mark.Zealey at webfusion.com Thu Nov 17 12:57:09 2011 From: Mark.Zealey at webfusion.com (Mark Zealey) Date: Thu, 17 Nov 2011 10:57:09 +0000 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <1321467929.21919.583.camel@hurina> References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> <4EB7BF9A.9020900@webfusion.com>,<1321467929.21919.583.camel@hurina> Message-ID: Thanks so much for that have tested in 2.0.16 release now and is working fine. Mark ________________________________________ From: Timo Sirainen [tss at iki.fi] Sent: 16 November 2011 18:25 To: Mark Zealey Cc: Dovecot Mailing List Subject: Re: [Dovecot] Port variable in LMTP userdb lookups? On Mon, 2011-11-07 at 13:23 +0200, Mark Zealey wrote: > 04-11-2011 23:56, Timo Sirainen yazm??: > From my reading of the code in 2.0.15, you pass the local & remote IP > in lmtp/commands.c:427 but the struct mail_storage_service_input only > has remote_ip and local_ip fields, not port fields (unlike the > auth_user_info struct which has both ip's and port entries as well). I > think that the mail_storage_service_input struct needs the uint > local_port, remote_port fields adding in & appropriate code changes to > pass these through? Yeah, you're right, I missed that one. Added to hg now. From marco at carcano.ch Thu Nov 17 18:47:34 2011 From: marco at carcano.ch (Marco Carcano) Date: Thu, 17 Nov 2011 17:47:34 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <1320437339.21919.165.camel@hurina> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: Hello Timo, and thanks for your reply I waited to reply until got it another time as I already said, it does not happen very often, for example it happened on 12 november - the log is at the end of this mail > Enable mail_log plugin to make sure of this. > http://wiki2.dovecot.org/Plugins/MailLog I already did, but just for a few days: it does not happen very often that we loose mails, so I'm afraid I can damage the disks of the production server if I keep logging enabled for too much time - it will be a pain, years ago I had a server damaged because of logging enabled for too much time. I do not want to repeat such a painfull experience > >> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>> : saved mail to INBOX > > If Dovecot logs this, then the message definitely was saved to INBOX. it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - I suspect however that could be mine misconfiguration somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs (however I'm wondering why sometimes?!?) I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are cd /home/mailboxstore/theuser/Maildir grep "6000029222" */* |grep "RE:" grep "6000029222" .Drafts/* |grep "RE:" grep "6000029222" .Drafts/*/* |grep "RE:" grep "6000029222" .Junk/* |grep "RE:" grep "6000029222" .Posta\ eliminata/* |grep "RE:" grep "6000029222" .Posta\ indesiderata/* |grep "RE:" grep "6000029222" .Posta\ inviata/* |grep "RE:" grep "6000029222" .Sent/* |grep "RE:" grep "6000029222" .Templates/* |grep "RE:" grep "6000029222" .Trash/* |grep "RE:" and never got anything here is the log instead Nov 12 08:48:01 srv001 postfix/smtpd[1430]: connect from mail.tasnee.com[62.3.52.58] Nov 12 08:48:02 srv001 postfix/smtpd[1430]: 6C3874E4A9F: client=mail.tasnee.com[62.3.52.58] Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: warning: header Subject: RE: RFQ NO. 6000029222 - OUR OFFER NO. 2111221 from mail.tasnee.com[62.3.52.58]; from= to= proto=ESMTP helo= Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa > Nov 12 08:48:03 srv001 postfix/qmgr[4876]: 6C3874E4A9F: from=, size=9920, nrcpt=4 (queue active) Nov 12 08:48:06 srv001 postfix/smtpd[1442]: connect from localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 postfix/smtpd[1442]: 244774E4AA2: client=localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 postfix/cleanup[1434]: 244774E4AA2: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: from=, size=10323, nrcpt=4 (queue active) Nov 12 08:48:06 srv001 postfix/smtpd[1442]: disconnect from localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 amavis[8902]: (08902-05) Passed CLEAN, [62.3.52.58] [62.3.52.58] -> < user2 @ourdomain.local>,,,, Message-ID: <899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >, mail_id: z4aAgl2gBrfV, Hits: -0.592, size: 9919, queued_as: 244774E4AA2, 2632 ms Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 6C3874E4A9F: removed Nov 12 08:48:06 srv001 dovecot: lda(user3): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 dovecot: lda(user2): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1452]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.02/0/0.05, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 postfix/pipe[1445]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.01/0/0.06, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 dovecot: lda(theuser): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1447]: 244774E4AA2: to=, relay=dovecot, delay=0.09, delays=0.01/0.01/0/0.07, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 dovecot: lda(user4): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1450]: 244774E4AA2: to=, relay=dovecot, delay=0.26, delays=0.01/0.02/0/0.23, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: removed Nov 12 08:48:08 srv001 postfix/smtpd[1430]: disconnect from mail.tasnee.com[62.3.52.58] as you can see again, the mail seems delivered, but is not I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the whole config) thank you Marco Carcano From bind at enas.net Thu Nov 17 19:29:03 2011 From: bind at enas.net (Urban Loesch) Date: Thu, 17 Nov 2011 18:29:03 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: <4EC5445F.8@enas.net> Hi, On 17.11.2011 17:47, Marco Carcano wrote: > Hello Timo, and thanks for your reply > > I waited to reply until got it another time > > as I already said, it does not happen very often, for example it happened on 12 november - the log is at the end of this mail > > >> Enable mail_log plugin to make sure of this. >> http://wiki2.dovecot.org/Plugins/MailLog > > I already did, but just for a few days: it does not happen very often that we loose mails, so I'm afraid I can damage the disks of the production > server if I keep logging enabled for too much time - it will be a pain, years ago I had a server damaged because of logging enabled for too much time. > I do not want to repeat such a painfull experience I think logging is not a main reason for damaging disks. I have enabled mail_log plugin since march 2011 without problems and it helps me very often in such cases. > >> >>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>> : saved mail to INBOX >> >> If Dovecot logs this, then the message definitely was saved to INBOX. > > it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - I suspect however that could be mine misconfiguration > somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs > (however I'm wondering why sometimes?!?) > Could it be that some other person is downloading the mail via pop3 and then the client is deleting it from the server? This happens to me sometimes when a customer is accessing his account with a new client but forgot to disable the same account on his old PC. So it happens, that the old pc downloads all new mails, delete it then and the customer never see new mails on his new client. Regards Urban > I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are > > cd /home/mailboxstore/theuser/Maildir > > grep "6000029222" */* |grep "RE:" > grep "6000029222" .Drafts/* |grep "RE:" > grep "6000029222" .Drafts/*/* |grep "RE:" > grep "6000029222" .Junk/* |grep "RE:" > grep "6000029222" .Posta\ eliminata/* |grep "RE:" > grep "6000029222" .Posta\ indesiderata/* |grep "RE:" > grep "6000029222" .Posta\ inviata/* |grep "RE:" > grep "6000029222" .Sent/* |grep "RE:" > grep "6000029222" .Templates/* |grep "RE:" > grep "6000029222" .Trash/* |grep "RE:" > > and never got anything > > here is the log instead > > Nov 12 08:48:01 srv001 postfix/smtpd[1430]: connect from mail.tasnee.com[62.3.52.58] > Nov 12 08:48:02 srv001 postfix/smtpd[1430]: 6C3874E4A9F: client=mail.tasnee.com[62.3.52.58] > Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: warning: header Subject: RE: RFQ NO. 6000029222 - OUR OFFER NO. 2111221 from > mail.tasnee.com[62.3.52.58]; from= to= proto=ESMTP helo= > Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa> > Nov 12 08:48:03 srv001 postfix/qmgr[4876]: 6C3874E4A9F: from=, size=9920, nrcpt=4 (queue active) > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: connect from localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: 244774E4AA2: client=localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 postfix/cleanup[1434]: 244774E4AA2: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa> > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: from=, size=10323, nrcpt=4 (queue active) > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: disconnect from localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 amavis[8902]: (08902-05) Passed CLEAN, [62.3.52.58] [62.3.52.58] -> > ,,,, Message-ID: > <899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>, mail_id: z4aAgl2gBrfV, Hits: -0.592, size: 9919, queued_as: 244774E4AA2, 2632 ms > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, > relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: > queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 6C3874E4A9F: removed > Nov 12 08:48:06 srv001 dovecot: lda(user3): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 dovecot: lda(user2): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1452]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.02/0/0.05, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 postfix/pipe[1445]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.01/0/0.06, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 dovecot: lda(theuser): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1447]: 244774E4AA2: to=, relay=dovecot, delay=0.09, delays=0.01/0.01/0/0.07, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 dovecot: lda(user4): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1450]: 244774E4AA2: to=, relay=dovecot, delay=0.26, delays=0.01/0.02/0/0.23, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: removed > Nov 12 08:48:08 srv001 postfix/smtpd[1430]: disconnect from mail.tasnee.com[62.3.52.58] > > as you can see again, the mail seems delivered, but is not > > I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the > whole config) > > > thank you > > Marco Carcano > > > > > From tss at iki.fi Thu Nov 17 19:34:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 19:34:01 +0200 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> On 17.11.2011, at 18.47, Marco Carcano wrote: >>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>> : saved mail to INBOX >> >> If Dovecot logs this, then the message definitely was saved to INBOX. > > it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - Most likely reason for this is that the user's client deletes the message. Possibly an automatic client side filter or some UI issue that causes user to accidentally delete a mail. The mail_log plugin's logging would have showed if this was the case. > I suspect however that could be mine misconfiguration somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs (however I'm wondering why sometimes?!?) I can't think of any reason why it would randomly write to a wrong place. > I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are > > cd /home/mailboxstore/theuser/Maildir > > grep "6000029222" */* |grep "RE:" > grep "6000029222" .Drafts/* |grep "RE:" > grep "6000029222" .Drafts/*/* |grep "RE:" > grep "6000029222" .Junk/* |grep "RE:" > grep "6000029222" .Posta\ eliminata/* |grep "RE:" > grep "6000029222" .Posta\ indesiderata/* |grep "RE:" > grep "6000029222" .Posta\ inviata/* |grep "RE:" > grep "6000029222" .Sent/* |grep "RE:" > grep "6000029222" .Templates/* |grep "RE:" > grep "6000029222" .Trash/* |grep "RE:" Only the grep "6000029222" .Drafts/*/* |grep "RE:" was grepping from mail files. Easier would be just: grep -r "RE:.*6000029222" . > I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the whole config) I doubt this is related to configuration. But you could enable http://wiki2.dovecot.org/Plugins/Lazyexpunge so that messages won't be lost if they are expunged. The next time a message is lost, you would most likely find it from the lazy-expunge namespace. (Then you could write a script that deletes e.g. >1 week old files nightly.) From sf.rique at gmail.com Thu Nov 17 19:38:18 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Thu, 17 Nov 2011 15:38:18 -0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111117093449.GA18157@dibs.tanso.net> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> <20111117093449.GA18157@dibs.tanso.net> Message-ID: []'sf.rique How many servers have access to your maildir on ext4 so that you could > switch from ocfs2 to ext4? > I use ocfs2 in my test environment for four servers (2 MX and 2 IMAP) I used have 3 serves one mailman and 2 imaps. I experence that if take my loadbalancer and takes it all to just one server, itwould go faster becasue of cache. But once thsi server fails, the loadbalaner send it to the other one, but it simple does not work, clients experence timeouts, because the filesystem were too slow. On Thu, Nov 17, 2011 at 7:34 AM, Jan-Frode Myklebust wrote: On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > > > Wich clustered filesytem do you have? > > We use IBM GPFS, with currently 7 servers working against shared LUNs > from an IBM DS4800. > > > > > My ocfs2 setup had some problems... but still.. > > Some numbers: > > > > OCFS2 > > 1TB of maildir files. > > Full backup 36 Hours > > Incremental 15 hours > > > > Ext4 > > 1TB of maildir files. > > Full backup 16 Hours > > Incremental 1 hour > > Wow, local fs's are fast! > Much faster! I could not run a "du" before, if i did would crash the system, now i can! > We have split the backup process up to run on 6 of the servers, with each > server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). > The backup processing time varies quite a bit every day, but are mostly > somewhere between 14-24 hours on each server. This sounds like something > between 1.5x to 2x the incremental-performance you're seeing: > > 15 hours/incremental of 1TB ocfs2 = 15h/TB > 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB > 6x 20 hours for incremental of 12 TB GPFS = 10h/TB > > All our backups are incremental. > > > -jf > From e-frog at gmx.de Thu Nov 17 19:45:50 2011 From: e-frog at gmx.de (e-frog) Date: Thu, 17 Nov 2011 18:45:50 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <1321466541.21919.576.camel@hurina> References: <4EB6AFE3.3020001@gmx.de> <4EBACFC0.6060703@gmx.de> <1321466541.21919.576.camel@hurina> Message-ID: <4EC5484E.1040709@gmx.de> On 16.11.2011 19:02, wroteTimo Sirainen: > On Wed, 2011-11-09 at 20:08 +0100, e-frog wrote: >> On 06.11.2011 17:03, wrote e-frog: >>> 1 lsub "" "virtual/*" >>> * LSUB () "/" "virtual" >>> * LSUB () "/" "virtual/Lists" >>> * LSUB () "/" "virtual/unread" >>> 1 OK Lsub completed. >> >> Just tested this again with 2.1beta1 with the same result. > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/16024d03f66e > Thanks, Timo. Tested and works now as expected. From marco at carcano.ch Thu Nov 17 22:50:09 2011 From: marco at carcano.ch (Marco Carcano) Date: Thu, 17 Nov 2011 21:50:09 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> Message-ID: <126C0E4E-460B-447D-A85C-9AEC956020B5@carcano.ch> Hello Timo and Urban, and thank you for the very quick reply I have already thought about these things: I have already disabled mailbox deletion from POP3 clients, that actually just leave the messages on the server however, this time noone of the 4 recipients got the email - so it is really strange that all che clients deleted the same message and left all the others we can notice this only a few days after the issue, because customers send us a reminder forwarding the lost mail Il giorno 17/nov/11, alle ore 18:34, Timo Sirainen ha scritto: > On 17.11.2011, at 18.47, Marco Carcano wrote: > >>>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>>> : saved mail to INBOX >>> >>> If Dovecot logs this, then the message definitely was saved to >>> INBOX. >> >> it is exactly what I told to my colleagues, but belive me, sometime >> some mail get lost - > > Most likely reason for this is that the user's client deletes the > message. Possibly an automatic client side filter or some UI issue > that causes user to accidentally delete a mail. The mail_log > plugin's logging would have showed if this was the case. > >> I suspect however that could be mine misconfiguration somewhere, so >> that lda sometimes write the email not in the right place, but >> elsewhere, and just write the phrase " saved mail to INBOX in the >> logs (however I'm wondering why sometimes?!?) > > I can't think of any reason why it would randomly write to a wrong > place. maybe something related to LDAP lookups? what can happen if for some reason the LDAP server does not reply to the lookup? Timo, belive to me, I use dovecot since 3 years, and I am one of the thousands of happy user, I really can't explain what is happening on this server I'll do this trial: I'll enable lda debug logging ont a file on a USB hard disk - in order to know where exactly messages are stored, and enable also maillog too I'll keep you informed on what I'll find - maybe it will take some days, as I told it happens quite seldom thank you > >> I tried to find the missed email in the Maildir, but have not been >> able to get it - the commands used are >> >> cd /home/mailboxstore/theuser/Maildir >> >> grep "6000029222" */* |grep "RE:" >> grep "6000029222" .Drafts/* |grep "RE:" >> grep "6000029222" .Drafts/*/* |grep "RE:" >> grep "6000029222" .Junk/* |grep "RE:" >> grep "6000029222" .Posta\ eliminata/* |grep "RE:" >> grep "6000029222" .Posta\ indesiderata/* |grep "RE:" >> grep "6000029222" .Posta\ inviata/* |grep "RE:" >> grep "6000029222" .Sent/* |grep "RE:" >> grep "6000029222" .Templates/* |grep "RE:" >> grep "6000029222" .Trash/* |grep "RE:" > > Only the grep "6000029222" .Drafts/*/* |grep "RE:" was grepping from > mail files. Easier would be just: > > grep -r "RE:.*6000029222" . > >> I really think is some kind of misconfiguration of mine, may you >> help me, please? Just tell me what pieces of config to show (just >> not to flood the whole config) > > I doubt this is related to configuration. > > But you could enable http://wiki2.dovecot.org/Plugins/Lazyexpunge so > that messages won't be lost if they are expunged. The next time a > message is lost, you would most likely find it from the lazy-expunge > namespace. (Then you could write a script that deletes e.g. >1 week > old files nightly.) From andy.robbins at firespring.com Thu Nov 17 23:30:07 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 15:30:07 -0600 Subject: [Dovecot] Dovecot performance issues with many writes Message-ID: We are currently experiencing performance issue with our Dovecot system which we believe is caused by excessive writes to the dovecot files. The confusing thing is that we are seeing more writes than reads on our Dovecot volume when you would assume that most of the IO should be reads from customers checking their mail. We're seeing reads vs. writes similar to the following: # iostat -d 5 -x Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 8.00 28.69 19.63 0.70 94.00 drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 8.00 111.42 94.44 0.74 99.60 We can't figure out why we would be seeing twice as many writes as we are reads. Could this have something to do with the index files? Appreciate any assistance on this issue. From micah at riseup.net Thu Nov 17 23:33:37 2011 From: micah at riseup.net (micah anderson) Date: Thu, 17 Nov 2011 16:33:37 -0500 Subject: [Dovecot] dsync problems In-Reply-To: <1321389804.21919.530.camel@hurina> References: <871ut9i3z1.fsf@algae.riseup.net> <1321389804.21919.530.camel@hurina> Message-ID: <87d3cqe8ni.fsf@algae.riseup.net> On Tue, 15 Nov 2011 22:43:24 +0200, Timo Sirainen wrote: > On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > > When a user renames their username, I am using dsync to copy their mail > > over to the new username's mail location[0]. > > > > Some of the dsyncs are failing with errors that I dont know how to work > > with, for example: > > > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > > dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > > These shouldn't really happen. Something's going internally wrong with > dsync. Can you reproduce this reliably somehow? Well, I dont know if I can do it reliably, but its been happening a lot. One point of information that might be useful is that these users were converted from courier maildir to mdbox, and their courier bits are still around in the source mailbox (I haven't become brave enough to remove them yet). > > > The errors cause a non-zero exit code from dsync, which causes my rename > > script to bail out. What are these errors, and how can I fix them? > > Does a second dsync on error succeed? :) Before I tried it again, I looked at their mailboxes: # doveadm mailbox list -u ): Info: INBOX: only in dest (guid=14bf0409fa08c04e682900007ef44c7a) dsync(): Error: Trying to open a non-listed mailbox with guid=14bf0409fa08c04e682900007ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 14bf0409fa08c04e682900007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=14bf0409fa08c04e682900007ef44c7a and the mailboxes: # doveadm mailbox list -u Trash_084ed82bc59ca54eb53700007ef44c7a Sent Drafts INBOX_094ed82bc59ca54eb53700007ef44c7a I tried it a third time, and... it worked, no error, and now their mailbox list: # doveadm mailbox list -u Trash Sent Drafts INBOX This particular user only has one test email > > > 0. Why use dsync instead of a simple mv operation? This seems to be > > necessary for two corner cases: > > > > 1. dovecot creates the new mailbox automatically when the user logs in > > or receives a mail, so if the user changes their mail and logs in or > > receives an email before the move has been done, then the mailbox is > > created and then a move command will fail. > > > > 2. If there has been new mail created under the new name, we can't just > > simply remove the stuff that is automatically created there and replace > > it with the old things because we could potentially be removing mail > > that has been delivered in the mean time. > > You could temporarily change the permissions for the home directory so > that no new mailboxes/mails could be created during the move (e.g. 0700 > root). The problem is there are a number of users on the system and all the mail is stored under /srv/mailstorage//. So if "foo" wants to change their username to "bar" -- I dont have a deterministic way of determining that "bar" exists yet because mail could be delivered or they could login and dovecot would create it and I can't set /srv/mailstorage/ 0700 root or nobody would be able to receive mail. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Thu Nov 17 23:51:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 23:51:06 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> On 17.11.2011, at 23.30, Andy Robbins wrote: > We can't figure out why we would be seeing twice as many writes as we are > reads. Could this have something to do with the index files? dovecot -n output would be helpful. From andy.robbins at firespring.com Fri Nov 18 00:20:15 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 16:20:15 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> Message-ID: # dovecot -n # 1.2.9: //etc/dovecot.conf # OS: Linux 2.6.18-164.9.1.el5 x86_64 CentOS release 5.4 (Final) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log protocols: imap pop3 ssl_cert_file: /etc/postfix/ssl/smtp1.domain.com.cert ssl_key_file: /etc/postfix/ssl/smtp1.domain.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): //libexec/dovecot/imap-login login_executable(imap): //libexec/dovecot/imap-login login_executable(pop3): //libexec/dovecot/pop3-login login_user: postfix login_greeting: Welcome login_process_per_connection: no login_processes_count: 2 mail_max_userip_connections: 50 mail_uid: 500 mail_gid: 500 mail_location: maildir:~/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /etc/dovecot/last-login-imap.sh mail_executable(imap): /etc/dovecot/last-login-imap.sh mail_executable(pop3): /etc/dovecot/last-login-pop3.sh mail_plugins: quota mail_plugin_dir(default): //lib/dovecot/imap mail_plugin_dir(imap): //lib/dovecot/imap mail_plugin_dir(pop3): //lib/dovecot/pop3 pop3_uidl_format: %f lda: postmaster_address: postmaster at domain.com auth_socket_path: /var/run/dovecot/auth-master log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver.log fsync_disable: no auth default: mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP user: vmail username_translation: %@ verbose: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: maildir:User quota quota_rule: *:storage=0 From tss at iki.fi Fri Nov 18 00:34:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 00:34:33 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> Message-ID: <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> On 18.11.2011, at 0.20, Andy Robbins wrote: > # iostat -d 5 -x > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz > avgqu-sz await svctm %util > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > 8.00 28.69 19.63 0.70 94.00 > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > 8.00 111.42 94.44 0.74 99.60 These devices don't look like NFS, but you have: > mmap_disable: yes > mail_nfs_storage: yes > mail_nfs_index: yes So what filesystem is that? Are they storing only mails, or other stuff too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing things up at least somewhat.. > lda: > fsync_disable: no This is explicitly set for LDA, but it's the default already elsewhere. Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? From janfrode at tanso.net Fri Nov 18 00:46:30 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 17 Nov 2011 23:46:30 +0100 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <20111117224630.GA1315@dibs.tanso.net> On Thu, Nov 17, 2011 at 03:30:07PM -0600, Andy Robbins wrote: > We are currently experiencing performance issue with our Dovecot system > which we believe is caused by excessive writes to the dovecot files. The > confusing thing is that we are seeing more writes than reads on our Dovecot > volume when you would assume that most of the IO should be reads from > customers checking their mail. Have you mounted the fs without noatime options? -jf From andy.robbins at firespring.com Fri Nov 18 00:38:46 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 16:38:46 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Well, the iostat command was run from the NFS server and dovecot was run from the mail server where it is mounted, hence the discrepancy there. On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > # iostat -d 5 -x > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > avgrq-sz > > avgqu-sz await svctm %util > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 28.69 19.63 0.70 94.00 > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 111.42 94.44 0.74 99.60 > > These devices don't look like NFS, but you have: > > > mmap_disable: yes > > mail_nfs_storage: yes > > mail_nfs_index: yes > > So what filesystem is that? Are they storing only mails, or other stuff > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > things up at least somewhat.. > > > lda: > > fsync_disable: no > > This is explicitly set for LDA, but it's the default already elsewhere. > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > From tss at iki.fi Fri Nov 18 00:42:12 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 00:42:12 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Do you have more than one Dovecot server? If only one, you can still disable the mail_nfs_* settings. Also it would be interesting to see nfsstat numbers from the Dovecot server, compared to those iostat numbers.. On 18.11.2011, at 0.38, Andy Robbins wrote: > Well, the iostat command was run from the NFS server and dovecot was run from the mail server where it is mounted, hence the discrepancy there. > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > # iostat -d 5 -x > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz > > avgqu-sz await svctm %util > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 28.69 19.63 0.70 94.00 > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 111.42 94.44 0.74 99.60 > > These devices don't look like NFS, but you have: > > > mmap_disable: yes > > mail_nfs_storage: yes > > mail_nfs_index: yes > > So what filesystem is that? Are they storing only mails, or other stuff too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing things up at least somewhat.. > > > lda: > > fsync_disable: no > > This is explicitly set for LDA, but it's the default already elsewhere. Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > From p.heinlein at heinlein-support.de Fri Nov 18 01:43:01 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Fri, 18 Nov 2011 00:43:01 +0100 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <201111180043.01419.p.heinlein@heinlein-support.de> Am Donnerstag, 17. November 2011, 22:30:07 schrieb Andy Robbins: > We are currently experiencing performance issue with our Dovecot > system which we believe is caused by excessive writes to the dovecot > files. The confusing thing is that we are seeing more writes than > reads on our Dovecot volume when you would assume that most of the > IO should be reads from customers checking their mail. Yes, but your read-requests are answered from your local kernel-buffer / cache and aren't really read from your hard-disk. (At least if you have enough free memory for a cache, that's large enought.) Incoming mails ARE written on the disk, but read later on just from your cache. Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From victormanuelo at gmail.com Fri Nov 18 05:24:53 2011 From: victormanuelo at gmail.com (Victor) Date: Thu, 17 Nov 2011 22:54:53 -0430 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 Message-ID: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> Hi, I'm new in this List, but I have 6 years using Dovecot on my debian from etch,lenny and now squeeze Package: dovecot-imapd Version: 1:1.2.15-4 Tags: squeeze -- System Information: Debian Release: 6.0 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/24 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages dovecot-imapd depends on: ii dovecot-common 1:1.2.15-4 secure mail server that supports m ii libc6 2.11.2-10 Embedded GNU C Library: Shared lib ii libldap-2.4-2 2.4.23-7 OpenLDAP libraries ii libssl0.9.8 0.9.8o-4 SSL shared libraries the quota_imap plugin not working when I use against MailquotaSize Attribute of LDAP users. this setup works perfectly on lenny and dovecot-imapd 1:1.0.15-2.3 but on squeeze dovecot-imad 1:1.2.15-7 doesn't work, the service is up, but The quota does not display. In logs nothing happens the configuration in dovecot.conf below protocol imap { mail_plugins = quota imap_quota } protocol lda { mail_plugins = quota } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } the configuration in dovecot-ldap.conf below uris = ldap://server-ldap.com.ve auth_bind = yes ldap_version = 3 base = dc=dominio,dc=com,dc=uk deref = never scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota=dirsize:storage user_filter = (&(objectClass=posixAccount)(uid=%u)) pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid pass_filter = (&(objectClass=posixAccount)(uid=%u)) This is the same configuration in the 6 last years. So, I would really appreciate if give me a hand which are more than 15000 Users who administered easily taking the quota in the LDAP attribute. and I want to keep the scheme. Thz Victor Onate Usuario Linux From oni-neko at gmx.net Fri Nov 18 15:34:28 2011 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Fri, 18 Nov 2011 14:34:28 +0100 Subject: [Dovecot] seting acl for master user? Message-ID: <20111118133428.63140@gmx.net> Hello! I'm sorry, but i don't really understand it the way it is written in the wiki, so i'm trying my luck here =) i followed the wiki for creating a master user (the non-sql-version) and i can login as another user via this master user. my dovecot version is 1.2.9 on an ubuntu server 10.4.3 lts. what i don't get is the acl-setting, so that the master-user can access all the other users mails and folders. i made the changes in the dovecot.conf as written in the wiki: protocol imap { mail_plugins = acl imap_acl } protocol lda { mail_plugins = sieve acl } plugin { acl = vfile:/etc/dovecot/acls } in my case I only want the master user (called cyrus) to be able to do everything with/in all mailboxes, so I made a file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". i'm pretty sure i'm missing something rather obvious here, but somehow i'm just stuck. any help would be greatly appreciated. thanks in advance =) silvana -- NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zur?ck-Garantie! Jetzt informieren: http://www.gmx.net/de/go/freephone From robert at schetterer.org Fri Nov 18 15:47:29 2011 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 18 Nov 2011 14:47:29 +0100 Subject: [Dovecot] seting acl for master user? In-Reply-To: <20111118133428.63140@gmx.net> References: <20111118133428.63140@gmx.net> Message-ID: <4EC661F1.4000201@schetterer.org> Am 18.11.2011 14:34, schrieb oni-neko at gmx.net: > Hello! > > I'm sorry, but i don't really understand it the way it is written in the wiki, so i'm trying my luck here =) > i followed the wiki for creating a master user (the non-sql-version) and i can login as another user via this master user. > my dovecot version is 1.2.9 on an ubuntu server 10.4.3 lts. > > what i don't get is the acl-setting, so that the master-user can access all the other users mails and folders. > i made the changes in the dovecot.conf as written in the wiki: > protocol imap { > mail_plugins = acl imap_acl > } > protocol lda { > mail_plugins = sieve acl > } > plugin { > acl = vfile:/etc/dovecot/acls > } > > in my case I only want the master user (called cyrus) to be able to do everything with/in all mailboxes, so I made a file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". > > i'm pretty sure i'm missing something rather obvious here, but somehow i'm just stuck. any help would be greatly appreciated. thanks in advance =) > > silvana are you sure that you dont mixed up master users with acl normally you use master user like described here http://wiki.dovecot.org/Authentication/MasterUsers -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dave.mehler at gmail.com Fri Nov 18 16:27:42 2011 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 18 Nov 2011 09:27:42 -0500 Subject: [Dovecot] setting up multiple listeners Message-ID: Hello, I'm using Dovecot v2 and am atempting to get webmail going. I'm trying to set up imap so that port 993 only is listening on the external interface, for both ipv4 and ipv6, but that on localhost only again for ipv4 and ipv6 unencrypted port 143 is active. I've tried various combinations but keep getting an error invalid number. I've googled and not come up with the missing point. Thanks. Dave. From tss at iki.fi Fri Nov 18 19:22:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:22:02 +0200 Subject: [Dovecot] seting acl for master user? In-Reply-To: <20111118133428.63140@gmx.net> References: <20111118133428.63140@gmx.net> Message-ID: <1321636922.11489.46.camel@hurina> On Fri, 2011-11-18 at 14:34 +0100, oni-neko at gmx.net wrote: > what i don't get is the acl-setting, so that the master-user can > access all the other users mails and folders. .. > in my case I only want the master user (called cyrus) to be able to do > everything with/in all mailboxes, so I made a > file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". Unfortunately currently there is no default ACL file. The above would create an ACL file for a mailbox called "default". I'll need to get the default ACLs implemented some day.. But if you don't need ACLs for anything else, you can simply disable the ACL plugin and the master user will have access to everything. If you do need ACLs, there are some kludges you can still do. From tss at iki.fi Fri Nov 18 19:24:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:24:31 +0200 Subject: [Dovecot] setting up multiple listeners In-Reply-To: References: Message-ID: <1321637071.11489.48.camel@hurina> On Fri, 2011-11-18 at 09:27 -0500, David Mehler wrote: > Hello, > > I'm using Dovecot v2 and am atempting to get webmail going. I'm trying > to set up imap so that port 993 only is listening on the external > interface, for both ipv4 and ipv6, but that on localhost only again > for ipv4 and ipv6 unencrypted port 143 is active. I've tried various > combinations but keep getting an error invalid number. I've googled > and not come up with the missing point. A full error message would have been nice, but basically you can do: service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = 1.2.3.4 f00::baa # your external ip addresses } } From geek at uniserve.com Fri Nov 18 19:25:35 2011 From: geek at uniserve.com (Dave Stevens) Date: Fri, 18 Nov 2011 09:25:35 -0800 Subject: [Dovecot] CCing outbound IMAP mail? Message-ID: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> For some users on my site I'd like to be able to CC a particular user on all outgoing IMAP mails. Incoming I can do but don't see how to do outgoing. Ideas? Dave -- It is told that such are the aerodynamics and wing loading of the bumblebee that, in principle, it cannot fly...if all this be true...life among bumblebees must bear a remarkable resemblance to life in the United States. -- John Kenneth Galbraith, in American Capitalism: The Concept of Countervailing Power From tss at iki.fi Fri Nov 18 19:27:34 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:27:34 +0200 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 In-Reply-To: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> References: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> Message-ID: <1321637255.11489.50.camel@hurina> On Thu, 2011-11-17 at 22:54 -0430, Victor wrote: > the quota_imap plugin not working when I use against MailquotaSize Attribute of LDAP users. > this setup works perfectly on lenny and dovecot-imapd 1:1.0.15-2.3 but on squeeze dovecot-imad > 1:1.2.15-7 doesn't work, the service is up, but The quota does not display. In logs nothing happens The quota configuration changed between v1.0 and v1.1. > user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota=dirsize:storage > user_filter = (&(objectClass=posixAccount)(uid=%u)) maildirQuotaSize=quota_rule=*:storage=%$ and to dovecot.conf add: plugin { quota = dirsize } From tss at iki.fi Fri Nov 18 19:30:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:30:08 +0200 Subject: [Dovecot] CCing outbound IMAP mail? In-Reply-To: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> References: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> Message-ID: <1321637408.11489.53.camel@hurina> On Fri, 2011-11-18 at 09:25 -0800, Dave Stevens wrote: > For some users on my site I'd like to be able to CC a particular user > on all outgoing IMAP mails. Incoming I can do but don't see how to do > outgoing. Ideas? Typically the users would be sending the outgoing mails via your SMTP server and you should be able to do it the same way in MTA. If your users aren't sending mails via your SMTP server, then it depends on if they are saving the mails to the "Sent" mailbox on IMAP server. That "Sent" mailbox could be named different things, and in any case there's really no simple way to trigger sending a mail when a mail is being saved via IMAP.. From robert at schetterer.org Fri Nov 18 19:33:03 2011 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 18 Nov 2011 18:33:03 +0100 Subject: [Dovecot] CCing outbound IMAP mail? In-Reply-To: <1321637408.11489.53.camel@hurina> References: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> <1321637408.11489.53.camel@hurina> Message-ID: <4EC696CF.3090807@schetterer.org> Am 18.11.2011 18:30, schrieb Timo Sirainen: > On Fri, 2011-11-18 at 09:25 -0800, Dave Stevens wrote: >> For some users on my site I'd like to be able to CC a particular user >> on all outgoing IMAP mails. Incoming I can do but don't see how to do >> outgoing. Ideas? > > Typically the users would be sending the outgoing mails via your SMTP > server and you should be able to do it the same way in MTA. > > If your users aren't sending mails via your SMTP server, then it depends > on if they are saving the mails to the "Sent" mailbox on IMAP server. > That "Sent" mailbox could be named different things, and in any case > there's really no simple way to trigger sending a mail when a mail is > being saved via IMAP.. > > perhaps not the right thing in postfix there is sender_bcc_maps (default: empty) Optional BCC (blind carbon-copy) address lookup tables, indexed by sender address. The BCC address (multiple results are not supported) is added when mail enters from outside of Postfix. This feature is available in Postfix 2.1 and later. The table search order is as follows: Look up the "user+extension at domain.tld" address including the optional address extension. Look up the "user at domain.tld" address without the optional address extension. Look up the "user+extension" address local part when the sender domain equals $myorigin, $mydestination, $inet_interfaces or $proxy_interfaces. Look up the "user" address local part when the sender domain equals $myorigin, $mydestination, $inet_interfaces or $proxy_interfaces. Look up the "@domain.tld" part. Specify the types and names of databases to use. After change, run "postmap /etc/postfix/sender_bcc". Note: if mail to the BCC address bounces it will be returned to the sender. -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stephan at rename-it.nl Sat Nov 19 18:34:38 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 19 Nov 2011 17:34:38 +0100 Subject: [Dovecot] Released Pigeonhole v0.2.5 for Dovecot v2.0.16 Message-ID: <4EC7DA9E.6010601@rename-it.nl> Hello Dovecot users, Before I move active development of Pigeonhole to Dovecot v2.1, I first release all the pending bug fixes and small improvements. Changelog v0.2.5: + Sieve vacation extension: made discard message for implicit delivery more verbose - The sieve-test tool: mixed up original and final envelope recipient in implementation of command line arguments. - Sieve vacation extension: resolved FIXME regarding the use of variables in the :handle argument. Variables are now handled correctly. - Sieve body extension: fixed handling of :content "message/rfc822". This now yields the headers of the embedded message as required by the specification. Handling of :content "multipart" remains to be fixed. - LDA Sieve plugin: fixed problem with recipient_delimiter configuration. Now it falls back to global recipient_delimiter setting if plugin/recipient_delimiter is not set. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.5.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.5.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From metro_domain_admin at fastmail.fm Sat Nov 19 21:05:13 2011 From: metro_domain_admin at fastmail.fm (metro_domain_admin at fastmail.fm) Date: Sat, 19 Nov 2011 14:05:13 -0500 Subject: [Dovecot] issues with 2.1b1 on OS X Message-ID: <1321729513.18027.140661001080053@webmail.messagingengine.com> Two issues came up for me when building 2.1beta1 on OS X (10.5.8 on PPC). The first was already mentioned in an earlier thread by a BSD user (http://dovecot.org/list/dovecot/2011-September/061201.html). I just wanted to point out that the error is still occurring in spite of a new test for GNU ld in the configure script. I needed to remove "-no-undefined" specifically: #NOPLUGIN_LDFLAGS="-no-undefined" if test "$with_gnu_ld" = yes; then NOPLUGIN_LDFLAGS="$NOPLUGIN_LDFLAGS -Wl,--as-needed" fi More important, Dovecot aborts on launch with the following error: Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument I configured using the same args as I had used successfully on 2.0.15: --with-gssapi --with-ldap --with-sql --prefix=/opt/dovecot-2.X.X. I also tried an exact copy of my known-good config from 2.0.15, to no avail. Thanks, Tony From tss at iki.fi Sat Nov 19 22:34:36 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 19 Nov 2011 22:34:36 +0200 Subject: [Dovecot] issues with 2.1b1 on OS X In-Reply-To: <1321729513.18027.140661001080053@webmail.messagingengine.com> References: <1321729513.18027.140661001080053@webmail.messagingengine.com> Message-ID: On 19.11.2011, at 21.05, metro_domain_admin at fastmail.fm wrote: > Two issues came up for me when building 2.1beta1 on OS X (10.5.8 on > PPC). The first was already mentioned in an earlier thread by a BSD user > (http://dovecot.org/list/dovecot/2011-September/061201.html). I just > wanted to point out that the error is still occurring in spite of a new > test for GNU ld in the configure script. I needed to remove > "-no-undefined" specifically: This error? Undefined symbols: "_environ", referenced from: _env_clean in env-util.o http://hg.dovecot.org/dovecot-2.1/rev/b2c9298e981 and http://hg.dovecot.org/dovecot-2.1/rev/9d022d3fba42 fixes it. > More important, Dovecot aborts on launch with the following error: > > Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, > READ, 16) failed: Invalid argument > I configured using the same args as I had used successfully on 2.0.15: > --with-gssapi --with-ldap --with-sql --prefix=/opt/dovecot-2.X.X. I also > tried an exact copy of my known-good config from 2.0.15, to no avail. Works in 10.6. What is the gdb backtrace of the crash with the attached patch? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 577 bytes Desc: not available URL: From davide at cavion.eu Sun Nov 20 11:08:20 2011 From: davide at cavion.eu (Davide Cavion) Date: Sun, 20 Nov 2011 10:08:20 +0100 Subject: [Dovecot] Using google 2 step authentication Message-ID: Google authenticator has a PAM interface: http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions Have someone tried implementing it onto Dovecot? It would be really great if you consider your emails important! From dovecot.user at seibercom.net Sun Nov 20 18:09:22 2011 From: dovecot.user at seibercom.net (Jerry) Date: Sun, 20 Nov 2011 11:09:22 -0500 Subject: [Dovecot] Using google 2 step authentication In-Reply-To: References: Message-ID: <20111120110922.3488d46d@scorpio> On Sun, 20 Nov 2011 10:08:20 +0100 Davide Cavion articulated: > Google authenticator has a PAM interface: > > http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions > > Have someone tried implementing it onto Dovecot? It would be really > great if you consider your emails important! Isn't this an oxymoron; ie "Security" & "Google"? -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From davide at cavion.eu Sun Nov 20 18:18:01 2011 From: davide at cavion.eu (Davide Cavion) Date: Sun, 20 Nov 2011 17:18:01 +0100 Subject: [Dovecot] Using google 2 step authentication In-Reply-To: <20111120110922.3488d46d@scorpio> References: <20111120110922.3488d46d@scorpio> Message-ID: <924E1EE7-6487-443E-AE88-FBD3AF7AA62E@cavion.eu> Haha, you are right =) However this is open source and don't speak with google's servers ;) Il giorno 20/nov/2011, alle ore 17:09, Jerry ha scritto: > On Sun, 20 Nov 2011 10:08:20 +0100 > Davide Cavion articulated: > >> Google authenticator has a PAM interface: >> >> http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions >> >> Have someone tried implementing it onto Dovecot? It would be really >> great if you consider your emails important! > > Isn't this an oxymoron; ie "Security" & "Google"? > > -- > Jerry ? > Dovecot.user at seibercom.net > > Disclaimer: off-list followups get on-list replies or get ignored. > Please do not ignore the Reply-To header. > __________________________________________________________________ > From victormanuelo at gmail.com Mon Nov 21 02:05:27 2011 From: victormanuelo at gmail.com (Victor) Date: Sun, 20 Nov 2011 19:35:27 -0430 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 Message-ID: <9BD55AF0-B054-4AD3-AC94-E6490E1B495D@gmail.com> Thank a lot, your were completely in the truth Victor Onate From tss at iki.fi Mon Nov 21 04:06:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 04:06:18 +0200 Subject: [Dovecot] issues with 2.1b1 on OS X In-Reply-To: <1321729513.18027.140661001080053@webmail.messagingengine.com> References: <1321729513.18027.140661001080053@webmail.messagingengine.com> Message-ID: <1321841181.24607.23.camel@hurina> On Sat, 2011-11-19 at 14:05 -0500, metro_domain_admin at fastmail.fm wrote: > More important, Dovecot aborts on launch with the following error: > > Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, > READ, 16) failed: Invalid argument This probably helps: service stats { fifo_listener stats-mail { mode = 0 } } Since this fails only in OSX 10.5 (not 10.6), I don't know if I'll bother adding any special code to work around it. In any case it means that kevent() can't be used on a FIFO, and that means you can't use v2.1's stats tracking feature. Or possibly you could try if it works with configure --with-ioloop=poll or select instead of kqueue. From preen at informatik.uni-freiburg.de Mon Nov 21 11:18:02 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Mon, 21 Nov 2011 10:18:02 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4DACA6E3.5020301@bio.umass.edu> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> Message-ID: <4ECA174A.1020909@informatik.uni-freiburg.de> Hello, after serveral tests (and reading a lot of howto's) I'm now fairly convinced that the Solaris HW-crypto engine is not automatically used. Even when dovecot ist compiled with the OpenSSL version provided by Solaris. Currently I have only a T1-CPU available for testing (Sun Fire T2000) and after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with ENGINE *e; ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); ENGINE_set_default_ciphers(e); in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() the crypto device gets used. I'm sure that this is not the whole story since this only seems to affect the IMAP login. One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in the various articles). Is there any chance that Dovecot gets updated/patched for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. Regards. Martin Chris Hoogendyk wrote: > > On 4/18/11 6:30 AM, Martin Preen wrote: >> Hello, >> I tried to find out how about to use the hardware crypto engines under >> Solaris (Sun Fire T2000). It seems, that its not just a compilation >> issue: >> >>> For operations that are to be offloaded, it is necessary to restrict >>> use to subset >> > of OpenSSL functions (the EVP_ functions) and explicitly indicate >> the use of the PKCS11 >> > engine; something like the following works for bulk ciphers (the >> process for RSA is similar): >>> >>> ENGINE *e; >>> ENGINE_load_builtin_engines(); >>> e = ENGINE_by_id("pkcs11"); >>> ENGINE_set_default_ciphers(e); >>> EVP_CIPHER_CTX_init (&ctx); >>> EVP_EncryptInit (&ctx, EVP_des_cbc (), key, iv); >>> EVP_EncryptUpdate (.....); >> >> Since I'm not familiar with SSL programming, I wonder if it's possible to >> modify the dovecot 1.2.x source code. Maybe someone has already tried >> this >> or can give me hint. > > You don't want to do that. > > The objective is for the ssl engine to be low level and basically > invisible to layers of application above that use it. So, build openssl > properly and it just works for everything else. > > I'm not at work today, so I don't have reference to my notes. Ah, just > found an email I sent to my colleagues -- copied it below. --------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/~preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6337 bytes Desc: S/MIME Cryptographic Signature URL: From hummel at pasteur.fr Mon Nov 21 12:19:42 2011 From: hummel at pasteur.fr (Thomas Hummel) Date: Mon, 21 Nov 2011 11:19:42 +0100 Subject: [Dovecot] Dovecot Log analysis Message-ID: <20111121101941.GA25724@parmesan.sis.pasteur.fr> Hello, Before writing them myself, I'd like to figure out if some tools exist (anvil ? ...) and how to use them to extract from dovecot logs things like who is doing what and how hard or how fast in order to do some rate limiting. For instance, I'd like to be able to identify, slow or block users who . read or write to much / slice of time . login/logout too often / slice of time Note : I'm still using the 1.2.x branch. Thanks -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From pareshchalke at yahoo.com Mon Nov 21 08:52:44 2011 From: pareshchalke at yahoo.com (Paresh Chalke) Date: Mon, 21 Nov 2011 06:52:44 +0000 (UTC) Subject: [Dovecot] uid=error Message-ID: Hi Timo, We are facing this issue on our mail server 1. Two email id's are mapped to a same account 2. If I send a mail to both the account simultaneously i get below error "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" 3. Due to this the mail is delivered only to 1 email id. 4. I am loosing all my mails. Kindly help in the same ASAP. Regards, Paresh Chalke From wgrcunha at gmail.com Mon Nov 21 16:06:16 2011 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 21 Nov 2011 12:06:16 -0200 Subject: [Dovecot] Iterate over domain Message-ID: Hello, We need to use doveadm commands on all accounts of a single domain, for now i need to query the database using scripts and for each one user, run doveadm. I can use iterate but, only for all server accounts, have any way to use doveadm iterate over a single domain? Thanks From duihi77 at gmail.com Mon Nov 21 16:20:54 2011 From: duihi77 at gmail.com (Duane Hill) Date: Mon, 21 Nov 2011 14:20:54 +0000 Subject: [Dovecot] Iterate over domain In-Reply-To: References: Message-ID: <1178881680.20111121142054@gmail.com> On Monday, November 21, 2011 at 14:06:16 UTC, wgrcunha at gmail.com confabulated: > Hello, > We need to use doveadm commands on all accounts of a single domain, for now > i need to query the database using scripts and for each one user, run > doveadm. I can use iterate but, only for all server accounts, have any way > to use doveadm iterate over a single domain? Instead of using the -A option for all, use: -u '*@thedomain.tld' -- There are 10 kinds of people in the world... Those who understand binary, and those who don't. From jgiles at cybermesa.com Mon Nov 21 17:44:13 2011 From: jgiles at cybermesa.com (jgiles at cybermesa.com) Date: Mon, 21 Nov 2011 08:44:13 -0700 (MST) Subject: [Dovecot] Question about imaps Message-ID: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> Hi List! Had a questions about imap vs. imaps. I read the ssl article at http://wiki2.dovecot.org/SSL and from what I am reading, imaps is no longer needed as imap with initiate SSL/TLS on its own if STARTTLS is started and I have disable_plaintext_auth=yes and ssl=required settings. I do have SSL up and working, and can connect using imap (143) to my server without issue, but was not able to connect using imaps (993) using Kmail as my client. I Was getting a client error stating that the authentication failed using my password. However, there was nothing in the mail.err or mail.log file even when I enable debugging. If I specify imaps as my only protocol, then dovecot fails to even start without any error loging at all. As I stated, imap works just fine. So, is imaps deprecated as mentioned in the article and imaps no longer needed? I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. This server is simply an imap serer and not a MTA (No postfix or sendmail installed.) Thanks! Joe From jgiles at cybermesa.com Mon Nov 21 18:40:02 2011 From: jgiles at cybermesa.com (jgiles at cybermesa.com) Date: Mon, 21 Nov 2011 09:40:02 -0700 (MST) Subject: [Dovecot] Question about imaps In-Reply-To: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> Message-ID: <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > Hi List! > > Had a questions about imap vs. imaps. > > I read the ssl article at http://wiki2.dovecot.org/SSL and from what I am > reading, imaps is no longer needed as imap with initiate SSL/TLS on its > own if STARTTLS is started and I have disable_plaintext_auth=yes and > ssl=required settings. > > I do have SSL up and working, and can connect using imap (143) to my > server without issue, but was not able to connect using imaps (993) using > Kmail as my client. > > I Was getting a client error stating that the authentication failed using > my password. However, there was nothing in the mail.err or mail.log file > even when I enable debugging. > > If I specify imaps as my only protocol, then dovecot fails to even start > without any error loging at all. > > As I stated, imap works just fine. > > So, is imaps deprecated as mentioned in the article and imaps no longer > needed? > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > This server is simply an imap serer and not a MTA (No postfix or sendmail > installed.) > > Thanks! > Joe > > > > LOL, replying to my own e-mail... I just discovered that the problem is most likely in my Kmail 2 client. Tried to connect using the following: openssl s_client -connect mymailserver.com:993 I was able to connect, log in, and get information about my mail folders. Will try Thunderbird when I get home from work. Kmail 2 is still a WIP I guess. That said, I am still reading that imap with starttls is the way to go, so might be a moot issue. Thanks! Joe From tss at iki.fi Mon Nov 21 20:06:19 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 20:06:19 +0200 Subject: [Dovecot] uid=error In-Reply-To: References: Message-ID: On 21.11.2011, at 8.52, Paresh Chalke wrote: > We are facing this issue on our mail server > 1. Two email id's are mapped to a same account > 2. If I send a mail to both the account simultaneously i get below error > > "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, > uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. From dovecot.user at seibercom.net Mon Nov 21 20:20:55 2011 From: dovecot.user at seibercom.net (Jerry) Date: Mon, 21 Nov 2011 13:20:55 -0500 Subject: [Dovecot] Question about imaps In-Reply-To: <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> Message-ID: <20111121132055.15af04cf@scorpio> On Mon, 21 Nov 2011 09:40:02 -0700 (MST) jgiles at cybermesa.com articulated: > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > > Hi List! > > > > Had a questions about imap vs. imaps. > > > > I read the ssl article at http://wiki2.dovecot.org/SSL and from > > what I am reading, imaps is no longer needed as imap with initiate > > SSL/TLS on its own if STARTTLS is started and I have > > disable_plaintext_auth=yes and ssl=required settings. > > > > I do have SSL up and working, and can connect using imap (143) to my > > server without issue, but was not able to connect using imaps (993) > > using Kmail as my client. > > > > I Was getting a client error stating that the authentication failed > > using my password. However, there was nothing in the mail.err or > > mail.log file even when I enable debugging. > > > > If I specify imaps as my only protocol, then dovecot fails to even > > start without any error loging at all. > > > > As I stated, imap works just fine. > > > > So, is imaps deprecated as mentioned in the article and imaps no > > longer needed? > > > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > > This server is simply an imap serer and not a MTA (No postfix or > > sendmail installed.) > > > > Thanks! > > Joe > > LOL, replying to my own e-mail... > > I just discovered that the problem is most likely in my Kmail 2 > client. > > Tried to connect using the following: > > openssl s_client -connect mymailserver.com:993 > > I was able to connect, log in, and get information about my mail > folders. > > Will try Thunderbird when I get home from work. Kmail 2 is still a > WIP I guess. > > That said, I am still reading that imap with starttls is the way to > go, so might be a moot issue. > > Thanks! > Joe Try this: (substitute your actual host for "remote.host") openssl s_client -connect remote.host:143 -starttls imap See if you get a connection. If so, then the server offers STARTTLS. -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From andy.robbins at firespring.com Mon Nov 21 20:34:45 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Mon, 21 Nov 2011 12:34:45 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Actually, some of the information I provided previously was incorrect. We have four mail servers in our cluster which each have the NFS share mounted. These four servers handle all SMTP and sending of mail. However, it was thought that we could boost performance by simply running POP3 and IMAP directly from the NFS server, since nothing besides Dovecot is currently running there. So, here is the dovecot -n output from our IMAP/POP3 server where the file system is local storage and is where we believe the performance issues are originating. # dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 2304). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.32-24-server x86_64 Ubuntu 10.04.1 LTS protocols: imap pop3 ssl_cert_file: /etc/dovecot/smtp1.domain.com.cert ssl_key_file: /etc/dovecot/smtp1.domain.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Welcome login_process_per_connection: no login_processes_count: 2 max_mail_processes: 2048 mail_max_userip_connections: 50 mail_uid: 500 mail_gid: 500 mail_location: maildir:~/Maildir fsync_disable: yes maildir_very_dirty_syncs: yes mbox_write_locks: fcntl dotlock mail_executable(default): /etc/dovecot/last-login-imap.sh mail_executable(imap): /etc/dovecot/last-login-imap.sh mail_executable(pop3): /etc/dovecot/last-login-pop3.sh mail_plugins: quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format: %f auth default: mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP username_translation: %@ verbose: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf plugin: quota: maildir:User quota quota_rule: *:storage=0 So you can see that we have already tried a number of performance modifications such as setting fsync_disable: yes with little or no effect. If I run nfsstat on the email servers where the volumes are mounted and only running SMTP I get: # nfsstat Client rpc stats: calls retrans authrefrsh 51762532 72668 0 Client nfs v3: null getattr setattr lookup access readlink 0 0% 9639844 18% 3846547 7% 10495207 20% 11930633 23% 8 0% read write create mkdir symlink mknod 7011062 13% 5372278 10% 1593131 3% 95 0% 0 0% 0 0% remove rmdir rename link readdir readdirplus 830662 1% 0 0% 865484 1% 2576 0% 110845 0% 24963 0% fsstat fsinfo pathconf commit 2074 0% 8 0% 0 0% 37113 0% My apologies for the somewhat garbled output, there, but I it seems to me that we're not witnessing the same higher IO writes as compared to reads. Thanks for your help so far. Hopefully, this sheds more light on your previous questions. On Thu, Nov 17, 2011 at 4:42 PM, Timo Sirainen wrote: > Do you have more than one Dovecot server? If only one, you can still > disable the mail_nfs_* settings. Also it would be interesting to see > nfsstat numbers from the Dovecot server, compared to those iostat numbers.. > > On 18.11.2011, at 0.38, Andy Robbins wrote: > > > Well, the iostat command was run from the NFS server and dovecot was run > from the mail server where it is mounted, hence the discrepancy there. > > > > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > > > # iostat -d 5 -x > > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > avgrq-sz > > > avgqu-sz await svctm %util > > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > 8.00 28.69 19.63 0.70 94.00 > > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > 8.00 111.42 94.44 0.74 99.60 > > > > These devices don't look like NFS, but you have: > > > > > mmap_disable: yes > > > mail_nfs_storage: yes > > > mail_nfs_index: yes > > > > So what filesystem is that? Are they storing only mails, or other stuff > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > things up at least somewhat.. > > > > > lda: > > > fsync_disable: no > > > > This is explicitly set for LDA, but it's the default already elsewhere. > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > > > > > > From jgiles at cybermesa.com Mon Nov 21 21:17:57 2011 From: jgiles at cybermesa.com (Joe Giles) Date: Mon, 21 Nov 2011 12:17:57 -0700 Subject: [Dovecot] Question about imaps In-Reply-To: <20111121132055.15af04cf@scorpio> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> <20111121132055.15af04cf@scorpio> Message-ID: <21698362.YkgfMOa32X@linuxbox> On Monday, November 21, 2011 01:20:55 PM Jerry wrote: > On Mon, 21 Nov 2011 09:40:02 -0700 (MST) > jgiles at cybermesa.com articulated: > > > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > > > Hi List! > > > > > > Had a questions about imap vs. imaps. > > > > > > I read the ssl article at http://wiki2.dovecot.org/SSL and from > > > what I am reading, imaps is no longer needed as imap with initiate > > > SSL/TLS on its own if STARTTLS is started and I have > > > disable_plaintext_auth=yes and ssl=required settings. > > > > > > I do have SSL up and working, and can connect using imap (143) to my > > > server without issue, but was not able to connect using imaps (993) > > > using Kmail as my client. > > > > > > I Was getting a client error stating that the authentication failed > > > using my password. However, there was nothing in the mail.err or > > > mail.log file even when I enable debugging. > > > > > > If I specify imaps as my only protocol, then dovecot fails to even > > > start without any error loging at all. > > > > > > As I stated, imap works just fine. > > > > > > So, is imaps deprecated as mentioned in the article and imaps no > > > longer needed? > > > > > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > > > This server is simply an imap serer and not a MTA (No postfix or > > > sendmail installed.) > > > > > > Thanks! > > > Joe > > > > LOL, replying to my own e-mail... > > > > I just discovered that the problem is most likely in my Kmail 2 > > client. > > > > Tried to connect using the following: > > > > openssl s_client -connect mymailserver.com:993 > > > > I was able to connect, log in, and get information about my mail > > folders. > > > > Will try Thunderbird when I get home from work. Kmail 2 is still a > > WIP I guess. > > > > That said, I am still reading that imap with starttls is the way to > > go, so might be a moot issue. > > > > Thanks! > > Joe > > Try this: (substitute your actual host for "remote.host") > > openssl s_client -connect remote.host:143 -starttls imap > > See if you get a connection. If so, then the server offers STARTTLS. > > Hi, Jerry, Thanks for the information. Yes, I ran that command and got my cert and other information proving to me that TLS is working. I was also able to log in and get mailbox information. Thanks again! Joe From victormanuelo at gmail.com Mon Nov 21 21:42:56 2011 From: victormanuelo at gmail.com (=?UTF-8?Q?Victor_O=C3=B1ate?=) Date: Mon, 21 Nov 2011 15:12:56 -0430 Subject: [Dovecot] Question about imaps In-Reply-To: <21698362.YkgfMOa32X@linuxbox> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> <20111121132055.15af04cf@scorpio> <21698362.YkgfMOa32X@linuxbox> Message-ID: Try to conect from shell # telnet server 993 2011/11/21 Joe Giles : > On Monday, November 21, 2011 01:20:55 PM Jerry wrote: >> On Mon, 21 Nov 2011 09:40:02 -0700 (MST) >> jgiles at cybermesa.com articulated: >> >> > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: >> > > Hi List! >> > > >> > > Had a questions about imap vs. imaps. >> > > >> > > I read the ssl article at http://wiki2.dovecot.org/SSL and from >> > > what I am reading, imaps is no longer needed as imap with initiate >> > > SSL/TLS on its own if STARTTLS is started and I have >> > > disable_plaintext_auth=yes and ssl=required settings. >> > > >> > > I do have SSL up and working, and can connect using imap (143) to my >> > > server without issue, but was not able to connect using imaps (993) >> > > using Kmail as my client. >> > > >> > > I Was getting a client error stating that the authentication failed >> > > using my password. However, there was nothing in the mail.err or >> > > mail.log file even when I enable debugging. >> > > >> > > If I specify imaps as my only protocol, then dovecot fails to even >> > > start without any error loging at all. >> > > >> > > As I stated, imap works just fine. >> > > >> > > So, is imaps deprecated as mentioned in the article and imaps no >> > > longer needed? >> > > >> > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. >> > > This server is simply an imap serer and not a MTA (No postfix or >> > > sendmail installed.) >> > > >> > > Thanks! >> > > Joe >> > >> > LOL, replying to my own e-mail... >> > >> > I just discovered that the problem is most likely in my Kmail 2 >> > client. >> > >> > Tried to connect using the following: >> > >> > openssl s_client -connect mymailserver.com:993 >> > >> > I was able to connect, log in, and get information about my mail >> > folders. >> > >> > Will try Thunderbird when I get home from work. Kmail 2 is still a >> > WIP I guess. >> > >> > That said, I am still reading that imap with starttls is the way to >> > go, so might be a moot issue. >> > >> > Thanks! >> > Joe >> >> Try this: (substitute your actual host for "remote.host") >> >> openssl s_client -connect remote.host:143 -starttls imap >> >> See if you get a connection. If so, then the server offers STARTTLS. >> >> > > Hi, Jerry, > > Thanks for the information. > > Yes, I ran that command and got my cert and other information proving to me that TLS is working. I was also able to log in and get mailbox information. > > Thanks again! > Joe > From stan at hardwarefreak.com Mon Nov 21 21:45:54 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 21 Nov 2011 13:45:54 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: <4ECAAA72.4040802@hardwarefreak.com> On 11/21/2011 12:34 PM, Andy Robbins wrote: > Warning: fd limit 1024 is lower than what Dovecot can use under full load > (more than 2304). Either grow the limit or change login_max_processes_count > and max_mail_processes settings Why haven't you increased the file descriptor limit? -- Stan From achekalin at lazurit.com Mon Nov 21 22:02:39 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 21 Nov 2011 23:02:39 +0300 Subject: [Dovecot] Unique message IDs? Message-ID: <4ECAAE5F.1040507@lazurit.com> Hello, I'm working with indexing my mail box. What I need is to have index in which mailing specified header string is located. I work with the messages via IMAP. I know that sounds like FTS can help me but no, I don't want to index whole message nor I want to patch FTS source to make it to index headers only. I need a way to identify message across my mail box. The idea is that I can move messages between IMAP folders and the index still should be able to identify it. What I want to know, will UID strings be unique across all my messages no matter which IMAP folder it belongs now to, or the UID can change if I move message? If I remove dovecot-uidlist from the folder will UIDs of messages in the folder be changed? What happened if I put a message to the given IMAP folder (and dovecot-uidlist will record UID for it), then shut down dovecot, and (given maildir storage model) move the message's file info another folder in the same mail box - after I start dovecot, will the UID for the message be the same or not? Thank you in advance, Alexander Chekalin From robert at schetterer.org Mon Nov 21 22:14:53 2011 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 21 Nov 2011 21:14:53 +0100 Subject: [Dovecot] uid=error In-Reply-To: References: Message-ID: <4ECAB13D.3050501@schetterer.org> Am 21.11.2011 19:06, schrieb Timo Sirainen: > On 21.11.2011, at 8.52, Paresh Chalke wrote: > >> We are facing this issue on our mail server >> 1. Two email id's are mapped to a same account >> 2. If I send a mail to both the account simultaneously i get below error >> >> "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, >> uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" > > This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. > Hi Timo, just for info i also have sporadic error like this Nov 21 20:36:10 mail01 dovecot: lmtp(10398, user at user.de): save: box=INBOX, uid=error, msgid=<20111121193610.69B4D200A2F at server.de>, size=6356 as mails get delivered i dont care, i see this happen sometimes during rsync backuptimes but i have not really an idea what exact cause this -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Mon Nov 21 22:22:58 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 22:22:58 +0200 Subject: [Dovecot] uid=error In-Reply-To: <4ECAB13D.3050501@schetterer.org> References: <4ECAB13D.3050501@schetterer.org> Message-ID: On 21.11.2011, at 22.14, Robert Schetterer wrote: >> This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. >> > Hi Timo, > just for info > i also have sporadic error like this > Nov 21 20:36:10 mail01 dovecot: lmtp(10398, user at user.de): save: > box=INBOX, uid=error, msgid=<20111121193610.69B4D200A2F at server.de>, > size=6356 > > as mails get delivered i dont care, i see > this happen sometimes during rsync backuptimes > but i have not really an idea what exact cause this Oh, I was wrong. The uid=error simply means that the UID is unknown for some reason. Possibly an internal bug.. The mail got saved just fine. Some backends like saving to a virtual mailbox will also give uid=error. So anyway, back to the original question: This log message doesn't indicate any error. The problem of some mail not being saved is somewhere else, possibly in MTA configuration? From tss at iki.fi Mon Nov 21 22:27:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 22:27:59 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECAAE5F.1040507@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> Message-ID: On 21.11.2011, at 22.02, Alexander Chekalin wrote: > I need a way to identify message across my mail box. The idea is that I can move messages between IMAP folders and the index still should be able to identify it. Message GUIDs are pretty good for that. > What I want to know, will UID strings be unique across all my messages no matter which IMAP folder it belongs now to, or the UID can change if I move message? If I remove dovecot-uidlist from the folder will UIDs of messages in the folder be changed? What happened if I put a message to the given IMAP folder (and dovecot-uidlist will record UID for it), then shut down dovecot, and (given maildir storage model) move the message's file info another folder in the same mail box - after I start dovecot, will the UID for the message be the same or not? With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). Assuming you're using Dovecot v2.x, when mail is copied to another mailbox its filename is preserved. So deleting dovecot* files won't lose the GUID. The only problem is that if you copy the same mail twice to another mailbox, it can't of course have the same filename twice, so Dovecot will assign it a new filename. But in a new enough version (probably v2.0.something) it still preserves the GUID by writing it to dovecot-uidlist file. In this situation if you delete the uidlist, the GUID changes to its filename. From tss at iki.fi Mon Nov 21 23:41:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:41:57 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: <1321911718.24607.51.camel@hurina> I don't see anything obviously bad in your setup. And since you said you're running Dovecot in the IMAP server, perhaps it's not even Dovecot at all that is causing the writes?.. Here are anyway some thoughts: * Log files? Make sure syslog isn't fsyncing each Dovecot log line to disk (or just set log_path = /var/log/dovecot.log and it definitely won't fsync) * Try if disabling index files lowers writes (I don't see why it would make a big difference): mail_location = maildir:~/Maildir:INDEX=MEMORY * Try if temporarily disabling Maildir++ quota makes any difference. There aren't really any other potential writers in Dovecot.. On Mon, 2011-11-21 at 12:34 -0600, Andy Robbins wrote: > Actually, some of the information I provided previously was incorrect. We > have four mail servers in our cluster which each have the NFS share > mounted. These four servers handle all SMTP and sending of mail. However, > it was thought that we could boost performance by simply running POP3 and > IMAP directly from the NFS server, since nothing besides Dovecot is > currently running there. So, here is the dovecot -n output from our > IMAP/POP3 server where the file system is local storage and is where we > believe the performance issues are originating. > > # dovecot -n > # 1.2.9: /etc/dovecot/dovecot.conf > Warning: fd limit 1024 is lower than what Dovecot can use under full load > (more than 2304). Either grow the limit or change login_max_processes_count > and max_mail_processes settings > # OS: Linux 2.6.32-24-server x86_64 Ubuntu 10.04.1 LTS > protocols: imap pop3 > ssl_cert_file: /etc/dovecot/smtp1.domain.com.cert > ssl_key_file: /etc/dovecot/smtp1.domain.com.key > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > login_greeting: Welcome > login_process_per_connection: no > login_processes_count: 2 > max_mail_processes: 2048 > mail_max_userip_connections: 50 > mail_uid: 500 > mail_gid: 500 > mail_location: maildir:~/Maildir > fsync_disable: yes > maildir_very_dirty_syncs: yes > mbox_write_locks: fcntl dotlock > mail_executable(default): /etc/dovecot/last-login-imap.sh > mail_executable(imap): /etc/dovecot/last-login-imap.sh > mail_executable(pop3): /etc/dovecot/last-login-pop3.sh > mail_plugins: quota > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > pop3_uidl_format: %f > auth default: > mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP > username_translation: %@ > verbose: yes > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > plugin: > quota: maildir:User quota > quota_rule: *:storage=0 > > So you can see that we have already tried a number of performance > modifications such as setting fsync_disable: yes with little or no effect. > If I run nfsstat on the email servers where the volumes are mounted and > only running SMTP I get: > > # nfsstat > Client rpc stats: > calls retrans authrefrsh > 51762532 72668 0 > > Client nfs v3: > null getattr setattr lookup access > readlink > 0 0% 9639844 18% 3846547 7% 10495207 20% 11930633 23% 8 > 0% > read write create mkdir symlink > mknod > 7011062 13% 5372278 10% 1593131 3% 95 0% 0 0% 0 > 0% > remove rmdir rename link readdir > readdirplus > 830662 1% 0 0% 865484 1% 2576 0% 110845 0% 24963 > 0% > fsstat fsinfo pathconf commit > 2074 0% 8 0% 0 0% 37113 0% > > My apologies for the somewhat garbled output, there, but I it seems to me > that we're not witnessing the same higher IO writes as compared to reads. > > Thanks for your help so far. Hopefully, this sheds more light on your > previous questions. > > > > On Thu, Nov 17, 2011 at 4:42 PM, Timo Sirainen wrote: > > > Do you have more than one Dovecot server? If only one, you can still > > disable the mail_nfs_* settings. Also it would be interesting to see > > nfsstat numbers from the Dovecot server, compared to those iostat numbers.. > > > > On 18.11.2011, at 0.38, Andy Robbins wrote: > > > > > Well, the iostat command was run from the NFS server and dovecot was run > > from the mail server where it is mounted, hence the discrepancy there. > > > > > > > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > > > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > > > > > # iostat -d 5 -x > > > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > > avgrq-sz > > > > avgqu-sz await svctm %util > > > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > > 8.00 28.69 19.63 0.70 94.00 > > > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > > 8.00 111.42 94.44 0.74 99.60 > > > > > > These devices don't look like NFS, but you have: > > > > > > > mmap_disable: yes > > > > mail_nfs_storage: yes > > > > mail_nfs_index: yes > > > > > > So what filesystem is that? Are they storing only mails, or other stuff > > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > > things up at least somewhat.. > > > > > > > lda: > > > > fsync_disable: no > > > > > > This is explicitly set for LDA, but it's the default already elsewhere. > > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > > > > > > > > > > > From tss at iki.fi Mon Nov 21 23:45:39 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:45:39 +0200 Subject: [Dovecot] Dovecot Log analysis In-Reply-To: <20111121101941.GA25724@parmesan.sis.pasteur.fr> References: <20111121101941.GA25724@parmesan.sis.pasteur.fr> Message-ID: <1321911939.24607.55.camel@hurina> On Mon, 2011-11-21 at 11:19 +0100, Thomas Hummel wrote: > Hello, > > Before writing them myself, I'd like to figure out if some tools exist (anvil ? > ...) and how to use them to extract from dovecot logs things like who is doing > what and how hard or how fast in order to do some rate limiting. For instance, > I'd like to be able to identify, slow or block users who > > . read or write to much / slice of time > > . login/logout too often / slice of time > > Note : I'm still using the 1.2.x branch. The reading/writing rate limiting could be a bit tricky. I guess with Linux you could look into /proc//io files for all imap/pop3 processes and determine from there which processes are doing too much. For pop3 there's also http://dovecot.org/patches/1.2/pop3-throttle-plugin.c Other than that, I don't know of any existing stuff for v1.2. For v2.1 there's the new "stats" service and you can ask all kinds of statistics from it, including what users/processes have used the most disk IO. From pw at wk-serv.de Mon Nov 21 23:45:49 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 21 Nov 2011 22:45:49 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <1321472837.21919.598.camel@hurina> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> Message-ID: <4ECAC68D.3060007@wk-serv.de> Timo Sirainen schrieb: > On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: >> I already use lmtp:unix:private/dovecot-lmtp as transport but where is >> the link to the indexes? > > You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot > LMTP proxy, which would forward the connection to the backend server > which handles that user's IMAP/POP3/LMTP connections. I don't know if we're talking about the same :) From achekalin at lazurit.com Mon Nov 21 23:47:15 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 00:47:15 +0300 Subject: [Dovecot] Unique message IDs? In-Reply-To: References: <4ECAAE5F.1040507@lazurit.com> Message-ID: <4ECAC6E3.9090307@lazurit.com> > Message GUIDs are pretty good for that. Oh, thank you! Nice news! > With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). But what if I one day decide to convert my maildir's to mbox'es? I really plan to do such conversion in a while (as soon as I finish the indexing system). Yours, Alexander From tss at iki.fi Mon Nov 21 23:52:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:52:28 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4ECA174A.1020909@informatik.uni-freiburg.de> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> Message-ID: <1321912348.24607.58.camel@hurina> On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: > Hello, > after serveral tests (and reading a lot of howto's) I'm now fairly convinced > that the Solaris HW-crypto engine is not automatically used. Even when dovecot > ist compiled with the OpenSSL version provided by Solaris. > > Currently I have only a T1-CPU available for testing (Sun Fire T2000) and > after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with > > ENGINE *e; > ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); Does "openssl engine" return this "pkcs11" string? > ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); > ENGINE_set_default_ciphers(e); > > in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() > the crypto device gets used. I'm sure that this is not the whole story since > this only seems to affect the IMAP login. It should work for POP3 as well, all of the SSL code is shared. > One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in > the various articles). Is there any chance that Dovecot gets updated/patched > for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. I guess I could add ssl_crypto_device setting for this. But I'll need to figure out proper ifdefs to avoid compile failures with older OpenSSL versions. From tss at iki.fi Mon Nov 21 23:56:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:56:46 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECAC6E3.9090307@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> Message-ID: <1321912606.24607.61.camel@hurina> On Tue, 2011-11-22 at 00:47 +0300, Alexander Chekalin wrote: > > With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). > > But what if I one day decide to convert my maildir's to mbox'es? > I really plan to do such conversion in a while (as soon as I finish the > indexing system). mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of fakes it by returning MD5 of specific headers as GUIDs (so not 100% reliable). With mdbox GUIDs work even better than with Maildir, the GUID is always stored in the message's metadata. From hoogendyk at bio.umass.edu Tue Nov 22 00:26:58 2011 From: hoogendyk at bio.umass.edu (Chris Hoogendyk) Date: Mon, 21 Nov 2011 17:26:58 -0500 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1321912348.24607.58.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> Message-ID: <4ECAD032.6060902@bio.umass.edu> On 11/21/11 4:52 PM, Timo Sirainen wrote: > On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: >> Hello, >> after serveral tests (and reading a lot of howto's) I'm now fairly convinced >> that the Solaris HW-crypto engine is not automatically used. Even when dovecot >> ist compiled with the OpenSSL version provided by Solaris. >> >> Currently I have only a T1-CPU available for testing (Sun Fire T2000) and >> after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with >> >> ENGINE *e; >> ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); > Does "openssl engine" return this "pkcs11" string? > >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >> ENGINE_set_default_ciphers(e); >> >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >> the crypto device gets used. I'm sure that this is not the whole story since >> this only seems to affect the IMAP login. > It should work for POP3 as well, all of the SSL code is shared. > >> One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in >> the various articles). Is there any chance that Dovecot gets updated/patched >> for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. > I guess I could add ssl_crypto_device setting for this. But I'll need to > figure out proper ifdefs to avoid compile failures with older OpenSSL > versions. Just a quick comment -- the Sun instructions for setting up Apache specifically noted the importance of specifying "SSLCryptoDevice pkcs11" in httpd.conf. So, the OpenSSL provided by Solaris will do what it should, IFF pkcs11 is specified by the program that is calling it. I had an email exchange with a Solaris developer at some point, who said that GnuPG simply could not use the crypto device, because it was built with libcrypt, did not use the pkcs11 engine, and it would take a significant rewrite to make it do so. -- --------------- Chris Hoogendyk - O__ ---- Systems Administrator c/ /'_ --- Biology& Geology Departments (*) \(*) -- 140 Morrill Science Center ~~~~~~~~~~ - University of Massachusetts, Amherst --------------- Erd?s 4 From preen at informatik.uni-freiburg.de Tue Nov 22 00:49:28 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Mon, 21 Nov 2011 23:49:28 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1321912348.24607.58.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> Message-ID: <4ECAD578.4060304@informatik.uni-freiburg.de> Timo Sirainen wrote: > On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: >> Hello, >> after serveral tests (and reading a lot of howto's) I'm now fairly convinced >> that the Solaris HW-crypto engine is not automatically used. Even when dovecot >> ist compiled with the OpenSSL version provided by Solaris. >> >> Currently I have only a T1-CPU available for testing (Sun Fire T2000) and >> after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with >> >> ENGINE *e; >> ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); > > Does "openssl engine" return this "pkcs11" string? This requires the Solaris OpenSSL version (or another version using the pkcs11 patch). # /usr/sfw/bin/openssl engine (pkcs11) PKCS #11 engine support >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >> ENGINE_set_default_ciphers(e); >> >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >> the crypto device gets used. I'm sure that this is not the whole story since >> this only seems to affect the IMAP login. > > It should work for POP3 as well, all of the SSL code is shared. I couldn't find the EncryptUpdate call which has to be changed too (due to the howto documents). Maybe some other call needs e patch. But I don't know which. Martin >> One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in >> the various articles). Is there any chance that Dovecot gets updated/patched >> for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. > > I guess I could add ssl_crypto_device setting for this. But I'll need to > figure out proper ifdefs to avoid compile failures with older OpenSSL > versions. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6350 bytes Desc: S/MIME Cryptographic Signature URL: From Damon_Atkins at yahoo.com.au Tue Nov 22 04:17:39 2011 From: Damon_Atkins at yahoo.com.au (Damon Atkins) Date: Tue, 22 Nov 2011 13:17:39 +1100 Subject: [Dovecot] Solaris hardware crypto engines Message-ID: <4ECB0643.7070308@yahoo.com.au> Here are some blogs on the topic. http://wikis.sun.com/display/CryptoPerf/Using+the+UltraSPARC+cryptographic+accelerators Solaris 10 # /usr/sfw/bin/openssl engine -c -t # cc -fast*-I /usr/sfw/include -L /usr/sfw/lib -lcrypto* aes_test.c -o aes_test.out http://blogs.oracle.com/DanX/entry/sparc_t4_openssl_engine http://blogs.oracle.com/DanX/entry/where_s_the_crypto_libraries http://blogs.oracle.com/DanX/entry/solaris_x86_aesni_openssl_engine http://blogs.oracle.com/chichang1/entry/rsa_performance_of_sun_fire Here is some info from my intel box Solaris 11 # /usr/bin/openssl engine -c -t (aesni) Intel AES-NI engine (no-aesni) % no-aesni means no aes H/W acceleration [ available ] (dynamic) Dynamic engine loading support [ unavailable ] (pkcs11) PKCS #11 engine support [RSA, DSA, DH, RAND, DSA] [ available ] $ isainfo -v # My cpu does not have 'aes' support 64-bit amd64 applications cx16 sse3 sse2 sse fxsr mmx cmov amd_sysc cx8 tsc fpu 32-bit i386 applications ahf cx16 sse3 sse2 sse fxsr mmx cmov sep cx8 tsc fpu # ldd /opt/dovecot/libexec/dovecot/ssl-build-param **** libssl.so.1.0.0 => /lib/libssl.so.1.0.0 *** *** libcrypto.so.1.0.0 => /lib/libcrypto.so.1.0.0* *** libc.so.1 => /lib/libc.so.1 libnsl.so.1 => /lib/libnsl.so.1 libsocket.so.1 => /lib/libsocket.so.1 librt.so.1 => /lib/librt.so.1 libsendfile.so.1 => /lib/libsendfile.so.1 libmp.so.2 => /lib/libmp.so.2 libmd.so.1 => /lib/libmd.so.1 libm.so.2 => /lib/libm.so.2 ./configure --prefix=/opt/dovecot --with-ldap=yes --with-gssapi --with-ssldir=/etc/openssl Install prefix . : /opt/dovecot File offsets ... : 64bit I/O polling .... : poll I/O notifys .... : none SSL ............ : yes (OpenSSL) GSSAPI ......... : yes passdbs ........ : passwd passwd-file shadow pam checkpassword ldap : -bsdauth -sia -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword ldap : -sql -vpopmail -nss SQL drivers .... : : -pgsql -mysql -sqlite Note Under OpenSolaris I did the following: CPPFLAGS=-I/usr/sfw/include LDFLAGS=-R/usr/sfw/lib ./configure --prefix=/opt/dovecot --with-ldap=yes --with-gssapi --with-ssldir=/etc/openssl (most likely Solaris 10 is the same as above, openssl may look old but I believe it is patched with compatible *fixes* from current openssl so application do not break. Apparently it took 5mths to update Solaris 11 to OpenSSL 1.0 and test everything) Cheers Damon. From achekalin at lazurit.com Tue Nov 22 07:39:30 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 08:39:30 +0300 Subject: [Dovecot] Unique message IDs? In-Reply-To: <1321912606.24607.61.camel@hurina> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> Message-ID: <4ECB3592.1010500@lazurit.com> >>> With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). >> >> But what if I one day decide to convert my maildir's to mbox'es? >> I really plan to do such conversion in a while (as soon as I finish the >> indexing system). > > mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of > fakes it by returning MD5 of specific headers as GUIDs (so not 100% > reliable). With mdbox GUIDs work even better than with Maildir, the GUID > is always stored in the message's metadata. I'd like to use the best optimized one (mdbox), but there is a reason not to do that is when I use mbox or maildir I can see where given IMAP folder mails are stored, so, say, if I want to copy only one IMAP folder to some remote site, I just copy know dir or file. With mdbox this is different, I simply can not guess where my messages exactly are. If it be possible to have per-IMAP-folders mdboxes, I love to use it. But from what I know and tried this is not way mdbox used to work, right? Yours, Alexander Chekalin From janfrode at tanso.net Tue Nov 22 09:16:24 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 22 Nov 2011 08:16:24 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4ECAC68D.3060007@wk-serv.de> References: <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> Message-ID: <20111122071624.GA6110@dibs.tanso.net> On Mon, Nov 21, 2011 at 10:45:49PM +0100, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: > >>I already use lmtp:unix:private/dovecot-lmtp as transport but where is > >>the link to the indexes? > > > >You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot > >LMTP proxy, which would forward the connection to the backend server > >which handles that user's IMAP/POP3/LMTP connections. > > I don't know if we're talking about the same :) > I wondered that too. It looked to me like you tried to ask where the lmtp-service picks up the path to indexes, right? AFAIU it picks that up from the /var/run/dovecot/auth-master socket. -jf From achekalin at lazurit.com Tue Nov 22 11:10:12 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 12:10:12 +0300 Subject: [Dovecot] No duplicates on imap copy? Message-ID: <4ECB66F4.7060204@lazurit.com> Hello, when from my script I try to copy messages from one imap folder into another own (within the same imap mailbox). That works perfectly, but... If I copy the same message N times from one folder into another one, in the destination folder I see N copies of the same message, each with its own UID. I wonder if there any way to deduplicate folder content (so in the destination folder I'll get only one copy of each message)? I'd implement something like that myself in my script but I just don't know which part of the messages should I compare - probably md5 of headers? Thank you in advance! Yours, Alexander Chekalin From tss at iki.fi Tue Nov 22 11:58:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 11:58:46 +0200 Subject: [Dovecot] No duplicates on imap copy? In-Reply-To: <4ECB66F4.7060204@lazurit.com> References: <4ECB66F4.7060204@lazurit.com> Message-ID: <74E4AA7F-089C-4FF4-BE74-F80E8D1848D3@iki.fi> On 22.11.2011, at 11.10, Alexander Chekalin wrote: > when from my script I try to copy messages from one imap folder into another own (within the same imap mailbox). That works perfectly, but... If I copy the same message N times from one folder into another one, in the destination folder I see N copies of the same message, each with its own UID. Right. > I wonder if there any way to deduplicate folder content (so in the destination folder I'll get only one copy of each message)? Not any easy way currently. > I'd implement something like that myself in my script but I just don't know which part of the messages should I compare - probably md5 of headers? What Dovecot version and what mailbox format? From tss at iki.fi Tue Nov 22 12:01:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 12:01:28 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECB3592.1010500@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> <4ECB3592.1010500@lazurit.com> Message-ID: <275436D2-0816-47E2-877C-FC55AC43B67D@iki.fi> On 22.11.2011, at 7.39, Alexander Chekalin wrote: > I'd like to use the best optimized one (mdbox), but there is a reason not to do that is when I use mbox or maildir I can see where given IMAP folder mails are stored, so, say, if I want to copy only one IMAP folder to some remote site, I just copy know dir or file. With mdbox this is different, I simply can not guess where my messages exactly are. > > If it be possible to have per-IMAP-folders mdboxes, I love to use it. But from what I know and tried this is not way mdbox used to work, right? sdbox would work like that. The reason mdbox doesn't work like that is because copying messages would be rather slow then. The idea with mdbox is anyway that you'd use Dovecot's tools to manage the mailboxes rather than access them directly through filesystem. So if you want to copy one IMAP folder, you'd use either dsync or doveadm import to do it. From pw at wk-serv.de Tue Nov 22 12:17:12 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 22 Nov 2011 11:17:12 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <20111122071624.GA6110@dibs.tanso.net> References: <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> Message-ID: <4ECB76A8.80208@wk-serv.de> Jan-Frode Myklebust schrieb: > I wondered that too. It looked to me like you tried to ask where the > lmtp-service picks up the path to indexes, right? AFAIU it picks that up > from the /var/run/dovecot/auth-master socket. No. I want to know if dovecot writes to the indexes if it receives a mail via lmtp. Someone proposed to store the index files on a locally installed SSD on a frontend (imap) machine and stick the users to that machine but if the lmtp-service writes to the indexes (and I think he does), that machine needs access to the indexes too which will bring us back to shared storage. From janfrode at tanso.net Tue Nov 22 12:45:47 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 22 Nov 2011 11:45:47 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4ECB76A8.80208@wk-serv.de> References: <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> Message-ID: <20111122104547.GA10680@dibs.tanso.net> On Tue, Nov 22, 2011 at 11:17:12AM +0100, Patrick Westenberg wrote: > > No. I want to know if dovecot writes to the indexes if it receives a > mail via lmtp. > > Someone proposed to store the index files on a locally installed SSD > on a frontend (imap) machine and stick the users to that machine but > if the lmtp-service writes to the indexes (and I think he does), > that machine needs access to the indexes too which will bring us > back to shared storage. Ah, then Timo's reply was right. He suggested you do the lmtp-deliveries to the same server that you would send you imap-user to. You can do this trough dovecot director and lmtp-proxying. So instead of: lmtp:unix:private/dovecot-lmtp you should use: lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be the Dovecot LMTP proxy that proxies to the same machine as you would use for imap for this particular recipient. -jf From stan at hardwarefreak.com Tue Nov 22 19:00:08 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 22 Nov 2011 11:00:08 -0600 Subject: [Dovecot] 1.2.15 can't get zlib working Message-ID: <4ECBD518.9020206@hardwarefreak.com> Following these instructions: http://wiki.dovecot.org/Plugins/Zlib I enabled zlib and restarted Dovecot. I made a backup copy of a 68MB mbox file and moved it to a non mail directory. The mbox file is an existing TBird IMAP folder in my UNIX user mail directory. It is an archive of a defunct mailing list. It was a regular IMAP folder prior to attempting this. I was able to access all emails in the folder. It worked fine. I gzipped the original mbox file down to 14MB and removed the write flag with chmod. I launched TBird and received the following error in Activity Manager when accessing this IMAP folder: The current operation on '1-Spam-l' did not succeed. The mail server for account stan at hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l I deleted the Dovecot index files and the TBird 1-Spam-l.msf file which didn't help. Curiously, there are no entries in either dovecot.err or dovecot.log, giving not even the slightest hint as to where I should troubleshoot. Kinda sucks when one follows instructions exactly, simple instructions at that, and it doesn't work, and generates zero log errors. A bit frustrating... Any ideas? /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_plugins: fts fts_squat zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster at hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10 -- Stan From mrvjtod at gmail.com Tue Nov 22 20:24:15 2011 From: mrvjtod at gmail.com (Chris Young) Date: Tue, 22 Nov 2011 13:24:15 -0500 Subject: [Dovecot] disable imap info logging Message-ID: Is there a way to disable the imap info logging? My logs are filling with tons of login/logout messages Nov 22 18:23:25 imap-login: Info: Login: user=, method={method}, rip=10.86.189.79, lip=10.86.189.82, mpid=22488 Nov 22 18:23:25 imap(USER1): Info: Disconnected: Logged out bytes=54/726 From tss at iki.fi Tue Nov 22 20:28:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 20:28:56 +0200 Subject: [Dovecot] disable imap info logging In-Reply-To: References: Message-ID: <4A94BBF7-5704-4542-9796-942B50940367@iki.fi> On 22.11.2011, at 20.24, Chris Young wrote: > Is there a way to disable the imap info logging? > > My logs are filling with tons of login/logout messages > > Nov 22 18:23:25 imap-login: Info: Login: user=, method={method}, > rip=10.86.189.79, lip=10.86.189.82, mpid=22488 > Nov 22 18:23:25 imap(USER1): Info: Disconnected: Logged out bytes=54/726 Rotate the logs and have it delete old ones? Or simply: info_log_path = /dev/null From tss at iki.fi Tue Nov 22 20:30:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 20:30:33 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECBD518.9020206@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> Message-ID: On 22.11.2011, at 19.00, Stan Hoeppner wrote: > The current operation on '1-Spam-l' did not succeed. The mail server > for account stan at hardwarefreak.com responded: Mailbox doesn't exist: > 1-Spam-l .. > /$ la /home/stan/mail/1-Spam-l.gz > -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) From Juergen.Obermann at hrz.uni-giessen.de Tue Nov 22 22:11:02 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Tue, 22 Nov 2011 21:11:02 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC Message-ID: Hello, compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC with Sun Studio 11 stops with the following error: Making all in lib-imap-client gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-imap-client' source='imapc-client.c' object='imapc-client.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-client.lo imapc-client.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-client.c -KPIC -DPIC -o .libs/imapc-client.o "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-client.c -o imapc-client.o >/dev/null 2>&1 source='imapc-connection.c' object='imapc-connection.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-connection.lo imapc-connection.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-connection.c -KPIC -DPIC -o .libs/imapc-connection.o "../../src/lib-imap/imap-util.h", line 6: warning: useless declaration "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) "imapc-connection.c", line 1446: operand cannot have void type: op "==" "imapc-connection.c", line 1446: operands have incompatible types: const void "==" pointer to struct imapc_command {pointer to struct pool {..} pool, pointer to struct buffer {..} data, unsigned int send_pos, unsigned int tag, enum imapc_command_flags {IMAPC_COMMAND_FLAG_RETRIABLE(4), IMAPC_COMMAND_FLAG_PRELOGIN(2), IMAPC_COMMAND_FLAG_SELECT(1)} flags, pointer to struct imapc_connection {..} conn, pointer to struct imapc_client_mailbox {..} box, union {..} streams, pointer to function(pointer to const struct imapc_command_reply {..}, pointer to void) returning void callback, pointer to void context, unsigned int idle :1, unsigned int wait_for_literal :1} cc: acomp failed for imapc-connection.c gmake[3]: *** [imapc-connection.lo] Error 1 gmake[3]: Leaving directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-imap-client' Viele Gr??e, J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From ramiblanco at gmail.com Tue Nov 22 22:52:38 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Tue, 22 Nov 2011 17:52:38 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search Message-ID: Hi, I found that doing a wildcard search with doveadm doesn't expand the %d global variable when doing a wildcard search like: doveadm search *@domain ALL LARGER 10000k doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users In /var/log/dovecot.log: Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 22 20:48:07 auth-worker: Error: ldap(?): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object Is this a bug? Configs: #file /usr/local/etc/dovecot/dovecot-ldap.conf.ext: hosts = x.x.x.x dn = cn=dovecot,dc=domain dnpass = mypass auth_bind = yes auth_bind_userdn = mail=%u,vd=%d,dc=domain ldap_version = 3 base = vd=%d,dc=domain deref = never scope = subtree user_attrs = mailbox=mail=maildir:/home/vmail/domains/%$,quota=quota_rule=*:storage=%$,vdHome=home=%$/%d/%1u/%1.1u/%n,=uid=504,=gid=12 user_filter = (&(&(objectClass=VirtualMailAccount)(mail=%u))(accountActive=TRUE)) pass_attrs = userPassword=password,mail=user pass_filter = (&(&(objectClass=VirtualMailAccount)(mail=%u))(accountActive=TRUE)) iterate_attrs = mail=user iterate_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)) default_pass_scheme = CRYPT # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.3.1.el5 x86_64 CentOS release 5.5 (Final) ext3 auth_mechanisms = plain login dict { expire = pgsql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } hostname = localhost lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot.log mail_gid = 12 mail_location = maildir:/home/vmail/domains/%d/%1u/%1.1u/%n mail_plugins = quota autocreate expire mail_privileged_group = mail mail_uid = 504 managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Enviados autocreate2 = Papelera autocreate3 = Borrador autocreate4 = Spam autosubscribe = Enviados autosubscribe2 = Papelera autosubscribe3 = Borrador autosubscribe4 = Spam expire = Papelera expire2 = Papelera/* expire3 = Spam expire4 = Spam/* expire_dict = proxy::expire quota = maildir:User quota quota_rule = *:storage=100M quota_rule2 = Papelera:storage=+100M sieve = ~/sieve/dovecot.sieve sieve_before = /usr/local/etc/sieve/spam.sieve sieve_dir = ~/sieve sieve_global_dir = /usr/local/etc/sieve/ } postmaster_address = postmaster at d omain protocols = imap sieve pop3 service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } vsz_limit = 64 M } ssl_cert = References: <4ECBD518.9020206@hardwarefreak.com> Message-ID: <4ECC2149.4070403@hardwarefreak.com> On 11/22/2011 12:30 PM, Timo Sirainen wrote: > On 22.11.2011, at 19.00, Stan Hoeppner wrote: > >> The current operation on '1-Spam-l' did not succeed. The mail server >> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >> 1-Spam-l > .. >> /$ la /home/stan/mail/1-Spam-l.gz >> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz > > The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) Aha. That was it. Thanks Timo. For some reason my read of the wiki page made me think this was handled transparently--just zip the file and everything works as it did before. Apparently it's not as simple as the wiki leads one (me anyway) to believe. This bit of the wiki caused me some confusion as well: "Compressed mbox files can be accessed only as read-only" Thus I chmod'ed the .gz file to read-only. This creates a problem. It appears that when Dovecot creates the .imap folder of the same name it inherits the permissions of the zipped mbox file. Thus it can't create the indexes: 2011-11-22 14:59:23 IMAP(stan): Error: file_dotlock_open(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log) failed: Permission denied (euid=1000(stan) egid=1000(stan) access(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log, 4) failed: No such file or directory) Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki could be reworded in a way that doesn't prompt some folks to manually make the zipped files read-only. It took a while for Dovecot to index the 15K+ messages. With that finished, accessing the folder is similar to before, but there's a small lag when opening messages. As this is an archive folder the contents won't change, so Squat FTS should be very fast after the first search, just as before. Interestingly, it appears my squat indexes aren't updating--for any folder. I've searched 4 folders via Tbird body search with xyzzyx (took forever) and I see no changes to the dates or sizes of existing indexes. I deleted the squat indexes for one folder and ran the search again. No new squat indexes were created. No errors in the logs. Any ideas why the squat indexes aren't updating? IIRC this happened once before and I was able to fix it. Don't recall how I did it though.... Once again: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_process_size: 320 mail_plugins: zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster at hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10 Thanks. -- Stan From dovecot at knutejohnson.com Wed Nov 23 00:53:30 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Tue, 22 Nov 2011 14:53:30 -0800 Subject: [Dovecot] How to disable pop3 in version 2? Message-ID: <4ECC27EA.2000200@knutejohnson.com> I'm about to bring up a new mail server running Ubuntu 11.10 with Dovecot 2. I've been using 1.2 all along and there are a bunch more config files in 2. In 1.2 I just edited the protocols statement but I can't find one in version 2. Is there a way to disable pop3 or do I just firewall the ports? And a related question, I've been using imaps on port 993 but I hear that's not the best solution anymore, I should be using imap with STARTTLS? How do I disable imaps? Thanks, -- Knute Johnson From tss at iki.fi Wed Nov 23 00:56:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 00:56:10 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: Message-ID: On 22.11.2011, at 22.52, Ramiro Blanco wrote: > Hi, I found that doing a wildcard search with doveadm doesn't expand the %d > global variable when doing a wildcard search like: > > Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain > scope=subtree Iteration is supposed to iterate through all users.. > Is this a bug? More like a missing feature I guess. I guess it could make it expand the %d if it's known.. From tss at iki.fi Wed Nov 23 01:01:44 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:01:44 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC In-Reply-To: References: Message-ID: On 22.11.2011, at 22.11, J?rgen Obermann wrote: > "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) What? No it doesn't.. Does changing the 0x400000000 in line 20 to 0x200000000 help? > "imapc-connection.c", line 1446: operand cannot have void type: op "==" > "imapc-connection.c", line 1446: operands have incompatible types: Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1cdd39d11ce4 From tss at iki.fi Wed Nov 23 01:03:35 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:03:35 +0200 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <4ECC27EA.2000200@knutejohnson.com> References: <4ECC27EA.2000200@knutejohnson.com> Message-ID: <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> On 23.11.2011, at 0.53, Knute Johnson wrote: > I'm about to bring up a new mail server running Ubuntu 11.10 with Dovecot 2. I've been using 1.2 all along and there are a bunch more config files in 2. In 1.2 I just edited the protocols statement but I can't find one in version 2. Is there a way to disable pop3 or do I just firewall the ports? In the example dovecot.conf there's a protocols setting. If you don't see any, you can just add your own wherever. "protocols=imap" should work. > And a related question, I've been using imaps on port 993 but I hear that's not the best solution anymore, I should be using imap with STARTTLS? How do I disable imaps? Well, http://wiki2.dovecot.org/SSL explains this more. I don't think you really need to disable imaps, but if you want to, you can do it with: service imap-login { inet_listener imaps { port = 0 } } From tss at iki.fi Wed Nov 23 01:07:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:07:28 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECC2149.4070403@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> Message-ID: <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> On 23.11.2011, at 0.25, Stan Hoeppner wrote: > On 11/22/2011 12:30 PM, Timo Sirainen wrote: >> On 22.11.2011, at 19.00, Stan Hoeppner wrote: >> >>> The current operation on '1-Spam-l' did not succeed. The mail server >>> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >>> 1-Spam-l >> .. >>> /$ la /home/stan/mail/1-Spam-l.gz >>> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz >> >> The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) > > Aha. That was it. Thanks Timo. For some reason my read of the wiki > page made me think this was handled transparently--just zip the file and > everything works as it did before. Apparently it's not as simple as the > wiki leads one (me anyway) to believe. I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :) > This bit of the wiki caused me some confusion as well: > "Compressed mbox files can be accessed only as read-only" > > Thus I chmod'ed the .gz file to read-only. This creates a problem. It > appears that when Dovecot creates the .imap folder of the same name it > inherits the permissions of the zipped mbox file. Thus it can't create > the indexes: I've fixed this in some version. I guess in v2.0. > Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki > could be reworded in a way that doesn't prompt some folks to manually > make the zipped files read-only. Well, I don't really care that much about v1.x anymore. > It took a while for Dovecot to index the 15K+ messages. With that > finished, accessing the folder is similar to before, but there's a small > lag when opening messages. Yeah, it's uncompressing the entire file until it finds the message you're opening. > As this is an archive folder the contents > won't change, so Squat FTS should be very fast after the first search, > just as before. Interestingly, it appears my squat indexes aren't > updating--for any folder. I've searched 4 folders via Tbird body search > with xyzzyx (took forever) and I see no changes to the dates or sizes of > existing indexes. I deleted the squat indexes for one folder and ran > the search again. No new squat indexes were created. No errors in the > logs. > > Any ideas why the squat indexes aren't updating? IIRC this happened > once before and I was able to fix it. Don't recall how I did it though.... > mail_plugins: zlib Doesn't look like fts, fts_squat is enabled? From dovecot at knutejohnson.com Wed Nov 23 02:40:31 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Tue, 22 Nov 2011 16:40:31 -0800 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> Message-ID: <4ECC40FF.8020103@knutejohnson.com> On 11/22/2011 3:03 PM, Timo Sirainen wrote: > On 23.11.2011, at 0.53, Knute Johnson wrote: > >> I'm about to bring up a new mail server running Ubuntu 11.10 with >> Dovecot 2. I've been using 1.2 all along and there are a bunch >> more config files in 2. In 1.2 I just edited the protocols >> statement but I can't find one in version 2. Is there a way to >> disable pop3 or do I just firewall the ports? > > In the example dovecot.conf there's a protocols setting. If you don't > see any, you can just add your own wherever. "protocols=imap" should > work. > >> And a related question, I've been using imaps on port 993 but I >> hear that's not the best solution anymore, I should be using imap >> with STARTTLS? How do I disable imaps? > > Well, http://wiki2.dovecot.org/SSL explains this more. I don't think > you really need to disable imaps, but if you want to, you can do it > with: > > service imap-login { inet_listener imaps { port = 0 } } > > Thanks for that. The service imap-login line above is in the file /etc/dovecot/conf.d/10-master.conf. I don't know if that is standard for version 2 or something that Ubuntu created. The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; !include_try /usr/share/dovecot/protocols.d/*.protocol That directory contains one file; dovecot-imapd.protocol and it contains one line; protocols = $protocols imap So I am assuming that it is adding imap to the protocols in the environment variable $protocols somehow? dovecot -n yields; protocols = " imap" I did get TBird to retrieve mail from port 993 using SSL however. Will it work without a protocols = imaps? Thanks, -- Knute Johnson From stan at hardwarefreak.com Wed Nov 23 02:56:29 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 22 Nov 2011 18:56:29 -0600 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> Message-ID: <4ECC44BD.3080806@hardwarefreak.com> On 11/22/2011 5:07 PM, Timo Sirainen wrote: > On 23.11.2011, at 0.25, Stan Hoeppner wrote: > >> On 11/22/2011 12:30 PM, Timo Sirainen wrote: >>> On 22.11.2011, at 19.00, Stan Hoeppner wrote: >>> >>>> The current operation on '1-Spam-l' did not succeed. The mail server >>>> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >>>> 1-Spam-l >>> .. >>>> /$ la /home/stan/mail/1-Spam-l.gz >>>> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz >>> >>> The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) >> >> Aha. That was it. Thanks Timo. For some reason my read of the wiki >> page made me think this was handled transparently--just zip the file and >> everything works as it did before. Apparently it's not as simple as the >> wiki leads one (me anyway) to believe. > > I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :) Out of curiosity, what (or who) prompted the development of the compressed mbox feature? Or was it that you wanted to do it for maildir, and then figured you should for mbox as well? The implementation seems to work ok. The instructions just seem a bit...thin. :) >> This bit of the wiki caused me some confusion as well: >> "Compressed mbox files can be accessed only as read-only" >> >> Thus I chmod'ed the .gz file to read-only. This creates a problem. It >> appears that when Dovecot creates the .imap folder of the same name it >> inherits the permissions of the zipped mbox file. Thus it can't create >> the indexes: > > I've fixed this in some version. I guess in v2.0. Yeah, I'm waiting for Debian to get a backport of 2.0.x. For many reasons the only thing I'm comfortable installing from source is the Linux kernel. >> Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki >> could be reworded in a way that doesn't prompt some folks to manually >> make the zipped files read-only. > > Well, I don't really care that much about v1.x anymore. Understandable. >> It took a while for Dovecot to index the 15K+ messages. With that >> finished, accessing the folder is similar to before, but there's a small >> lag when opening messages. > > Yeah, it's uncompressing the entire file until it finds the message you're opening. It's pretty damn fast at it. I haven't seen anything more than a couple of seconds lag while randomly accessing mail all over the folder. The original gzip of the file took >45 seconds. >> As this is an archive folder the contents >> won't change, so Squat FTS should be very fast after the first search, >> just as before. Interestingly, it appears my squat indexes aren't >> updating--for any folder. I've searched 4 folders via Tbird body search >> with xyzzyx (took forever) and I see no changes to the dates or sizes of >> existing indexes. I deleted the squat indexes for one folder and ran >> the search again. No new squat indexes were created. No errors in the >> logs. >> >> Any ideas why the squat indexes aren't updating? IIRC this happened >> once before and I was able to fix it. Don't recall how I did it though.... > >> mail_plugins: zlib > > Doesn't look like fts, fts_squat is enabled? Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins had to be declared in a single line directive. I did this as part of my troubleshooting when zlib wasn't working, thinking putting it on it's own line may help--not. -- Stan From Juergen.Obermann at hrz.uni-giessen.de Wed Nov 23 12:01:25 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Wed, 23 Nov 2011 11:01:25 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10on SPARC In-Reply-To: References: Message-ID: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> ----- Nachricht von tss at iki.fi --------- Datum: Wed, 23 Nov 2011 01:01:44 +0200 Von: Timo Sirainen Betreff: Re: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC An: J?rgen Obermann Cc: dovecot at dovecot.org > On 22.11.2011, at 22.11, J?rgen Obermann wrote: > >> "imapc-client.h", line 21: warning: enumerator value overflows >> INT_MAX (2147483647) > > What? No it doesn't.. Does changing the 0x400000000 in line 20 to > 0x200000000 help? No, but stripping off the last zero helps, because INT_MAX is 0x7FFFFFFF. >> "imapc-connection.c", line 1446: operand cannot have void type: op "==" >> "imapc-connection.c", line 1446: operands have incompatible types: > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1cdd39d11ce4 > > > OK now. ----- Ende der Nachricht von tss at iki.fi ----- One more compilation problem occurs: gmake[5]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-storage/index/imapc' source='imapc-storage.c' object='imapc-storage.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../../../depcomp \ /bin/bash ../../../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-storage.lo imapc-storage.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-storage.c -KPIC -DPIC -o .libs/imapc-storage.o "../../../../src/lib-storage/mailbox-list.h", line 13: warning: useless declaration "../../../../src/lib-storage/mailbox-list-private.h", line 14: warning: useless declaration "imapc-sync.h", line 4: warning: useless declaration "imapc-storage.c", line 500: void function cannot return value cc: acomp failed for imapc-storage.c gmake[5]: *** [imapc-storage.lo] Error 1 Greetings, J?rgen Obermann -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From patrickdk at patrickdk.com Wed Nov 23 14:24:39 2011 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 23 Nov 2011 07:24:39 -0500 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECB3592.1010500@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> <4ECB3592.1010500@lazurit.com> Message-ID: <20111123072439.Horde.FvMFLZLnE6FOzOYHNPlAn_A@kishi.patrickdk.com> Quoting Alexander Chekalin : >>>> With Maildir the message GUID is typically the same as the >>>> Maildir base filename (i.e. everything before ':' character). >>> >>> But what if I one day decide to convert my maildir's to mbox'es? >>> I really plan to do such conversion in a while (as soon as I finish the >>> indexing system). >> >> mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of >> fakes it by returning MD5 of specific headers as GUIDs (so not 100% >> reliable). With mdbox GUIDs work even better than with Maildir, the GUID >> is always stored in the message's metadata. > > I'd like to use the best optimized one (mdbox), but there is a > reason not to do that is when I use mbox or maildir I can see where > given IMAP folder mails are stored, so, say, if I want to copy only > one IMAP folder to some remote site, I just copy know dir or file. > With mdbox this is different, I simply can not guess where my > messages exactly are. > > If it be possible to have per-IMAP-folders mdboxes, I love to use > it. But from what I know and tried this is not way mdbox used to > work, right? > You can always use the info from the wiki, I took some code from it to create this little script, that dumps my spam folder and uses it to learn. Basically dumps a mailfolder back into maildir format. doveadm search -u $useraccount mailbox Spam | while read guid uid; do doveadm fetch -u $useraccount text mailbox-guid $guid uid $uid > msg.$uid; done If your using mdbox on the other end, you could reimport them I suppose, I haven't looked into doing that, since I haven't needed that yet. From elhijo at 0lim.net Wed Nov 23 16:19:34 2011 From: elhijo at 0lim.net (David LEROUX) Date: Wed, 23 Nov 2011 15:19:34 +0100 Subject: [Dovecot] dovecot as an imap "client" Message-ID: <4ECD00F6.4060908@0lim.net> Hi, I need to get emails from one of my users imap account on a server which is not managed by us to his local imap mailbox. I'm almost sure that I've read some times that dovecot would be able to do that, kind of imap relay, but I can't find it again. I'm not sure to be very clear. Thanks, -- David From frank at moltke28.B.Shuttle.DE Wed Nov 23 16:53:18 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 23 Nov 2011 15:53:18 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: <4ECD00F6.4060908@0lim.net> References: <4ECD00F6.4060908@0lim.net> Message-ID: (auto-added) On Wed, 23 Nov 2011 15:19:34 +0100 David LEROUX wrote: > Hi, > I need to get emails from one of my users imap account on a server which > is not managed by us to his local imap mailbox. imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner From micah at riseup.net Wed Nov 23 16:54:21 2011 From: micah at riseup.net (Micah Anderson) Date: Wed, 23 Nov 2011 09:54:21 -0500 Subject: [Dovecot] doveadm import assertion failed Message-ID: <87vcqax52a.fsf@algae.riseup.net> Restoring a user's mailbox yesterday resulted in 'doveadm import' panic'ing with an assertion failure and giving a backtrace: /usr/bin/doveadm import -u mdbox:/maildir/riseup.net/a//.daily.1/mdbox restored_from_backups/daily1 all doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac tion log sequence (3) doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac tion log sequence (3) doveadm(): Panic: file mail-transaction-log-file.c: line 187 (mail_transaction_log_file_add_to_list): assertion failed: ((*p)->hdr.file_se q < file->hdr.file_seq) doveadm(): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd1a) [0x7f05152bbd1a] -> /usr/lib/dovecot/libdovecot.so.0(default_f atal_handler+0x32) [0x7f05152bbe02] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f051529519f] -> /usr/lib/dovecot/libdovecot-storage.so .0(+0xa596a) [0x7f05159e696a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_file_open+0x1f8) [0x7f05159e7bc8] -> /usr/lib/ dovecot/libdovecot-storage.so.0(mail_transaction_log_find_file+0xd0) [0x7f05159e46e0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transa ction_log_view_set+0xcb) [0x7f05159e8bdb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0xbe) [0x7f05159dff6e] -> /usr/lib/ dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7f05159d1816] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a6da) [0x7f05159db6da] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x56) [0x7f05159db956] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_ index_sync_begin+0x1e) [0x7f05159dc21e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x121) [0x7f05159bbb41] -> /usr/lib/dove cot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7f05159bc1f6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x43) [0x7f 05159bc293] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7f0515974031] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7f05159748b7] -> /usr/bin/doveadm(doveadm_mail_iter_init+0x8b) [0x40c9db] -> /usr/bin/doveadm() [0x40c284] -> /usr/bin/doveadm() [0x40a673] -> /usr/bin/doveadm(doveadm_mail_single_user+0x61) [0x40a731] -> /usr/bin/doveadm() [0x40a95d] -> /usr/bin/doveadm(doveadm_mail_try_run+0x141) [0x40ad71] -> /usr/bin/doveadm(main+0x381) [0x4107e1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f0514b2dc4d] -> /usr/bin/doveadm() [0x409e59] This is with 2.0.15. Micah -- From tss at iki.fi Wed Nov 23 16:58:41 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 16:58:41 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10on SPARC In-Reply-To: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> Message-ID: <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> On 23.11.2011, at 12.01, J?rgen Obermann wrote: >> On 22.11.2011, at 22.11, J?rgen Obermann wrote: >> >>> "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) >> >> What? No it doesn't.. Does changing the 0x400000000 in line 20 to 0x200000000 help? > > No, but stripping off the last zero helps, because INT_MAX is 0x7FFFFFFF. Oops :) Looks like gcc automatically increases the enum's size to 64bit in that case. > "imapc-storage.c", line 500: void function cannot return value http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e From elhijo at 0lim.net Wed Nov 23 17:07:25 2011 From: elhijo at 0lim.net (David LEROUX) Date: Wed, 23 Nov 2011 16:07:25 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: (auto-added) References: <4ECD00F6.4060908@0lim.net> (auto-added) Message-ID: <4ECD0C2D.8080901@0lim.net> On 11/23/2011 03:53 PM, Frank Elsner wrote: > imapsync comes to my mind. - http://imapsync.lamiral.info/ - > http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync > --Frank Elsner Thanks, Thought that dovecot would be able to do that. David. From info at fduerr.de Wed Nov 23 10:16:30 2011 From: info at fduerr.de (Frank) Date: Wed, 23 Nov 2011 08:16:30 +0000 (UTC) Subject: [Dovecot] dsync strange duplicate directories Message-ID: Hello, i'm testing whether dsync will help me in setting up a redundant master - master imap setup and experience this: 1. Master 1 dovecot is running, master 2 dovecot is down 2. I use rsync to create an exact copy of the master 1 maildir on master 2 3. I start up master 2 dovecot 4. I do a dsync on master 2: dsync -D -u test at example.com mirror ssh root at master1 dsync -D -u test at example.com Now what i see is this: Each mail directory is duplicated on master 1 and master 2 to a directory name with some MD5 hash value (as it seems) e.g. before dsync: ls -a /home/vmail/example.com/test/Maildir cur new .Drafts .Junk .Sent ... after dsync: ls -a /home/vmail/example.com/test/Maildir cur new .Drafts .Drafts_af61060f6f9fcb4efc0a00002691362e .Junk .Junk_b261060f6f9fcb4efc0a00002691362e .Sent .Sent_b261060f6f9fcb4efc0a00002691362e ... The new directories contain the same files as the original directories. What am i doing wrong? Thanks for the help Frank From tss at iki.fi Wed Nov 23 17:29:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 17:29:16 +0200 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: <4ECD0C2D.8080901@0lim.net> References: <4ECD00F6.4060908@0lim.net> (auto-added) <4ECD0C2D.8080901@0lim.net> Message-ID: On 23.11.2011, at 17.07, David LEROUX wrote: > On 11/23/2011 03:53 PM, Frank Elsner wrote: >> imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner > Thanks, > Thought that dovecot would be able to do that. v2.1 has "imapc" backend, which you could use with dsync to do a migration, but for now it's probably easier to just use imapsync. From Juergen.Obermann at hrz.uni-giessen.de Wed Nov 23 19:07:09 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Wed, 23 Nov 2011 18:07:09 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris10on SPARC In-Reply-To: <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> Message-ID: Am 23.11.2011 15:58, schrieb Timo Sirainen: > On 23.11.2011, at 12.01, J?rgen Obermann wrote: > >>> On 22.11.2011, at 22.11, J?rgen Obermann wrote: >>> >>>> "imapc-client.h", line 21: warning: enumerator value overflows >>>> INT_MAX (2147483647) >>> >>> What? No it doesn't.. Does changing the 0x400000000 in line 20 to >>> 0x200000000 help? >> >> No, but stripping off the last zero helps, because INT_MAX is >> 0x7FFFFFFF. > > Oops :) Looks like gcc automatically increases the enum's size to > 64bit in that case. > >> "imapc-storage.c", line 500: void function cannot return value > > http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e There seems to be one more similar problem: gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/anvil' source='anvil-connection.c' object='anvil-connection.o' libtool=no \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-master -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c anvil-connection.c "anvil-connection.c", line 156: void function cannot return value cc: acomp failed for anvil-connection.c gmake[3]: *** [anvil-connection.o] Error 2 -- J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From tss at iki.fi Wed Nov 23 19:11:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 19:11:55 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris10on SPARC In-Reply-To: References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> Message-ID: <1322068318.24607.64.camel@hurina> On Wed, 2011-11-23 at 18:07 +0100, J?rgen Obermann wrote: > > http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e > > There seems to be one more similar problem: > > "anvil-connection.c", line 156: void function cannot return value Fixed: http://hg.dovecot.org/dovecot-2.1/rev/4f1d20b57c04 From pch0317 at gmail.com Wed Nov 23 21:55:58 2011 From: pch0317 at gmail.com (pch0317) Date: Wed, 23 Nov 2011 19:55:58 +0000 Subject: [Dovecot] Problem with Outlook 2010 Message-ID: <4ECD4FCE.4000400@gmail.com> Hi, Maybe this time somebody help. Thanks -------- Original Message -------- Subject: Problem with Outlook 2010 Date: Tue, 07 Jun 2011 20:20:56 +0100 From: pch0317 To: Dovecot Mailing List Hi, I use dovecot 2.0. I have problem with Outlook 2010. This application freezes for about 2 minutes when I move or delete messages. Other applications such as Outlook Express and Thunderbird work correctly. Has anyone had similar problem? Thanks From tss at iki.fi Wed Nov 23 22:09:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 22:09:18 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: Message-ID: <1322078958.24607.76.camel@hurina> On Wed, 2011-11-23 at 00:56 +0200, Timo Sirainen wrote: > > Hi, I found that doing a wildcard search with doveadm doesn't expand the %d > > global variable when doing a wildcard search like: > > > > Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain > > scope=subtree > > Iteration is supposed to iterate through all users.. > > > Is this a bug? > > > More like a missing feature I guess. I guess it could make it expand the %d if it's known.. hg version of v2.1 now supports this. From dovecot.user at seibercom.net Wed Nov 23 22:28:00 2011 From: dovecot.user at seibercom.net (Jerry) Date: Wed, 23 Nov 2011 15:28:00 -0500 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <20111123152800.6504a2ba@scorpio> On Wed, 23 Nov 2011 19:55:58 +0000 pch0317 articulated: > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work > correctly. > > Has anyone had similar problem? Seriously, do you have any log messages, etcetera to diagnose this problem with? -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From dovecot.user at seibercom.net Wed Nov 23 22:31:56 2011 From: dovecot.user at seibercom.net (Jerry) Date: Wed, 23 Nov 2011 15:31:56 -0500 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <20111123153156.0c02bb74@scorpio> On Wed, 23 Nov 2011 19:55:58 +0000 pch0317 articulated: > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work > correctly. You could start here to get some useful information as to what Outlook 2010 is doing. http://www.theemailadmin.com/2010/08/troubleshooting-outlook-2010-connections/ -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Wed Nov 23 22:57:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 22:57:50 +0200 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <4E754D10.5010404@babelmonkeys.de> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> Message-ID: <1322081870.24607.78.camel@hurina> On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: > Am 16.09.2011 14:40, schrieb Timo Sirainen: > > On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: > > > > Looks pretty good. Below are a few things I noticed. I could fix these > > myself next week also, or you can do them during weekend if you want > > to. :) > > > I decided to do it myself, hope this fixes all issues. Committed finally to http://hg.dovecot.org/dovecot-2.1 with some cleanups. From tss at iki.fi Wed Nov 23 23:02:30 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:02:30 +0200 Subject: [Dovecot] Can't delete sub-folder In-Reply-To: <4EBAFFD7.8020201@gmail.com> References: <4EBAFFD7.8020201@gmail.com> Message-ID: <1322082151.24607.79.camel@hurina> On Wed, 2011-11-09 at 22:33 +0000, pch0317 wrote: > I use thunderbird 8.0 as a client and dovecot 2.0.9 with imap protocol .. > When I try to delete 'parent' folder which contain 'messages only' > folder, error message appear: [ALREADYEXISTS] Mailbox has children, > delete them first. Ok, so I delete 'message only' folder first and try > to delete parent folder, but the same error appear: [ALREADYEXISTS] > Mailbox has children, delete them first. > When I list mailboxes from console I see inside 'parent' folder ".imap" > folder. When I delete .imap folder from console and then delete parent > folder from thunderbird, parent folder is deleted correctly. You can reproduce this? I tried with v2.0.16 and it works ok, so I have probably fixed this since v2.0.9. From stephan at rename-it.nl Wed Nov 23 23:07:14 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 23 Nov 2011 22:07:14 +0100 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <1322081870.24607.78.camel@hurina> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> <1322081870.24607.78.camel@hurina> Message-ID: <4ECD6082.4070406@rename-it.nl> On 11/23/2011 9:57 PM, Timo Sirainen wrote: > On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: >> Am 16.09.2011 14:40, schrieb Timo Sirainen: >>> On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: >>> >>> Looks pretty good. Below are a few things I noticed. I could fix these >>> myself next week also, or you can do them during weekend if you want >>> to. :) >>> >> I decided to do it myself, hope this fixes all issues. > Committed finally to http://hg.dovecot.org/dovecot-2.1 with some > cleanups. Great, the Pigeonhole ManageSieve implementation is now suddenly fully RFC compliant, without any effort on my part! :) Regards, Stephan. From tss at iki.fi Wed Nov 23 23:11:53 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:11:53 +0200 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <4ECD6082.4070406@rename-it.nl> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> <1322081870.24607.78.camel@hurina> <4ECD6082.4070406@rename-it.nl> Message-ID: <1322082713.24607.81.camel@hurina> On Wed, 2011-11-23 at 22:07 +0100, Stephan Bosch wrote: > On 11/23/2011 9:57 PM, Timo Sirainen wrote: > > On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: > >> Am 16.09.2011 14:40, schrieb Timo Sirainen: > >>> On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: > >>> > >>> Looks pretty good. Below are a few things I noticed. I could fix these > >>> myself next week also, or you can do them during weekend if you want > >>> to. :) > >>> > >> I decided to do it myself, hope this fixes all issues. > > Committed finally to http://hg.dovecot.org/dovecot-2.1 with some > > cleanups. > > Great, the Pigeonhole ManageSieve implementation is now suddenly fully > RFC compliant, without any effort on my part! :) I'm not actually sure about that :) The final replying is probably wrong, since ManageSieve supports sending it to client, unlike IMAP/POP3.. From elhijo at 0lim.net Wed Nov 23 23:13:20 2011 From: elhijo at 0lim.net (elhijo) Date: Wed, 23 Nov 2011 22:13:20 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: References: <4ECD00F6.4060908@0lim.net> (auto-added) <4ECD0C2D.8080901@0lim.net> Message-ID: <4ECD61F0.1010806@0lim.net> On 23/11/2011 16:29, Timo Sirainen wrote: > On 23.11.2011, at 17.07, David LEROUX wrote: > >> On 11/23/2011 03:53 PM, Frank Elsner wrote: >>> imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner >> Thanks, >> Thought that dovecot would be able to do that. > v2.1 has "imapc" backend, which you could use with dsync to do a migration, but for now it's probably easier to just use imapsync. > I'll stick to imapsync which seems to prefectly fit to my needs. Thanks for your help, David From stephan at rename-it.nl Wed Nov 23 23:15:00 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 23 Nov 2011 22:15:00 +0100 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <4EB9BD21.6060501@rename-it.nl> References: <1320791738.21919.350.camel@hurina> <4EB9BD21.6060501@rename-it.nl> Message-ID: <4ECD6254.2020606@rename-it.nl> On 11/9/2011 12:37 AM, Stephan Bosch wrote: > On 11/8/2011 11:35 PM, Timo Sirainen wrote: >> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz >> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig >> >> Here's the first beta release of Dovecot v2.1. This version has already >> been tested quite a lot, so I'm not expecting any major bugs. So please >> upgrade and see if you can find any problems. I'm optimistic about >> getting rc1 released this year and perhaps even v2.1.0. >> > > Ok, I guess it is time for a Pigeonhole release for Dovecot v2.1. I'm > a bit swamped this week, but next weekend I should be able to compose > one. For now you can use the dovecot-2.0-pigeonhole hg and patch it > with > http://hg.rename-it.nl/pigeonhole-0.2-dovecot-2.1-patches/file/tip/pigeonhole-0.2-dovecot-2.1.patch > to make it compile against v2.1. I have an update on this. Although there is no tarball release so far, I did start a separate repository with Pigeonhole (v0.3.x) for Dovecot v2.1: http://hg.rename-it.nl/dovecot-2.1-pigeonhole This means that the patch procedure mentioned above is no longer necessary. From this point onwards, new features are only implemented for Pigeonhole v0.3.x, unless a back-port can be achieved with little risk and effort. First in line is support for the editheader extension, which I expect to finish next week. Regards, Stephan. From ramiblanco at gmail.com Wed Nov 23 23:19:35 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 18:19:35 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322078958.24607.76.camel@hurina> References: <1322078958.24607.76.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > > hg version of v2.1 now supports this. > Does it need some extra configuration? I've upgraded to hg ver. of 2.1 and still the same error... Thanks for the quick response! -- Ramiro Blanco From tss at iki.fi Wed Nov 23 23:27:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:27:47 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> Message-ID: <1322083667.24607.82.camel@hurina> On Wed, 2011-11-23 at 18:19 -0300, Ramiro Blanco wrote: > 2011/11/23 Timo Sirainen : > > > > hg version of v2.1 now supports this. > > > > Does it need some extra configuration? No. > I've upgraded to hg ver. of 2.1 and still the same error... Does it still show "%d" in the error messages, or is it "" now? From ramiblanco at gmail.com Thu Nov 24 00:08:20 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 19:08:20 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322083667.24607.82.camel@hurina> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > > Does it still show "%d" in the error messages, or is it "" now? > It still shows "%d" Nov 23 18:42:59 auth-worker(729): Debug: ldap: iterate: base=vd=%d,dc=domain scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 23 18:42:59 auth-worker(729): Error: ldap(): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object -- Ramiro Blanco From tss at iki.fi Thu Nov 24 01:13:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:13:06 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> Message-ID: <1322089987.24607.84.camel@hurina> On Wed, 2011-11-23 at 19:08 -0300, Ramiro Blanco wrote: > 2011/11/23 Timo Sirainen : > > > > Does it still show "%d" in the error messages, or is it "" now? > > > It still shows "%d" > > Nov 23 18:42:59 auth-worker(729): Debug: ldap: iterate: > base=vd=%d,dc=domain scope=subtree Actually only the debug message was wrong. The expansion should have worked. Anyway, fixed the debug message: http://hg.dovecot.org/dovecot-2.1/rev/1fed3c1c166e From tss at iki.fi Thu Nov 24 01:21:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:21:04 +0200 Subject: [Dovecot] doveadm import assertion failed In-Reply-To: <87vcqax52a.fsf@algae.riseup.net> References: <87vcqax52a.fsf@algae.riseup.net> Message-ID: <1322090464.24607.87.camel@hurina> On Wed, 2011-11-23 at 09:54 -0500, Micah Anderson wrote: > Restoring a user's mailbox yesterday resulted in 'doveadm import' > panic'ing with an assertion failure and giving a backtrace: > > /usr/bin/doveadm import -u mdbox:/maildir/riseup.net/a//.daily.1/mdbox restored_from_backups/daily1 all > doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac > tion log sequence (3) So this path points to the backed up mailbox you're importing, not the destination mailbox? And you can reproduce this crash reliably? Could you send me these files: mailboxes/INBOX/dbox-Mails/dovecot.index mailboxes/INBOX/dbox-Mails/dovecot.index.log files? storage/dovecot.map.index storage/dovecot.map.index.log None of them contain anything sensitive about the user's mailbox. From tss at iki.fi Thu Nov 24 01:23:26 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:23:26 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECC44BD.3080806@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> <4ECC44BD.3080806@hardwarefreak.com> Message-ID: <1322090606.24607.89.camel@hurina> On Tue, 2011-11-22 at 18:56 -0600, Stan Hoeppner wrote: > Out of curiosity, what (or who) prompted the development of the > compressed mbox feature? Or was it that you wanted to do it for > maildir, and then figured you should for mbox as well? The > implementation seems to work ok. The instructions just seem a > bit...thin. :) I think it was one of my friends saying he can't start using Dovecot until it supports compressed mboxes. > >> mail_plugins: zlib > > > > Doesn't look like fts, fts_squat is enabled? > > Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So > 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins > had to be declared in a single line directive. I did this as part of my > troubleshooting when zlib wasn't working, thinking putting it on it's > own line may help--not. With v2.0 you can do: mail_plugins = $mail_plugins zlib mail_plugins = $mail_plugins fts fts_squat From tss at iki.fi Thu Nov 24 01:25:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:25:03 +0200 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <4ECC40FF.8020103@knutejohnson.com> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> <4ECC40FF.8020103@knutejohnson.com> Message-ID: <1322090703.24607.91.camel@hurina> On Tue, 2011-11-22 at 16:40 -0800, Knute Johnson wrote: > The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; > > !include_try /usr/share/dovecot/protocols.d/*.protocol > > That directory contains one file; > > dovecot-imapd.protocol > > and it contains one line; > > protocols = $protocols imap > > So I am assuming that it is adding imap to the protocols in the > environment variable $protocols somehow? Not environment variable $protocols, but the previous "protocols" setting's value in dovecot.conf. > I did get TBird to retrieve mail from port 993 using SSL however. Will > it work without a protocols = imaps? Yeah, there's no such thing as "imaps protocol". I should never have included it as a protocol in previous Dovecot versions either.. From tss at iki.fi Thu Nov 24 01:55:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:55:08 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4ECAD578.4060304@informatik.uni-freiburg.de> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> Message-ID: <1322092508.24607.93.camel@hurina> On Mon, 2011-11-21 at 23:49 +0100, Martin Preen wrote: > # /usr/sfw/bin/openssl engine > (pkcs11) PKCS #11 engine support > > >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); > >> ENGINE_set_default_ciphers(e); > >> > >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() > >> the crypto device gets used. I'm sure that this is not the whole story since > >> this only seems to affect the IMAP login. > > > > It should work for POP3 as well, all of the SSL code is shared. > > I couldn't find the EncryptUpdate call which has to be changed too > (due to the howto documents). Maybe some other call needs e patch. > But I don't know which. What EncryptUpdate?.. I've anyway added the engine init/deinit calls in your email to v2.1 hg. Lets hope it works :) At least it didn't break when I tried it with "dynamic" value (which is the only engine my OpenSSL supports). From tss at iki.fi Thu Nov 24 02:08:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:08:23 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1322092508.24607.93.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> <1322092508.24607.93.camel@hurina> Message-ID: <1322093304.24607.94.camel@hurina> On Thu, 2011-11-24 at 01:55 +0200, Timo Sirainen wrote: > What EncryptUpdate?.. I've anyway added the engine init/deinit calls in > your email to v2.1 hg. Lets hope it works :) At least it didn't break > when I tried it with "dynamic" value (which is the only engine my > OpenSSL supports). Oh, and in Solaris "pkcs11" value also works. But I don't know if it's actually using hardware crypto. From tss at iki.fi Thu Nov 24 02:25:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:25:17 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC4922B.5050000@goldey.net> References: <4EC4922B.5050000@goldey.net> Message-ID: <1322094317.24607.96.camel@hurina> On Wed, 2011-11-16 at 23:48 -0500, Mark Goldey wrote: > (gdb) bt full > #0 0x00000000 in ?? () > No symbol table info available. > #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, > args=0x82601c8) at auth-worker-client.c:405 Still doesn't make any sense.. Try what it logs with attached patch. And perhaps also try running via valgrind: service auth-worker { executable = /usr/bin/valgrind -q /usr/local/libexec/dovecot/auth -w } -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 800 bytes Desc: not available URL: From nanitous at xs4all.nl Thu Nov 24 02:44:44 2011 From: nanitous at xs4all.nl (Nanitous) Date: Thu, 24 Nov 2011 01:44:44 +0100 Subject: [Dovecot] Excluding some system users after configuration Message-ID: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> Dear readers, Consider an installed Postfix configuration (2.3.3) on CentOS. Now I want to install Dovecot (alas only version 1.0.7.7 is available) to provide IMAP services for a few system users, but not all. Most importantly the mail delivery by postfix for some of these system user /must/ not change after the installation of Dovecot. How can I prevent the installation of Dovecot to touch some specific system user accounts? Thanks in advance, /Twan From dovecot at knutejohnson.com Thu Nov 24 02:45:10 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Wed, 23 Nov 2011 16:45:10 -0800 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <1322090703.24607.91.camel@hurina> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> <4ECC40FF.8020103@knutejohnson.com> <1322090703.24607.91.camel@hurina> Message-ID: <4ECD9396.1050703@knutejohnson.com> On 11/23/2011 3:25 PM, Timo Sirainen wrote: > On Tue, 2011-11-22 at 16:40 -0800, Knute Johnson wrote: > >> The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; >> >> !include_try /usr/share/dovecot/protocols.d/*.protocol >> >> That directory contains one file; >> >> dovecot-imapd.protocol >> >> and it contains one line; >> >> protocols = $protocols imap >> >> So I am assuming that it is adding imap to the protocols in the >> environment variable $protocols somehow? > > Not environment variable $protocols, but the previous "protocols" > setting's value in dovecot.conf. > >> I did get TBird to retrieve mail from port 993 using SSL however. Will >> it work without a protocols = imaps? > > Yeah, there's no such thing as "imaps protocol". I should never have > included it as a protocol in previous Dovecot versions either.. > Thanks very much Timo! -- Knute Johnson From tss at iki.fi Thu Nov 24 02:46:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:46:56 +0200 Subject: [Dovecot] v2.1.rc1 released Message-ID: <1322095616.24607.108.camel@hurina> http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz.sig I'll probably release v2.1.0 pretty soon, unless people report some new bugs. I know v2.1 is already being used to serve mails to tens (or hundreds?) of thousands of users, so it should be pretty stable. Most of the changes since v2.1.beta1 have been for rather small issues. Many of the fixed bugs exist also in v2.0, but since they are so minor I didn't bother backporting the fixes. As new features since v2.1.beta1 there are the things I did also for v2.0.16, and: * ssl_client_cert/key: Proxying can now send SSL certificate to backend server * ssl_crypto_device for enabling hardware encryption * SCRAM-SHA-1 authentication mechanism support by Florian Zeitz * passdb/userdb checkpassword: Export all auth %variables to AUTH_* environment. * maildir_broken_filename_sizes setting for people upgrading from other maildir servers that wrote tons of broken S= values And as a reminder, here are the biggest new things since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now From jayw at interoceansystems.com Thu Nov 24 02:54:16 2011 From: jayw at interoceansystems.com (Jay Welch) Date: Wed, 23 Nov 2011 16:54:16 -0800 Subject: [Dovecot] Random Loss of Connection with Thunderbird and Dovecot Message-ID: <4ECD95B8.8070807@interoceansystems.com> Hi, Dovecot Version: 1.2.16 OS: CENTOS 5.5 I am having issues with Thunderbird connecting to Dovecot at random times. Usually an error comes up and states that Thunderbird cannot connect to the server. The only way I have been able to fix this is reset/repair the network connection or reboot the computer. Is there any parameters I can change in Dovecot to fix this issue? Does anyone know what is likely going on? I have been reading forums and I cannot find anything. Thanks From tss at iki.fi Thu Nov 24 02:51:48 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:51:48 +0200 Subject: [Dovecot] Excluding some system users after configuration In-Reply-To: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> References: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> Message-ID: <1322095908.24607.110.camel@hurina> On Thu, 2011-11-24 at 01:44 +0100, Nanitous wrote: > Dear readers, > > Consider an installed Postfix configuration (2.3.3) on CentOS. > Now I want to install Dovecot (alas only version 1.0.7.7 is available) to provide IMAP services for a few system users, but not all. Most importantly the mail delivery by postfix for some of these system user /must/ not change after the installation of Dovecot. > > How can I prevent the installation of Dovecot to touch some specific system user accounts? As long as you don't configure Postfix to deliver mails via Dovecot's "deliver", there is no change to how mails are delivered. If you want to restrict IMAP access to only some users, you can do it with pam_listfile: http://wiki.dovecot.org/Authentication/RestrictAccess From tss at iki.fi Thu Nov 24 02:56:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:56:07 +0200 Subject: [Dovecot] Random Loss of Connection with Thunderbird and Dovecot In-Reply-To: <4ECD95B8.8070807@interoceansystems.com> References: <4ECD95B8.8070807@interoceansystems.com> Message-ID: <1322096168.24607.114.camel@hurina> On Wed, 2011-11-23 at 16:54 -0800, Jay Welch wrote: > Dovecot Version: 1.2.16 > > > I am having issues with Thunderbird connecting to Dovecot at random > times. Usually an error comes up and states that Thunderbird cannot > connect to the server. The only way I have been able to fix this is > reset/repair the network connection or reboot the computer. Is there any > parameters I can change in Dovecot to fix this issue? Does anyone know > what is likely going on? I have been reading forums and I cannot find > anything. Dovecot always logs the reason for logout, e.g.: > Nov 24 02:53:24 imap(tss): Info: Connection closed in=0 out=291 This means that the connection got disconnected (i.e. Dovecot didn't do the disconnection) > Nov 24 02:52:47 imap(tss): Info: Disconnected: Logged out in=8 out=334 And this means that the client used a LOGOUT command. and so on. Find the log entries about the times when you're having problems. Dovecot logs about every single connection it sees. If you don't see a log message about some connection, it means that Dovecot never saw it, and the problem is outside Dovecot. I'd first look into any antivirus/software firewalls you've installed. They're usually the reason for connection problems with Windows. From ramiblanco at gmail.com Thu Nov 24 04:00:28 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 23:00:28 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322089987.24607.84.camel@hurina> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > Actually only the debug message was wrong. The expansion should have > worked. Anyway, fixed the debug message: > http://hg.dovecot.org/dovecot-2.1/rev/1fed3c1c166e Still failing: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users Now "%d" it's empty, it shows "vd=" instead of "vd=%d": Nov 23 22:50:46 auth-worker(26556): Debug: ldap: iterate: base=vd=,dc=uvq,dc=edu,dc=ar scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 23 22:50:46 auth-worker(26556): Error: ldap(): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object It is supposed to parse domain part from input in "doveadm search *@domain ..", right? -- Ramiro Blanco From tss at iki.fi Thu Nov 24 04:20:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 04:20:25 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> Message-ID: <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> On 24.11.2011, at 4.00, Ramiro Blanco wrote: > Now "%d" it's empty, it shows "vd=" instead of "vd=%d": > > It is supposed to parse domain part from input in "doveadm search > *@domain ..", right? Too tired to test search myself now, but this is what I used to test it myself: doveadm user '*@domain' If that works, then the problem is with doveadm and it needs more patching.. From ramiblanco at gmail.com Thu Nov 24 04:47:24 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 23:47:24 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> Message-ID: 2011/11/23 Timo Sirainen : > Too tired to test search myself now, but this is what I used to test it myself: > You've done too much already! > doveadm user '*@domain' > > If that works, then the problem is with doveadm and it needs more patching.. That works just fine. I guess it needs some patching then. -- Ramiro Blanco From robert at schetterer.org Thu Nov 24 08:05:47 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 24 Nov 2011 07:05:47 +0100 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <4ECDDEBB.6000608@schetterer.org> Am 23.11.2011 20:55, schrieb pch0317: > Hi, > > Maybe this time somebody help. > > Thanks > > -------- Original Message -------- > Subject: Problem with Outlook 2010 > Date: Tue, 07 Jun 2011 20:20:56 +0100 > From: pch0317 > To: Dovecot Mailing List > > > > Hi, > > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work correctly. > > Has anyone had similar problem? > > > Thanks > > no problem with any outlook version what are your logs about the problem? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From oni-neko at gmx.net Thu Nov 24 09:10:27 2011 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Thu, 24 Nov 2011 08:10:27 +0100 Subject: [Dovecot] seting acl for master user? In-Reply-To: <1321636922.11489.46.camel@hurina> References: <20111118133428.63140@gmx.net> <1321636922.11489.46.camel@hurina> Message-ID: <20111124071027.28190@gmx.net> thank you very much for the reply, it works now, after i removed all the acl-stuff. =) i don't really know why it didn't work directly after i set up the master user (but before trying around with acls), but hey, I'm not complaning that it's working now =) thanks again and sorry for the late re =) bye sil -------- Original-Nachricht -------- > Datum: Fri, 18 Nov 2011 19:22:02 +0200 > Von: Timo Sirainen > An: oni-neko at gmx.net > CC: dovecot at dovecot.org > Betreff: Re: [Dovecot] seting acl for master user? > On Fri, 2011-11-18 at 14:34 +0100, oni-neko at gmx.net wrote: > > > what i don't get is the acl-setting, so that the master-user can > > access all the other users mails and folders. > .. > > in my case I only want the master user (called cyrus) to be able to do > > everything with/in all mailboxes, so I made a > > file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". > > Unfortunately currently there is no default ACL file. The above would > create an ACL file for a mailbox called "default". I'll need to get the > default ACLs implemented some day.. > > But if you don't need ACLs for anything else, you can simply disable the > ACL plugin and the master user will have access to everything. If you do > need ACLs, there are some kludges you can still do. > > -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de From preen at informatik.uni-freiburg.de Thu Nov 24 10:45:26 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Thu, 24 Nov 2011 09:45:26 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1322092508.24607.93.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> <1322092508.24607.93.camel@hurina> Message-ID: <4ECE0426.7020805@informatik.uni-freiburg.de> Timo Sirainen wrote: > On Mon, 2011-11-21 at 23:49 +0100, Martin Preen wrote: > >> # /usr/sfw/bin/openssl engine >> (pkcs11) PKCS #11 engine support >> >>>> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >>>> ENGINE_set_default_ciphers(e); >>>> >>>> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >>>> the crypto device gets used. I'm sure that this is not the whole story since >>>> this only seems to affect the IMAP login. >>> It should work for POP3 as well, all of the SSL code is shared. >> I couldn't find the EncryptUpdate call which has to be changed too >> (due to the howto documents). Maybe some other call needs e patch. >> But I don't know which. > > What EncryptUpdate?.. I've anyway added the engine init/deinit calls in > your email to v2.1 hg. Lets hope it works :) At least it didn't break > when I tried it with "dynamic" value (which is the only engine my > OpenSSL supports). Probably I'm wrong (I have no experience with SSL programming). I thught the EncryptUpdate was necessary for the encoding of the SSL data stream. But maybe there has to be a link between engine initialization and the SSL contexts ? Martin --------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/~preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6337 bytes Desc: S/MIME Cryptographic Signature URL: From olli.raisanen at telemail.fi Thu Nov 24 11:22:20 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-1?q?R=E4is=E4nen?=) Date: Thu, 24 Nov 2011 11:22:20 +0200 Subject: [Dovecot] How to create home directories for virtual users? Message-ID: <201111241122.21131.olli.raisanen@telemail.fi> Hello, I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home directory should not be the same as mail directory. When I create new user accounts with PostfixAdmin the mail directory is not a problem but is there some smart way to create the home directory as well (locating it for instance in '/srv/vmail/%d/%n/home')? Regards, Olli From pw at wk-serv.de Thu Nov 24 12:04:25 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 11:04:25 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <20111122104547.GA10680@dibs.tanso.net> References: <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> Message-ID: <1d5f057161f05918ab010a979c455d9c@wk-serv.de> On Tue, 22 Nov 2011 11:45:47 +0100, Jan-Frode Myklebust wrote: > Ah, then Timo's reply was right. He suggested you do the > lmtp-deliveries > to the same server that you would send you imap-user to. You can do > this > trough dovecot director and lmtp-proxying. > > So instead of: > > lmtp:unix:private/dovecot-lmtp > > you should use: > > lmtp:tcp:1.2.3.4:24 > > where 1.2.3.4 would be the Dovecot LMTP proxy that proxies to the > same > machine as you would use for imap for this particular recipient. I see. So as far as I understood: - I set up a new server as LMTP proxy for my two MX 10 to connect to - the proxy redirects to my backend imap servers which will then store the mails on my shared storage and the index files to a local disk (so I have to enable LMTP additionally to enable this servers to store the mails) - I set up a frontend imap server for my users to connect to which will redirect them to the backend servers Am I right so far? Patrick From johannes at sipsolutions.net Thu Nov 24 15:11:06 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 14:11:06 +0100 Subject: [Dovecot] modules directory Message-ID: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Hi! I'm a little confused with how plugin loading works in dovecot 2, I can't seem to make it load from $(moduledir)/imap/. Is that intended to work? Just a configuration issue? Reason I ask is that I ported my antispam plugin (I know there's a fork, but I still like mine better) to dovecot 2 and the default "make install" no longer loads properly with just "mail_plugins = antispam". So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, but I still see subdirectories for at least auth. Should I install into just $(moduledir)/ now? johannes From pw at wk-serv.de Thu Nov 24 15:35:55 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 14:35:55 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <20111124104709.GB24297@dibs.tanso.net> References: <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> Message-ID: <4ECE483B.5090208@wk-serv.de> Jan-Frode Myklebust schrieb: > Yes, and in ascii-art this becomes :-) > > MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > \---(lmtp)---> backend-server2 > > > IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > \--(imap) --> backend-server2 I failed creating a nice ascii view ;-) However, I have some more questions: - What happens if the MTA accepted a message but the backend-server fails and is unable to store it? - What about load-balancing and/or failover? Maybe all users on backend-server1 are fetching their mails causing high load while backend-server2 is idle. Is it possible to specify more than one backend-server IPs as I can do for the database hosts for userdb queries? From pw at wk-serv.de Thu Nov 24 15:38:43 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 14:38:43 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <20111124104709.GB24297@dibs.tanso.net> References: <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> Message-ID: <4ECE48E3.1070806@wk-serv.de> Jan-Frode Myklebust schrieb: > MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > \---(lmtp)---> backend-server2 > > > IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > \--(imap) --> backend-server2 In this scenario it should be possible to use a non cluster filesystem to store my mails, isn't it? Patrick From tss at iki.fi Thu Nov 24 16:02:24 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 16:02:24 +0200 Subject: [Dovecot] modules directory In-Reply-To: <1322140266.5366.8.camel@jlt3.sipsolutions.net> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Message-ID: On 24.11.2011, at 15.11, Johannes Berg wrote: > I'm a little confused with how plugin loading works in dovecot 2, I > can't seem to make it load from $(moduledir)/imap/. Is that intended to > work? Just a configuration issue? > > Reason I ask is that I ported my antispam plugin (I know there's a fork, > but I still like mine better) to dovecot 2 and the default "make > install" no longer loads properly with just "mail_plugins = antispam". > > So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, > but I still see subdirectories for at least auth. There's no /imap/ directory anymore, because it was annoying having to create symlinks to most of the plugins. Then with v2.0 there came new binaries and it wouldn't have made much sense adding lots of more directories with lots of more symlinks.. > Should I install into just $(moduledir)/ now? Yep. From janfrode at tanso.net Thu Nov 24 16:21:55 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 24 Nov 2011 15:21:55 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <4ECE48E3.1070806@wk-serv.de> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE48E3.1070806@wk-serv.de> Message-ID: <20111124142155.GA28600@dibs.tanso.net> On Thu, Nov 24, 2011 at 02:38:43PM +0100, Patrick Westenberg wrote: > Jan-Frode Myklebust schrieb: > > >MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > > \---(lmtp)---> backend-server2 > > > > > >IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > > \--(imap) --> backend-server2 > > In this scenario it should be possible to use a non cluster filesystem > to store my mails, isn't it? Yes, that should work fine -- assuming you have acceptable uptime on the backend server that is hosting the messages. You also need to configure a user<->backend-server mapping trough proxy extrafield to make sure that the user lands on the correct server: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy -jf From johannes at sipsolutions.net Thu Nov 24 16:13:40 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 15:13:40 +0100 Subject: [Dovecot] modules directory In-Reply-To: References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Message-ID: <1322144020.5366.20.camel@jlt3.sipsolutions.net> On Thu, 2011-11-24 at 16:02 +0200, Timo Sirainen wrote: > On 24.11.2011, at 15.11, Johannes Berg wrote: > > > I'm a little confused with how plugin loading works in dovecot 2, I > > can't seem to make it load from $(moduledir)/imap/. Is that intended to > > work? Just a configuration issue? > > > > Reason I ask is that I ported my antispam plugin (I know there's a fork, > > but I still like mine better) to dovecot 2 and the default "make > > install" no longer loads properly with just "mail_plugins = antispam". > > > > So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, > > but I still see subdirectories for at least auth. > > There's no /imap/ directory anymore, because it was annoying having to > create symlinks to most of the plugins. Then with v2.0 there came new > binaries and it wouldn't have made much sense adding lots of more > directories with lots of more symlinks.. > > > Should I install into just $(moduledir)/ now? > > Yep. Alright, thanks! Do you remember if that would be compatible with older versions as well, or do those require it being in /imap/? johannes From tss at iki.fi Thu Nov 24 16:15:58 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 16:15:58 +0200 Subject: [Dovecot] modules directory In-Reply-To: <1322144020.5366.20.camel@jlt3.sipsolutions.net> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> <1322144020.5366.20.camel@jlt3.sipsolutions.net> Message-ID: <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> On 24.11.2011, at 16.13, Johannes Berg wrote: >>> Should I install into just $(moduledir)/ now? >> >> Yep. > > Alright, thanks! Do you remember if that would be compatible with older > versions as well, or do those require it being in /imap/? What changed was that the default mail_plugin_dir changed from being $moduledir/imap to just plain $moduledir. So older versions require it being in imap/ by default. From johannes at sipsolutions.net Thu Nov 24 16:19:41 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 15:19:41 +0100 Subject: [Dovecot] modules directory In-Reply-To: <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> <1322144020.5366.20.camel@jlt3.sipsolutions.net> <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> Message-ID: <1322144381.5366.21.camel@jlt3.sipsolutions.net> On Thu, 2011-11-24 at 16:15 +0200, Timo Sirainen wrote: > On 24.11.2011, at 16.13, Johannes Berg wrote: > > >>> Should I install into just $(moduledir)/ now? > >> > >> Yep. > > > > Alright, thanks! Do you remember if that would be compatible with older > > versions as well, or do those require it being in /imap/? > > What changed was that the default mail_plugin_dir changed from being > $moduledir/imap to just plain $moduledir. So older versions require it > being in imap/ by default. Ok, thanks. I'll find a way to make it depend on the version. johannes From janfrode at tanso.net Thu Nov 24 16:33:21 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 24 Nov 2011 15:33:21 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <4ECE483B.5090208@wk-serv.de> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> Message-ID: <20111124143321.GB28600@dibs.tanso.net> On Thu, Nov 24, 2011 at 02:35:55PM +0100, Patrick Westenberg wrote: > > - What happens if the MTA accepted a message but the backend-server > fails and is unable to store it? Your MTA will be notified about the failure trough SMTP/LMTP error codes, and retry if it was a temp-failure. > - What about load-balancing and/or failover? > Maybe all users on backend-server1 are fetching their mails causing > high load while backend-server2 is idle. > Is it possible to specify more than one backend-server IPs as I can > do for the database hosts for userdb queries? We use a clusterfs (so it's not very important which backend server the user is directed to), and have configured the dovecot director with: director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 passdb { args = proxy=y nopassword=y driver = static } which gives us loadbalancing of new logins, and also failover should a backend server fail. If you want to work with at local index and maildirs on NFS, you'll need to configure something smarter for selecting backend server. Maybe use proxy extrafield to specify a host for the user, and point this at a failover IP that will hang on the preferred server by default, but fail over to another working server if the preferred server fails.. http://wiki2.dovecot.org/PasswordDatabase/ExtraField -jf From simon.brereton at buongiorno.com Thu Nov 24 16:42:49 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 24 Nov 2011 09:42:49 -0500 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111241122.21131.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: On Nov 24, 2011 4:22 AM, "Olli R?is?nen" wrote: > > Hello, > > I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. > Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > directory should not be the same as mail directory. When I create new user > accounts with PostfixAdmin the mail directory is not a problem but is there > some smart way to create the home directory as well (locating it for instance > in '/srv/vmail/%d/%n/home')? Why do you need a home directory for virtual users? Simon From JTRUTWIN at CSBSJU.EDU Thu Nov 24 16:46:57 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Thu, 24 Nov 2011 14:46:57 +0000 Subject: [Dovecot] POP - preventing re-download with server move Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> Hi, Curious if anyone has any good suggestions to handle this. I try to get my hosting users to use IMAP but most use Outhouse and the IMAP support is somewhat annoying for many so I typically have them change their settings when using POP to leave copies on the server. That way when they get the inevitable virus or wanna use webmail they have a backup copy and webmail sees their saved mail. So now I have to move my sites to a new server and I'm trying to avoid having all these messages re-downloaded by POP clients. Some of the sites I've moved with low email usage have just dealt with the re-download and cleaned up afterwards. I have a few other accounts though that I'm more concerned about that have massive amounts of stored email. The mail is stored in Maildirs - a sample file: 1280409166.23580.foo:2,S If I move this to server "bar" would renaming the file to 1280409166.23580.bar:2,S prevent this somehow? Old server uses qmail-pop, new server uses dovecot 2.0. Thanks for any tips, Josh From robert at schetterer.org Thu Nov 24 17:07:44 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 24 Nov 2011 16:07:44 +0100 Subject: [Dovecot] POP - preventing re-download with server move In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> Message-ID: <4ECE5DC0.4000503@schetterer.org> Am 24.11.2011 15:46, schrieb Trutwin, Joshua: > Hi, > > Curious if anyone has any good suggestions to handle this. > > I try to get my hosting users to use IMAP but most use Outhouse and > the IMAP support is somewhat annoying for many so I typically have > them change their settings when using POP to leave copies on the > server. That way when they get the inevitable virus or wanna use > webmail they have a backup copy and webmail sees their saved mail. > > So now I have to move my sites to a new server and I'm trying to > avoid having all these messages re-downloaded by POP clients. Some > of the sites I've moved with low email usage have just dealt with the > re-download and cleaned up afterwards. I have a few other accounts > though that I'm more concerned about that have massive amounts of > stored email. > > The mail is stored in Maildirs - a sample file: 1280409166.23580.foo:2,S > > If I move this to server "bar" would renaming the file to > 1280409166.23580.bar:2,S prevent this somehow? > > Old server uses qmail-pop, new server uses dovecot 2.0. > > Thanks for any tips, > > Josh perhaps you can do it i.e like this use imapsync with diff between the two servers then block pop3 with firewall for public on the old one, run a last sync session, change dns and/or ip etc , users use new server i think this is the most secured way, i did it like this the problem by redownload may stay cause the pop3 clients have their own list of mail downloaded etc build by i.e with some one hashes i.e. http://kb.mozillazine.org/Popstate.dat and i dont know some way to press them to download anything -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Thu Nov 24 17:37:53 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 17:37:53 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: On 24.11.2011, at 16.42, Simon Brereton wrote: >> Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > > Why do you need a home directory for virtual users? It says right there in the wiki. From joseba.torre at ehu.es Thu Nov 24 17:38:55 2011 From: joseba.torre at ehu.es (Joseba Torre) Date: Thu, 24 Nov 2011 16:38:55 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: <4ECE650F.2070700@ehu.es> El 24/11/11 15:42, Simon Brereton escribi?: > On Nov 24, 2011 4:22 AM, "Olli R?is?nen" wrote: >> >> Hello, >> >> I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. >> Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home >> directory should not be the same as mail directory. When I create new user >> accounts with PostfixAdmin the mail directory is not a problem but is > there >> some smart way to create the home directory as well (locating it for > instance >> in '/srv/vmail/%d/%n/home')? > > Why do you need a home directory for virtual users? For storing files wich are not mails. Sieve rules, subscription files and so on. Aaaaaaaaaaaagur. From mlists at edicom.eu Thu Nov 24 18:11:11 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 17:11:11 +0100 Subject: [Dovecot] Problem with lmtp proxy Message-ID: <201111241711.11504.mlists@edicom.eu> Hello everyone, I have set up postfix to deliver mails to dovecot (2.0.16) using LMTP. On the other hand, I've successfully configured the IMAP proxy setting in dovecot in order to be able to distribute mailboxes among different servers. I wanted to do the same proxy at LMTP level, but it's not working. If I put lmtp_proxy = no, then everything works ok (assuming the mailbox is local), but when I set lmtp_proxy = yes then the user is not found when deliverying the message via LMTP, so the mail remains in the postfix queue. Users are validated through active directory. However, this AD hasn't the SFU installed, thus its LDAP schema doesn't provide me with the required uid, gid, etc. To solve this, I have winbind configured in the system, so I'm doing this: - To validate users *and* be able to set the proxy extra fields, I use ldap as "passdb" (I believe it's not possible to use proxies with PAM). I'm using some LDAP field to store the host that has the user's mailbox. - In order to get the user account data that is not available in the AD, I use passwd as "userdb". I know that when using LMTP with proxy, a passdb needs to be configured. I assume this is needed for the service to lookup for the appropriate host to send the message. So, initially I set ldap as the passdb for LMTP, but just because it didn't work I put both ldap and passwd, and even a userdb (passwd), but the problems remain the same. Anyway the ldap should be the right one as it is there where I have the host information for each user. I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! This is my current config: ************* dovecot -n ************** # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37-gentoo-edicom-1104 x86_64 Gentoo Base System release 1.12.14 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_username_format = %n auth_verbose = yes base_dir = /var/run/dovecot/ listen = * lmtp_proxy = yes login_trusted_networks = 127.0.0.1 mail_debug = yes mail_location = maildir:~/maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at domain.com protocols = imap pop3 sieve lmtp quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = root mode = 0600 user = root } } service imap { vsz_limit = 512 M } service lmtp { inet_listener lmtp { address = 192.168.0.90 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } ssl_cert = , res_matched: <> dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_free_request (origid 1, msgid 1) dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_parse_result dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_msgfree dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_result ld 0x16518d0 msgid -1 dovecot [2011-11-24 15:53:50] [err] auth: Error: wait4msg ld 0x16518d0 msgid -1 (timeout 0 usec) dovecot [2011-11-24 15:53:50] [err] auth: Error: wait4msg continue ld 0x16518d0 msgid -1 all 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Connections: dovecot [2011-11-24 15:53:50] [err] auth: Error: * host: domain.com port: 389 (default) dovecot [2011-11-24 15:53:50] [err] auth: Error: refcnt: 1 status: Connected dovecot [2011-11-24 15:53:50] [err] auth: Error: last used: Thu Nov 24 15:53:50 2011 dovecot [2011-11-24 15:53:50] [err] auth: Error: dovecot [2011-11-24 15:53:50] [err] auth: Error: dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Outstanding Requests: dovecot [2011-11-24 15:53:50] [err] auth: Error: Empty dovecot [2011-11-24 15:53:50] [err] auth: Error: ld 0x16518d0 request count 0 (abandoned 0) dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Response Queue: dovecot [2011-11-24 15:53:50] [err] auth: Error: Empty dovecot [2011-11-24 15:53:50] [err] auth: Error: ld 0x16518d0 response count 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_chkResponseList ld 0x16518d0 msgid -1 all 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_chkResponseList returns ld 0x16518d0 NULL dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_int_select dovecot [2011-11-24 15:53:50] [info] lmtp(10506): Disconnect from local: Client quit (in reset) **************************************************** From tom at whyscream.net Thu Nov 24 18:13:47 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 24 Nov 2011 17:13:47 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111241122.21131.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: <4ECE6D3B.6020208@whyscream.net> On 24-11-11 10:22, Olli R?is?nen wrote: > Hello, > > I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. > Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > directory should not be the same as mail directory. When I create new user > accounts with PostfixAdmin the mail directory is not a problem but is there > some smart way to create the home directory as well (locating it for instance > in '/srv/vmail/%d/%n/home')? > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail as mailstore? Otherwise, postfixadmin has a post-create hook that is able to execute arbitrary scripts, creating homedirs should be among the included examples. -- Tom From mlists at edicom.eu Thu Nov 24 18:27:38 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 17:27:38 +0100 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241711.11504.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> Message-ID: <201111241727.38458.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 17:11:11 Miguel Tormo escribi?: > dn = cn=binduser,cn=Users,dc=edicom,dc=es Obviously this part should be "dn = cn=binduser,cn=Users,dc=domain,dc=com" to be coherent with the rest. I replace the real domain and user names used, but forgot this part :S. From ecasarero at gmail.com Thu Nov 24 18:30:03 2011 From: ecasarero at gmail.com (Eduardo Casarero) Date: Thu, 24 Nov 2011 13:30:03 -0300 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241727.38458.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> <201111241727.38458.mlists@edicom.eu> Message-ID: 2011/11/24 Miguel Tormo > El Jueves, 24 de Noviembre de 2011 17:11:11 Miguel Tormo escribi?: > > dn = cn=binduser,cn=Users,dc=edicom,dc=es > > Obviously this part should be "dn = cn=binduser,cn=Users,dc=domain,dc=com" > to be coherent with the rest. I replace the real domain and user names > used, but forgot this part :S. > > From tss at iki.fi Thu Nov 24 18:58:40 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 18:58:40 +0200 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241711.11504.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> Message-ID: <1322153921.24607.124.camel@hurina> On Thu, 2011-11-24 at 17:11 +0100, Miguel Tormo wrote: > I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! The problem is: > ************** cat dovecot-ldap.conf.ext ************** > auth_bind = yes auth_bind=yes requires a user authentication, but LMTP of course can't authenticate a user. So it fails: > dovecot [2011-11-24 15:53:50] [debug] auth: Debug: password(myuser): passdb doesn't support credential lookups Either try to get auth_bind=no working, or I think you can also set up a separate passdb for lmtp: protocol lmtp { passdb { driver = ldap args = some-other-ldap.conf } } And in this other ldap.conf have auth_bind=no, and possibly return password field always as something like "foo". From mlists at edicom.eu Thu Nov 24 19:26:07 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 18:26:07 +0100 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <1322153921.24607.124.camel@hurina> References: <201111241711.11504.mlists@edicom.eu> <1322153921.24607.124.camel@hurina> Message-ID: <201111241826.07847.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 17:58:40 Timo Sirainen escribi?: > On Thu, 2011-11-24 at 17:11 +0100, Miguel Tormo wrote: > > I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! > > The problem is: > > > ************** cat dovecot-ldap.conf.ext ************** > > auth_bind = yes > > auth_bind=yes requires a user authentication, but LMTP of course can't > authenticate a user. So it fails: > > > dovecot [2011-11-24 15:53:50] [debug] auth: Debug: password(myuser): passdb doesn't support credential lookups > > Either try to get auth_bind=no working, or I think you can also set up a > separate passdb for lmtp: > > protocol lmtp { > passdb { > driver = ldap > args = some-other-ldap.conf > } > } > > And in this other ldap.conf have auth_bind=no, and possibly return > password field always as something like "foo". > I just configured your suggestion (set a separate passdb for lmtp with ldap and auth_bind = no ) and it works perfectly. Thank you very much! From lists at luigirosa.com Thu Nov 24 20:04:12 2011 From: lists at luigirosa.com (Luigi Rosa) Date: Thu, 24 Nov 2011 19:04:12 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322095616.24607.108.camel@hurina> References: <1322095616.24607.108.camel@hurina> Message-ID: <4ECE871C.3000804@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen said the following on 24/11/11 01:46: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz Works as advertised. I noticed this warining: Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < 4096), because of service auth { client_limit } What does exactly means? The server we are talking about is my home server, a CentOS/64 with a dozen of mailboxes. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Computers make excellent and efficient servants, but I have no wish to serve under them. Captain, a starship also runs on loyalty to one man. And nothing can replace it or him. --Spock, "The Ultimate Computer" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7OhxwACgkQ3kWu7Tfl6ZSJ0gCeIaS5ksyVHY3O9qOSCEyWUH2V By8AoMbUNSHBBc4102vScrozTOOkOeo+ =AZaL -----END PGP SIGNATURE----- From mlists at edicom.eu Thu Nov 24 20:23:00 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 19:23:00 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <4ECE871C.3000804@luigirosa.com> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> Message-ID: <201111241923.00215.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 19:04:12 Luigi Rosa escribi?: > Timo Sirainen said the following on 24/11/11 01:46: > > > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz > > Works as advertised. > > I noticed this warining: > > Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < > 4096), because of service auth { client_limit } > > What does exactly means? > > > The server we are talking about is my home server, a CentOS/64 with a dozen of > mailboxes. > The fd limit is the maximum number of concurrently open file descriptors per process. It can be configured on a per-user basis. You can check the actual value running 'ulimit -n'. It can be increased up to the value set in /proc/sys/fs/file-max. You can set it editing /etc/security/limits.conf. What the above comment means is that under max load the auth service could need up to 4096 file descriptors open, that's more than the default in most linux systems (1024). However, in your case with just a dozen of mailboxes I don't think you'll need to change it. From lists at luigirosa.com Thu Nov 24 20:29:41 2011 From: lists at luigirosa.com (Luigi Rosa) Date: Thu, 24 Nov 2011 19:29:41 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111241923.00215.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> Message-ID: <4ECE8D15.6050200@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Miguel Tormo said the following on 24/11/11 19:23: > The fd limit is the maximum number of concurrently open file descriptors > per process. It can be configured on a per-user basis. You can check the > actual value running 'ulimit -n'. It can be increased up to the value set > in /proc/sys/fs/file-max. You can set it editing > /etc/security/limits.conf. What the above comment means is that under max > load the auth service could need up to 4096 file descriptors open, that's > more than the default in most linux systems (1024). > > However, in your case with just a dozen of mailboxes I don't think you'll > need to change it. Thank you for the explaination. Ciao, luigi - -- / +--[Luigi Rosa]-- \ A woman was in love with fourteen soldiers. It was clearly platoonic. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7OjRUACgkQ3kWu7Tfl6ZQXAwCgyD6LAUYQX7/x1PIFda5CoFjx IYsAniqFYvdk6y982XjvWsWc4G5t9zEC =70Em -----END PGP SIGNATURE----- From olli.raisanen at telemail.fi Thu Nov 24 21:23:33 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-1?q?R=E4is=E4nen?=) Date: Thu, 24 Nov 2011 21:23:33 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <4ECE6D3B.6020208@whyscream.net> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> Message-ID: <201111242123.34156.olli.raisanen@telemail.fi> Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: > > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail > as mailstore? Well, yes - at least the home directory will be created that way. Still, PostfixAdmin will not store the path in the maibox table home field. Perhaps it is enough to include the path in the Dovecot user_query? > Otherwise, postfixadmin has a post-create hook that is able to execute > arbitrary scripts, creating homedirs should be among the included > examples. If you are referring to maildir_name_hook in config.inc.php script, no, homedirs are not included (in Postfix Admin version 2.3.4). Thank you anyway, Tom! I must consider your advice. Olli From tss at iki.fi Thu Nov 24 21:25:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 21:25:47 +0200 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111241923.00215.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> Message-ID: <1322162747.24607.129.camel@hurina> On Thu, 2011-11-24 at 19:23 +0100, Miguel Tormo wrote: > The fd limit is the maximum number of concurrently open file descriptors per process. It can be configured on a per-user basis. > You can check the actual value running 'ulimit -n'. It can be increased up to the value set in /proc/sys/fs/file-max. You can set it editing /etc/security/limits.conf. I'm not sure if changing /etc/security/limits.conf helps. It's probably only used by PAM when user logs in, so if Dovecot is started in system bootup it's unlikely to have been even read yet. Also I think some OSes override the limits in /etc/init.d/ scripts. Of course, I could be completely wrong in all of the above, I haven't really tested any of it :) > What the above comment means is that under max load the auth service could need up to 4096 file descriptors open, that's more than the default in most linux systems (1024). I wonder if there's a good reason for why auth default should be that high. Probably in earlier times imap/pop3 processes just weren't disconnecting early enough from auth. So I guess I'll just drop it back to default and this warning would go away in most systems. I had started to get a little bit annoyed at that warning myself also. From tss at iki.fi Thu Nov 24 21:29:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 21:29:46 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111242123.34156.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> <201111242123.34156.olli.raisanen@telemail.fi> Message-ID: <1322162990.24607.131.camel@hurina> On Thu, 2011-11-24 at 21:23 +0200, Olli R?is?nen wrote: > Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: > > > > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail > > as mailstore? > > Well, yes - at least the home directory will be created that way. Still, > PostfixAdmin will not store the path in the maibox table home field. Perhaps > it is enough to include the path in the Dovecot user_query? It would be better if you used PostfixAdmin's "maildir" field as the home directory, and then mail_location=~/Maildir. Like the sql queries being: password_query = \ SELECT username AS user, password, \ CONCAT('/home/vmail/', maildir) AS userdb_home, \ CONCAT('*:bytes=', quota) AS userdb_quota_rule \ FROM mailbox \ WHERE username = '%u' and ACTIVE = '1' user_query = \ SELECT CONCAT('/home/vmail/', maildir) AS home, \ CONCAT('*:bytes=', quota) AS quota_rule \ FROM mailbox \ WHERE username = '%u' AND active = '1' I used these in another PostfixAdmin server's install. From tom at whyscream.net Thu Nov 24 21:50:39 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 24 Nov 2011 20:50:39 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <1322162990.24607.131.camel@hurina> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> <201111242123.34156.olli.raisanen@telemail.fi> <1322162990.24607.131.camel@hurina> Message-ID: <4ECEA00F.9040306@whyscream.net> On 24-11-11 20:29, Timo Sirainen wrote: > On Thu, 2011-11-24 at 21:23 +0200, Olli R?is?nen wrote: >> Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: >>> >>> How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail >>> as mailstore? >> >> Well, yes - at least the home directory will be created that way. Still, >> PostfixAdmin will not store the path in the maibox table home field. Perhaps >> it is enough to include the path in the Dovecot user_query? > > It would be better if you used PostfixAdmin's "maildir" field as the > home directory, and then mail_location=~/Maildir. Like the sql queries > being: > > password_query = \ > SELECT username AS user, password, \ > CONCAT('/home/vmail/', maildir) AS userdb_home, \ > CONCAT('*:bytes=', quota) AS userdb_quota_rule \ > FROM mailbox \ > WHERE username = '%u' and ACTIVE = '1' > > user_query = \ > SELECT CONCAT('/home/vmail/', maildir) AS home, \ > CONCAT('*:bytes=', quota) AS quota_rule \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' > > I used these in another PostfixAdmin server's install. > I even simplified this up to: # taken from stock dovecot2 config userdb { driver = static args = uid=vmail gid=mail home=/var/spool/vmail/%u } mail_location = maildir:~/Maildir password_query = SELECT username AS user, password FROM mailbox WHERE username = '%u' AND active = '1' (no quota's, no special cases for home/mail directory configuration) -- Tom From mlists at edicom.eu Thu Nov 24 22:35:09 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 21:35:09 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322162747.24607.129.camel@hurina> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> Message-ID: <201111242135.09681.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: > I'm not sure if changing /etc/security/limits.conf helps. It's probably > only used by PAM when user logs in, so if Dovecot is started in system > bootup it's unlikely to have been even read yet. Also I think some OSes > override the limits in /etc/init.d/ scripts. Of course, I could be > completely wrong in all of the above, I haven't really tested any of > it Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). Another permanent way between reboots that I just googled is changing this line of includes/linux/fs.h: #define INR_OPEN 1024 /* Initial setting for nfile rlimits */ and recompile the kernel :P. From noel.butler at ausics.net Fri Nov 25 00:30:18 2011 From: noel.butler at ausics.net (Noel Butler) Date: Fri, 25 Nov 2011 08:30:18 +1000 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111242135.09681.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> <201111242135.09681.mlists@edicom.eu> Message-ID: <1322173818.6600.5.camel@tardis> On Thu, 2011-11-24 at 21:35 +0100, Miguel Tormo wrote: > El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: > > I'm not sure if changing /etc/security/limits.conf helps. It's probably > > only used by PAM when user logs in, so if Dovecot is started in system > > bootup it's unlikely to have been even read yet. Also I think some OSes > > override the limits in /etc/init.d/ scripts. Of course, I could be > > completely wrong in all of the above, I haven't really tested any of > > it > > Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). > much simpler way without rebuilding is as Timo suggested, just pick your value #!/bin/sh # Start/stop dovecot. ulimit -n 16384 ... Be careful, building in kernel based higher limits means /every/ service and can lead to DoS, this way limits dovecot to 16384, but keeps other stuff like mysql at 4096, and everything else at 1024, bit safer. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From olli.raisanen at telemail.fi Fri Nov 25 00:43:51 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-15?q?R=E4is=E4nen?=) Date: Fri, 25 Nov 2011 00:43:51 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <1322162990.24607.131.camel@hurina> References: <201111241122.21131.olli.raisanen@telemail.fi> <201111242123.34156.olli.raisanen@telemail.fi> <1322162990.24607.131.camel@hurina> Message-ID: <201111250043.51861.olli.raisanen@telemail.fi> Timo Sirainen kirjoitti 2011-11-24 kello 21:29:46: > > It would be better if you used PostfixAdmin's "maildir" field as the > home directory, and then mail_location=~/Maildir. Thank you, Timo! (And thank you, Tom, for the simplification example!) I never thought ~/Maildir could refer to anything but UNIX system users! :) Timo's queries (beneath) work indeed. However, it took quite a while of testing, and I wonder if it really would cause some trouble if homedir and maildir were kept the same... Regards, Olli > Like the sql queries being: > > password_query = \ > SELECT username AS user, password, \ > CONCAT('/home/vmail/', maildir) AS userdb_home, \ > CONCAT('*:bytes=', quota) AS userdb_quota_rule \ > FROM mailbox \ > WHERE username = '%u' and ACTIVE = '1' > > user_query = \ > SELECT CONCAT('/home/vmail/', maildir) AS home, \ > CONCAT('*:bytes=', quota) AS quota_rule \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' > > I used these in another PostfixAdmin server's install. From pw at wk-serv.de Fri Nov 25 00:58:01 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 23:58:01 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111124143321.GB28600@dibs.tanso.net> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> Message-ID: <4ECECBF9.5070507@wk-serv.de> Jan-Frode Myklebust schrieb: > We use a clusterfs (so it's not very important which backend server the > user is directed to), and have configured the dovecot director with: > > director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 > director_servers = 192.168.42.15 192.168.42.17 > passdb { > args = proxy=y nopassword=y > driver = static > } > > which gives us loadbalancing of new logins, and also failover should a > backend server fail. That looks nice. What is your frontend configuration? How many users do you have that you need that many backend servers? Are your index stored together with the mails? Patrick From janfrode at tanso.net Fri Nov 25 10:17:15 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 25 Nov 2011 09:17:15 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <4ECECBF9.5070507@wk-serv.de> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> Message-ID: <20111125081715.GA24352@dibs.tanso.net> On Thu, Nov 24, 2011 at 11:58:01PM +0100, Patrick Westenberg wrote: > > What is your frontend configuration? The frontend is two servers running dovecot director, and ucarp for managing failover ip-addresses. ===================================================================== # 2.0.14: /etc/dovecot/dovecot.conf # OS: x86_64 Red Hat Enterprise Linux Server release 5.x director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * lmtp_proxy = yes passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 2051 } service auth { client_limit = 4196 unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 1024 } service lmtp { inet_listener lmtp { address = * port = 24 } } service pop3-login { executable = pop3-login director process_limit = 1024 } ssl_cert = How many users do you have that you need that many backend servers? I don't think I'm allowed to share that externally, sorry. This is the mail-system of an ISP, so we have quite a few users.. :-) > Are your index stored together with the mails? They're not stored together, because we made the error of not creating a home directory for our users, but both maildirs and indexes are stored on similar cluster fs' (GPFS). -jf From pw at wk-serv.de Fri Nov 25 11:02:19 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Fri, 25 Nov 2011 10:02:19 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111125081715.GA24352@dibs.tanso.net> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> <20111125081715.GA24352@dibs.tanso.net> Message-ID: <19dc5fa7b626f394a4e54fb0105ac5e8@wk-serv.de> On Fri, 25 Nov 2011 09:17:15 +0100, Jan-Frode Myklebust wrote: > On Thu, Nov 24, 2011 at 11:58:01PM +0100, Patrick Westenberg wrote: >> >> What is your frontend configuration? > > The frontend is two servers running dovecot director, and ucarp for > managing failover ip-addresses. Two additional directors or the two directors from your posted config (.15/.17)? Will I have to configure external IPs for the backend servers or just internal for the connections to and from the directors? I'm not sure if the director "redirects" the connections to the backend servers or if they pass through on the internal network. >> How many users do you have that you need that many backend servers? > > I don't think I'm allowed to share that externally, sorry. This is > the > mail-system of an ISP, so we have quite a few users.. :-) Ok :-) From joseba.torre at ehu.es Fri Nov 25 12:48:14 2011 From: joseba.torre at ehu.es (Joseba Torre) Date: Fri, 25 Nov 2011 11:48:14 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111242135.09681.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> <201111242135.09681.mlists@edicom.eu> Message-ID: <4ECF726E.6020109@ehu.es> El 24/11/11 21:35, Miguel Tormo escribi?: > El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: >> I'm not sure if changing /etc/security/limits.conf helps. It's probably >> only used by PAM when user logs in, so if Dovecot is started in system >> bootup it's unlikely to have been even read yet. Also I think some OSes >> override the limits in /etc/init.d/ scripts. Of course, I could be >> completely wrong in all of the above, I haven't really tested any of >> it > > Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). I tested this "intensively" some time ago, and Timo is right -as usual :) -: modifications in limits.conf are only relevant after a login. This means that they are not enforced at system boot, but they are if you reboot the service by hand (because you have already logged in). The permanent solution is, as Noel Butler has pointed, modifying the startup script and using ulimit before actually starting dovecot. HTH From janfrode at tanso.net Fri Nov 25 13:38:13 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 25 Nov 2011 12:38:13 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322162747.24607.129.camel@hurina> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> Message-ID: <20111125113813.GA27970@dibs.tanso.net> On Thu, Nov 24, 2011 at 09:25:47PM +0200, Timo Sirainen wrote: > I'm not sure if changing /etc/security/limits.conf helps. It's probably > only used by PAM when user logs in, so if Dovecot is started in system > bootup it's unlikely to have been even read yet. Also I think some OSes > override the limits in /etc/init.d/ scripts. Of course, I could be > completely wrong in all of the above, I haven't really tested any of > it :) The OP was running Centos, and the Red Hat way of increasing nofile for services is to add the ulimit setting to /etc/sysconfig/dovecot: ulimit -n 32768 This is what we use, and it survives dovecot upgrades (since this is tagged as a configfile in RPM, not as script that will be overwritten on upgrades). -jf From ef at math.uni-bonn.de Fri Nov 25 14:49:03 2011 From: ef at math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) Date: Fri, 25 Nov 2011 13:49:03 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <1320438508.21919.177.camel@hurina> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> Message-ID: <20111125124902.GB27737@gumme.math.uni-bonn.de> > That shouldn't happen because of Dovecot's indexes. Hm, also if the indexes are local? Fine. > Then the 'S' flag is added to the current Maildir filename without > losing any other changes. And this is supposed to work even over NFS? Great. So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? From sf.rique at gmail.com Fri Nov 25 17:29:35 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Fri, 25 Nov 2011 13:29:35 -0200 Subject: [Dovecot] Which FTS to use! Message-ID: Hello, i am currently using squat dovecot, but some users that have lots os emails ( 1GB ) are complaining about speed. Iam using dovecot 2.0.13 on debian 6 my squat conf is plugin { fts = squat fts_squat = partial=4 full=5 } I am thinking in maybe start to use Solr, so any one can help me with an decision? I am probably gonna update dovecot to 2.1 next year. and them maybe change the FTS plugin. This year i will get 10k disk only to store the dovecot index, rigth now they are saved on the same storage that the mails are and also change the "full" parameter to something like 10 or 15 I know i will use more disk space for it, but it will make a lot of differences right? it will match the message without the need of opening it for any search smaller than 10 character i guess. But this will be enough? or Solr or Lucene still better? Anyway, i am not current updating the indexes automatic, but i am planing on it, just not sure what is better, the command doveadm index or doveadm search... with one is better to do the task? Thanks! []'sf.rique From tss at iki.fi Fri Nov 25 18:47:51 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 25 Nov 2011 18:47:51 +0200 Subject: [Dovecot] Which FTS to use! In-Reply-To: References: Message-ID: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> On 25.11.2011, at 17.29, Henrique Santos Fernandes wrote: > Hello, i am currently using squat dovecot, but some users that have lots os > emails ( 1GB ) are complaining about speed. > Iam using dovecot 2.0.13 on debian 6 > > my squat conf is > plugin { > fts = squat > fts_squat = partial=4 full=5 > } > > I am thinking in maybe start to use Solr, so any one can help me with an > decision? With v2.0 Solr is the only other choice. > I am probably gonna update dovecot to 2.1 next year. and them maybe change > the FTS plugin. With v2.1 Lucene is another possibility, and it's somewhat easier to install than Solr. Also v2.1's Solr backend is improved, so I wouldn't really recommend spending a lot of time and disk I/O on v2.0's Solr since you'll need to rebuild the index anyway in v2.1 (or keep using "solr_old" backend with some missing featues). > This year i will get 10k disk only to store the dovecot index, rigth now > they are saved on the same storage that the mails are and also change the > "full" parameter to something like 10 or 15 I know i will use more disk > space for it, but it will make a lot of differences right? it will match > the message without the need of opening it for any search smaller than 10 > character i guess. > > > But this will be enough? or Solr or Lucene still better? The main problem with Squat is its index updating performance, which sucks. > Anyway, i am not current updating the indexes automatic, but i am planing > on it, just not sure what is better, the command doveadm index or doveadm > search... with one is better to do the task? doveadm index From lists at wildgooses.com Fri Nov 25 18:59:22 2011 From: lists at wildgooses.com (Ed W) Date: Fri, 25 Nov 2011 16:59:22 +0000 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <4ECFC96A.9020204@wildgooses.com> On 25/11/2011 12:49, Edgar Fu? wrote: >> That shouldn't happen because of Dovecot's indexes. > Hm, also if the indexes are local? Fine. > >> Then the 'S' flag is added to the current Maildir filename without >> losing any other changes. > And this is supposed to work even over NFS? Great. > > So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? You don't have any Thunderbird clients accessing this box do you? I have some wierd issue where our TB with the option "don't mark message read", still triggers messages to be marked read... Wierdly it only does it on some messages and all those from specific senders - can't correlate it with anything obvious in the message itself though... Regards Ed W From ef at math.uni-bonn.de Fri Nov 25 19:20:55 2011 From: ef at math.uni-bonn.de (=?iso-8859-1?Q?Edgar_Fu=DF?=) Date: Fri, 25 Nov 2011 18:20:55 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <4ECFC96A.9020204@wildgooses.com> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> <4ECFC96A.9020204@wildgooses.com> Message-ID: <476867D4-69AE-4854-96F8-43C4559DC51F@math.uni-bonn.de> > You don't have any Thunderbird clients accessing this box do you? Yes, I have. But I also experienced the problem myself only using mutt and Apple Mail. > I have some w[ei]rd issue where our TB with the option > "don't mark message read", still triggers messages to be marked read It's the other way round. People read their messages, they are flagged as read and all of a sudden revert to unread. From e-frog at gmx.de Fri Nov 25 19:23:59 2011 From: e-frog at gmx.de (e-frog) Date: Fri, 25 Nov 2011 18:23:59 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <4ECFC96A.9020204@wildgooses.com> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> <4ECFC96A.9020204@wildgooses.com> Message-ID: <4ECFCF2F.50307@gmx.de> On 25.11.2011 17:59, wrote Ed W: > On 25/11/2011 12:49, Edgar Fu? wrote: > You don't have any Thunderbird clients accessing this box do you? I > have some wierd issue where our TB with the option "don't mark message > read", still triggers messages to be marked read... Wierdly it only does > it on some messages and all those from specific senders - can't > correlate it with anything obvious in the message itself though... I had the same issue a while ago with Thunderbird. On one installation it worked on another it didn't. After comparing their config I found the culprit. Changing mail.server.default.mime_parts_on_demand to **false** fixed this issue for me. This setting is discussed in... http://kb.mozillazine.org/Entire_message_fetched_when_opening_a_IMAP_message ... but not mentioning the "don't mark message as read" issue. Regards, e-frog * * Englisch * Deutsch * Englisch * Deutsch From tss at iki.fi Fri Nov 25 19:47:22 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 25 Nov 2011 19:47:22 +0200 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <5539B894-4826-480F-95F6-A7D8CBF9B147@iki.fi> On 25.11.2011, at 14.49, Edgar Fu? wrote: >> That shouldn't happen because of Dovecot's indexes. > Hm, also if the indexes are local? Fine. Yep. It just means that indexes then aren't fully up to date. Hmm. I guess possibly one potential problem is if 1. dovecot.index.log says that "set flags to ()" but then it doesn't actually finish that 2. Next time indexes are accessed from second server, and flags are set to (\Seen) 3. And then finally user logs in back to original server, which sees that "aha, flags should be set to ()" and does that. But the problem is the first step. It shouldn't normally do it only halfway through, unless it crashes or something.. Also typically flags aren't replaced, they're added/removed, but this completely depends on the client and I don't remember what Apple Mail does. >> Then the 'S' flag is added to the current Maildir filename without >> losing any other changes. > And this is supposed to work even over NFS? Great. > > So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? You could try enabling mail_log plugin with flag_change event to see when it actually changes. http://wiki.dovecot.org/Plugins/MailLog From noel.butler at ausics.net Sat Nov 26 02:19:13 2011 From: noel.butler at ausics.net (Noel Butler) Date: Sat, 26 Nov 2011 10:19:13 +1000 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <1322266753.6288.6.camel@tardis> On Fri, 2011-11-25 at 13:49 +0100, Edgar Fu? wrote: > > Then the 'S' flag is added to the current Maildir filename without > > losing any other changes. > And this is supposed to work even over NFS? Great. Absolutely, always has worked over NFS -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From rajeev at teamsg.in Sat Nov 26 08:32:48 2011 From: rajeev at teamsg.in (Rajeev Sharma) Date: Sat, 26 Nov 2011 12:02:48 +0530 Subject: [Dovecot] Dovecot 1.2 ACL problem. Message-ID: Sir, I have migrate from courier-imap to dovecot without any problem but on courier in have shared folder which access on the basis on rights. I have manage these folders in dovecot but i have one problem all the shared folder is accessible by all user whether they have rights or not i have place dovecot-acl in shared folder and assign user my dovecot-acl is user=rajeev at teamsg.in tr. But all other user which is not listing on dovecot-acl also able to access these folder. Here is the output of dovecot -n. # 1.2.17: /usr/local/etc/dovecot.conf # OS: Linux 2.6.18-194.3.1.el5 i686 CentOS release 5.5 (Final) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log ssl_listen: 192.168.0.251:993 ssl_cert_file: /var/qmail/control/servercert.pem ssl_key_file: /var/qmail/control/servercert.pem login_dir: /var/run/dovecot//login login_executable: /usr/local/libexec/dovecot/imap-login first_valid_uid: 508 last_valid_uid: 508 mail_location: maildir:~/Maildir namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes namespace: type: shared separator: . prefix: INBOX.Shared. location: maildir:/home/vpopmail/domains/sgcricket.com/Maildir/Out:INDEX=~/Maildir/Shared list: children auth default: user: vpopmail passdb: driver: vpopmail userdb: driver: vpopmail plugin: acl: vfile Please help whats wrong i am doing. Rajeev Sharma. From dan.colascione at gmail.com Sat Nov 26 08:36:39 2011 From: dan.colascione at gmail.com (Daniel Colascione) Date: Fri, 25 Nov 2011 22:36:39 -0800 Subject: [Dovecot] Assertion failure in 2.0.16 - SEARCH () Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nov 25 22:25:39 dancol dovecot: master: Error: service(imap): child 25115 killed with signal 6 (core dumps disabled) Nov 25 22:25:59 dancol dovecot: imap(dancol): Panic: file mail-search-build.c: line 59 (mail_search_build_key_int): assertion failed: (sarg->value.subargs != NULL) Nov 25 22:25:59 dancol dovecot: imap(dancol): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3d291) [0xb7636291] -> /usr/lib/dovecot/libdovecot.so.0(+0x3d2ff) [0xb76362ff] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0xb760bb7a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x24109) [0xb7692109] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x24177) [0xb7692177] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_build+0x82) [0xb7692242] - -> dovecot/imap [dancol 127.0.0.1 SEARCH](imap_search_args_build+0x4a) [0x805db5a] -> dovecot/imap [dancol 127.0.0.1 SEARCH](cmd_search+0xbf) [0x80537ef] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x80568cc] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x80569a2] -> dovecot/imap [dancol 127.0.0.1 SEARCH](client_handle_input+0x175) [0x8056c25] -> dovecot/imap [dancol 127.0.0.1 SEARCH](client_input+0x5f) [0x80574cf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x51) [0xb76430d1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0xb7644193] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0xb7643050] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2a) [0xb762ec1a] -> dovecot/imap [dancol 127.0.0.1 SEARCH](main+0x315) [0x805fef5] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb74b6c76] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x804eb81] $ telnet localhost imap Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. . LOGIN dancol XXXXXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in . SELECT rss * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8 EXISTS * 0 RECENT * OK [UIDVALIDITY 1322264663] UIDs valid * OK [UIDNEXT 11] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. . SEARCH () Connection closed by foreign host. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk7QiNcACgkQ17c2LVA10VvhHACgihmbmTjVNl/PbKcreIDRTKpC 5wkAoJd9FHXM4GhPfnVUjwrpdsqnVQ6E =RlpQ -----END PGP SIGNATURE----- From mark.zealey at webfusion.com Sat Nov 26 18:33:30 2011 From: mark.zealey at webfusion.com (Mark Zealey) Date: Sat, 26 Nov 2011 18:33:30 +0200 Subject: [Dovecot] using dsync to convert mailboxes looses caching options Message-ID: <4ED114DA.7030707@webfusion.com> Hi there, We're trying to convert users from Maildir to sdbox at present; I'm using dsync to achieve this (2.0.16) however when the user's have been converted we only get minimal information in the caching files. Is there some way to preserve all the caching decisions that were previously made so that when the user logs in to the new mailbox we don't have to cause an io storm rebuilding the cache that we know was good? Dovecot seems to be partially doing this - if i remove the logs/cache from the source mailbox no cache files are built in the conversion; if i put them back then we get a cache file built but it only contains a few bits of information (guid, date.save). Looking into this a bit further i find that when the caches are present at source the fields are preserved but the 'last used' date and caching decisions are not which I suspect means dsync doesn't bother caching on import - only fields with a yes decision in the source are copied (but their decision is only copied as a tmp with the date of import). For example: Source idxview cache: -- Cache fields -- # Name Type Size Dec Last used 0: flags bit 4 tmp 2011-11-25 16:09 1: date.received fix 4 yes 2011-11-26 16:17 2: size.virtual fix 8 tmp 2011-11-25 16:09 3: imap.bodystructure str - tmp 2011-11-25 16:09 4: mime.parts var - tmp 2011-11-25 16:09 5: hdr.IMPORTANCE hdr - tmp 2011-11-25 16:09 6: hdr.X-PRIORITY hdr - tmp 2011-11-25 16:09 7: hdr.CONTENT-TYPE hdr - tmp 2011-11-25 16:09 ... 18: date.sent fix 8 no 1970-01-01 01:00 19: date.save fix 4 yes 2011-11-26 16:17 20: size.physical fix 8 no 1970-01-01 01:00 21: imap.body str - no 1970-01-01 01:00 ... (24 total entries. stripped ones are just hdr. with a tmp decision) dst cache: # Name Type Size Dec Last used 0: flags bit 4 tmp 1970-01-01 01:00 1: date.sent fix 8 no 1970-01-01 01:00 2: date.received fix 4 tmp 1970-01-01 01:00 3: date.save fix 4 tmp 2011-11-26 16:19 4: size.virtual fix 8 tmp 1970-01-01 01:00 5: size.physical fix 8 no 1970-01-01 01:00 6: imap.body str - no 1970-01-01 01:00 7: imap.bodystructure str - tmp 1970-01-01 01:00 8: imap.envelope str - no! 1970-01-01 01:00 9: pop3.uidl str - no 1970-01-01 01:00 10: guid str - tmp 2011-11-26 16:19 11: mime.parts var - tmp 1970-01-01 01:00 12: hdr.IMPORTANCE hdr - tmp 1970-01-01 01:00 13: hdr.X-PRIORITY hdr - tmp 1970-01-01 01:00 ... and dst cached message for example: RECORD: seq=57, uid=12207, flags=0x09 (Seen Answered) - ext 1 modseq : 1 (0100000000000000) - ext 3 cache : 5516 (8c150000) - cache offset=5516 size=92, prev_offset = 0 - guid: 1321895282.XXXt,S=7399760,W=7496225 - date.save: 1322324286 (3e11d14e) src idx record for this message contains pretty much all fields. Thanks, Mark From sf.rique at gmail.com Sun Nov 27 15:17:16 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Sun, 27 Nov 2011 11:17:16 -0200 Subject: [Dovecot] Which FTS to use! In-Reply-To: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> References: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> Message-ID: So i guess i will just stay with squat until i upgrade to 2.1 and then move to solr.. And make the cron update daily for active users! Thanks a lot! []'sf.rique On Fri, Nov 25, 2011 at 2:47 PM, Timo Sirainen wrote: > On 25.11.2011, at 17.29, Henrique Santos Fernandes wrote: > > > Hello, i am currently using squat dovecot, but some users that have lots > os > > emails ( 1GB ) are complaining about speed. > > Iam using dovecot 2.0.13 on debian 6 > > > > my squat conf is > > plugin { > > fts = squat > > fts_squat = partial=4 full=5 > > } > > > > I am thinking in maybe start to use Solr, so any one can help me with an > > decision? > > With v2.0 Solr is the only other choice. > > > I am probably gonna update dovecot to 2.1 next year. and them maybe > change > > the FTS plugin. > > With v2.1 Lucene is another possibility, and it's somewhat easier to > install than Solr. Also v2.1's Solr backend is improved, so I wouldn't > really recommend spending a lot of time and disk I/O on v2.0's Solr since > you'll need to rebuild the index anyway in v2.1 (or keep using "solr_old" > backend with some missing featues). > > > This year i will get 10k disk only to store the dovecot index, rigth now > > they are saved on the same storage that the mails are and also change > the > > "full" parameter to something like 10 or 15 I know i will use more disk > > space for it, but it will make a lot of differences right? it will match > > the message without the need of opening it for any search smaller than > 10 > > character i guess. > > > > > > But this will be enough? or Solr or Lucene still better? > > The main problem with Squat is its index updating performance, which sucks. > > > Anyway, i am not current updating the indexes automatic, but i am planing > > on it, just not sure what is better, the command doveadm index or doveadm > > search... with one is better to do the task? > > doveadm index From Patrick.Wood at tups.net.au Mon Nov 28 05:24:54 2011 From: Patrick.Wood at tups.net.au (Patrick Wood) Date: Mon, 28 Nov 2011 03:24:54 +0000 Subject: [Dovecot] POP3/IMAP crash signal 10 Message-ID: <0D810DB78132A8488DE17E655BF29B685C5EAA5C@TUPSDC01.tups.net.au> Hi, I'm building a Postfix/Dovecot mail server and while I am able to send/receive emails using telnet, after establishing a connection to Dovecot via a client (Mail Live, Thunderbird etc) the following appears in the logs: Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.info] pop3-login: Login: user=, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, mpid=282 78 Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.error] master: Error: service(pop3-login): child 28277 killed with signal 10 (core not dumped - set service pop3-log in { drop_priv_before_exec=yes }) Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.info] pop3(user at domain.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/5, size=3250 I also experience very poor connectivity to the server, with Microsoft Mail and Thunderbird regularly failing to connect (connection errors in the client and no login attempt in the dovecot logs). I assume this is related to the issue above. I am running dovecot 2.0.16 on Solaris 10 SPARC using a ZFS filesystem for mailbox storage. Here is my configuration: # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4u zfs auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:/storage0/home/vmail/%d/%n/Maildir mail_plugins = quota passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=1G } postmaster_address = root at domain.com sendmail_path = /usr/lib/sendmail ssl_cert = Hello, I tried to upgrade from 2.0.15 to 2.1.rc1 over the weekend but ran into some problems with squat. My two largest mailboxes were corrupted, producing this error: Nov 27 17:38:12 kerio1 dovecot[42860]: imap(bigmailbox): Error: Corrupted squat uidlist file /var/mail/metro-email.com/bigmailbox/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.search.uids: wrong indexid There is a good chance that I had shot my own foot, by forgetting to update postfix's dovecot-lda reference to the new instance. I had also enabled an inet based LMTP listener, although it is not being used yet. Even using non-corrupted indexes, it seemed that FTS was not being used. A basic search such as this one took many minutes to complete: doveadm search -u jsample mailbox INBOX body eps I found messages like this in mail.log after performing the search: Nov 27 18:21:39 kerio1 dovecot[42860]: indexer-worker(jsample): Indexed 0 messages in Drafts Nov 27 18:21:39 kerio1 dovecot[42860]: indexer-worker(jsample): Indexed 0 messages in INBOX When I rolled back to 2.0.15, the same search returned almost immediately. The larger mailboxes went quickly again, but only after they had a chance to rebuild. A few other notes about our configuration: We are on PPC, with the stats service disabled (because of kqueue issue discussed earlier). I made some minor changes in my config since the previous installed version (diff below). mail_location override is in place for all users via password file, e.g.: userdb_mail=mdbox:/var/mail/metro-email.com/jsample/mdbox Thanks, Tony kerio1:~ metroadmin$ /opt/dovecot-2.1.rc1/bin/doveconf -n -c /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf # 2.1.rc1: /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_mechanisms = plain login lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_proxy = yes mail_location = maildir:%h mail_plugins = fts fts_squat mail_log notify mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader mdbox_rotate_size = 10 M namespace { inbox = yes location = prefix = separator = / } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users driver = passwd-file } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from subject sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at metro-email.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = metroadmin } } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 192.168.20.10 127.0.0.1 port = 24 } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_cert = # 2.1.rc1: /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf 9a10 > lmtp_proxy = yes 15c16 < managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave --- > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader 24c25 < args = scheme=CRYPT username_format=%u /opt/dovecot-2.0.15/etc/dovecot/users --- > args = scheme=CRYPT username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users 50a52,57 > service lmtp { > inet_listener lmtp { > address = 192.168.20.10 127.0.0.1 > port = 24 > } > } 53a61,65 > service stats { > fifo_listener stats-mail { > mode = 00 > } > } 57c69 < args = username_format=%u /opt/dovecot-2.0.15/etc/dovecot/users --- > args = username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users kerio1:~ metroadmin$ From msbrogli-dovecot at vialink.com.br Tue Nov 29 00:14:19 2011 From: msbrogli-dovecot at vialink.com.br (Marcelo Salhab Brogliato) Date: Mon, 28 Nov 2011 20:14:19 -0200 Subject: [Dovecot] Fault tolerant architecture Message-ID: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Hi, I'm new to this list and want your help. I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). I'm using dovecot 1.2.9. I guess you already had some of these problems or maybe in another architecture these neither exists. Thanks for any help, Marcelo From list at airstreamcomm.net Tue Nov 29 01:16:37 2011 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 28 Nov 2011 17:16:37 -0600 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <8c0fae947f9c32073f798299f027684c@mail.airstreamcomm.net> On Mon, 28 Nov 2011 20:14:19 -0200, Marcelo Salhab Brogliato wrote: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we > have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. > Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS > using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is > using two IP addresses in our DNS records. > > About NFS using local indexes, I'm configuring a test server. But how to > configure local indexes when my mail_location comes from sql (userdb_home > actually). > I'm using dovecot 1.2.9. > > I guess you already had some of these problems or maybe in another > architecture these neither exists. > > Thanks for any help, > > Marcelo Marcelo, There are a number of ways to bring HA to a cluster of mail servers, one that we have experimented with lately is a bit exotic, but might work for you. >From the base layer we are experimenting with GlusterFS, distributed and replicated file system that offers very simple management and high availablity. It does run in userspace, which according to some will suffer from performance bottlenecks, but so far we have not seen any serious problems while running on 15k disks in raid 10. Assuming you have two virtual machines you could create a distributed file system between them, and have a mirrored copy of the data on both. Next is dovecot/postfix/webmail which would be setup to use the local GlusterFS mount on the system containing the mail storage and indexes. To provide HA on the connectivity side we used ucarp, which creates a virtual IP address between two servers and fails that virtual IP over to another server in the event of a server going down. I personally would never use DNS load balancing (two IPs for one DNS name) as it would round robin to each server regardless of whether it is running or not. This is somewhat exotic, but it works and provides a very high level of availability. However with HA comes more complexity and management. Good luck and let me know if you would like more specifics. From seandarcy2 at gmail.com Tue Nov 29 01:50:18 2011 From: seandarcy2 at gmail.com (sean darcy) Date: Mon, 28 Nov 2011 18:50:18 -0500 Subject: [Dovecot] can not get t'bird to create folder Message-ID: dovecot-2.0.16 on Fedora 15. thunderbird-8.0 I can access dovecot from thunderbird, imap maildir. But when I try to create a new folder, nothing happens. I'm not using LAYOUT=fs, so I have added, tb-extra-mailbox-sep, which didn't seem relevant anyhow. Any help appreciated. sean From micah at riseup.net Tue Nov 29 02:04:41 2011 From: micah at riseup.net (Micah Anderson) Date: Mon, 28 Nov 2011 19:04:41 -0500 Subject: [Dovecot] Indexes to MLC-SSD References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: <87sjl7lrom.fsf@algae.riseup.net> Dovecot-GDH writes: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net > > Flashcache: https://github.com/facebook/flashcache/ That site has no information about what flashcache is. From seandarcy2 at gmail.com Tue Nov 29 04:28:29 2011 From: seandarcy2 at gmail.com (sean darcy) Date: Mon, 28 Nov 2011 21:28:29 -0500 Subject: [Dovecot] can't get sieve to sort virus into spam Message-ID: I can't get sieve to put virus files in the SPAM folder. dovecot -n # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.8-32.fc15.i686.PAE i686 Fedora release 15 (Lovelock) auth_debug_passwords = yes info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_access_groups = mail mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { antispam_backend = dspam-exec antispam_dspam_args = --deliver;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam = Spam antispam_trash = trash;Trash;Deleted Items; Deleted Messages sieve = /home/vmail/%d/%n/dovecot.sieve.script sieve_dir = /home/vmail/%d/%n/dovecot.sieve/ sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } user = vmail } ssl_cert = References: Message-ID: <1266621424.20111129030956@gmail.com> On Tuesday, November 29, 2011 at 02:28:29 UTC, seandarcy2 at gmail.com confabulated: > I can't get sieve to put virus files in the SPAM folder. > dovecot -n > # 2.0.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.38.8-32.fc15.i686.PAE i686 Fedora release 15 (Lovelock) > auth_debug_passwords = yes > info_log_path = /var/log/dovecot-info.log > log_path = /var/log/dovecot.log > mail_access_groups = mail > mail_home = /home/vmail/%d/%n > mail_location = maildir:~/mail > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date > mbox_write_locks = fcntl > passdb { > args = scheme=CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file > } > plugin { > antispam_backend = dspam-exec > antispam_dspam_args = --deliver;--user;%u > antispam_dspam_binary = /usr/bin/dspam > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam = Spam > antispam_trash = trash;Trash;Deleted Items; Deleted Messages > sieve = /home/vmail/%d/%n/dovecot.sieve.script > sieve_dir = /home/vmail/%d/%n/dovecot.sieve/ > sieve_global_dir = /var/lib/dovecot/sieve/global/ > sieve_global_path = /var/lib/dovecot/sieve/default.sieve > } > service auth { > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service lmtp { > inet_listener lmtp { > address = 127.0.0.1 > port = 24 > } > user = vmail > } > ssl_cert = ssl_key = userdb { > driver = passwd > } > userdb { > args = username_format=%u /etc/dovecot/users > driver = passwd-file > } > protocol lmtp { > mail_plugins = quota sieve > } > protocol imap { > mail_plugins = " antispam" > } > cat /var/lib/dovecot/sieve/default.sieve > require "fileinto"; > if header :contains "X-DSPAM-Result" "Virus" { > fileinto "SPAM"; > } > if header :contains "X-DSPAM-Result" "Spam" { > fileinto "SPAM"; > } > and there is a folder SPAM: > cat mail/subscriptions > Sent > Trash > Drafts > SPAM > yet : > Content-Type: multipart/alternative; > boundary=14dae9399d3b9b67cc04b2d60790 > X-DSPAM-Result: Virus > X-DSPAM-Processed: Mon Nov 28 21:03:47 2011 > X-DSPAM-Confidence: 1.0000 > X-DSPAM-Probability: 1.0000 > X-DSPAM-Signature: 4ed43d8341479178914901 > --14dae9399d3b9b67cc04b2d60790 > Content-Type: text/plain; charset=ISO-8859-1 > X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H* > does not end up into SPAM, or anyplace else, including inbox. > How do I make this work? What gets logged? If sieve has an issue storing something, it is logged. If you don't have the autocreate plugin loaded, the directories have to be created some other way. Does SPAM exist as a directory? Have you compiled the global sieve file using sievec? Does the particular mail account have sieve rules? Global rules are not executed if there are account rules. If that is the case, check out the sieve_before and sieve_after parameters. -- There are 10 kinds of people in the world... Those who understand binary, and those who don't. From rainer.frey at inxmail.de Tue Nov 29 09:19:21 2011 From: rainer.frey at inxmail.de (Rainer Frey) Date: Tue, 29 Nov 2011 08:19:21 +0100 Subject: [Dovecot] Authentication listener configuration Message-ID: <22BC7966-5C73-4085-AB63-3B5E20FCE335@inxmail.de> Hi, I'm wondering about the authentication listeners that dovecot uses. In many configurations I see: service auth { unix_listener auth-userdb { Does any component of dovecot use this listener internally (lmtp, lda ...) and does that read the config and search for the listener, or is the exact name "auth-userdb" expected? Is this maybe dependent on the userdb? Which ones need it? And if I want to use postfix SMTP auth, should I define a second listener, or just re-configure that default one to postfix's needs? Also, in which cases do I need to define an auth-worker service, and a unix_listener for it? I don't find this in the documentation, if that's caused by my lacking search skills, please just send apointer. Thanks Rainer From robert at schetterer.org Tue Nov 29 09:49:52 2011 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 29 Nov 2011 08:49:52 +0100 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <4ED48EA0.4060706@schetterer.org> Am 28.11.2011 23:14, schrieb Marcelo Salhab Brogliato: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. > > About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). > I'm using dovecot 1.2.9. > > I guess you already had some of these problems or maybe in another architecture these neither exists. > > Thanks for any help, > > Marcelo > i have run , store drbd with ocfs behind keepalived loadbalancers this should run with vm too, but it might not optimal, depening i.e how much mailboxes etc search this list archive, there were posts about equal issues -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stephan at rename-it.nl Tue Nov 29 10:00:39 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 29 Nov 2011 09:00:39 +0100 Subject: [Dovecot] can't get sieve to sort virus into spam In-Reply-To: <1266621424.20111129030956@gmail.com> References: <1266621424.20111129030956@gmail.com> Message-ID: <4ED49127.6020000@rename-it.nl> On 11/29/2011 4:09 AM, Duane Hill wrote: > On Tuesday, November 29, 2011 at 02:28:29 UTC, seandarcy2 at gmail.com confabulated: > >> I can't get sieve to put virus files in the SPAM folder. >> does not end up into SPAM, or anyplace else, including inbox. >> How do I make this work? > What gets logged? If sieve has an issue storing something, it is > logged. The message must be stored somewhere, even if Sieve fails entirely. If your Dovecot logs provide no indication that a message was handled, are you even sure that LTMP is getting a message at all? In this regard your MTA's logs are of importance as well. Regards, Stephan. From pw at wk-serv.de Tue Nov 29 16:24:22 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 29 Nov 2011 15:24:22 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111125081715.GA24352@dibs.tanso.net> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> <20111125081715.GA24352@dibs.tanso.net> Message-ID: <4ED4EB16.1030704@wk-serv.de> Hi again, I have a working test environment not but I don't feel very comfortable with a non encrypted connection between the proxy/director and the backend servers. If I set ssl = required on my backend server I get this error: imap-login: Disconnected (tried to use disabled plaintext auth) Is there a chance to encrypt this connection? # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 auth_debug = yes auth_verbose = yes director_mail_servers = 172.17.0.31 director_servers = 172.17.213.213 disable_plaintext_auth = no lmtp_proxy = yes mail_debug = yes passdb { args = proxy=y nopassword=y driver = static } service auth { unix_listener auth-userdb { mode = 0600 user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } ssl = required ssl_cert = Hi, after upgrading from 2.0.15 to 2.0.16 authentication with vpopmail won?t work anymore. 2.0.x and 1.2.x works (this output is from 2.0.15): Nov 28 18:29:21 mail dovecot: auth: Debug: auth client connected (pid=3855) Nov 28 18:29:21 mail dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=10.114.57.11 rip=10.114.57.11 lport=143 rport=37693 resp=d3NjaG9lbGxlckBodGwtcGVyZy5hYy5hdAB3c2Nob2VsbGVyQGh0bC1wZXJnLmFjLmF0AGRpYWJsbw== Nov 28 18:29:21 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: client out: OK 1 user=wschoeller at htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: master in: REQUEST 4001103873 3855 1 7a21a4d39cc6b187b3b14dc4cfbf077a Nov 28 18:29:21 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: master out: USER 4001103873 wschoeller at htl-perg.ac.at uid=89 gid=89 home=/var/vpopmail/domains/htl-perg.ac.at/wschoeller Nov 28 18:29:21 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=10.114.57.11, lip=10.114.57.11, mpid=3856, secured Nov 28 18:29:21 mail dovecot: imap(wschoeller at htl-perg.ac.at): Disconnected: Logged out bytes=29/405 Nov 28 18:29:21 mail dovecot: auth: Debug: auth client connected (pid=3858) 2.0.16 doesn?t work: Nov 28 18:43:15 mail dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/auth Nov 28 18:43:15 mail dovecot: auth: Debug: auth client connected (pid=27392) Nov 28 18:43:15 mail dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=10.114.57.11 rip=10.114.57.11 lport=143 rport=42768 resp=d3NjaG9lbGxlckBodGwtcGVyZy5hYy5hdAB3c2Nob2VsbGVyQGh0bC1wZXJnLmFjLmF0AGRpYWJsbw== Nov 28 18:43:15 mail dovecot: auth-worker: Debug: Loading modules from directory: /usr/lib/dovecot/auth Nov 28 18:43:15 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:43:15 mail dovecot: auth-worker: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): Password mismatch Nov 28 18:43:15 mail dovecot: auth-worker: Error: md5_verify(wschoeller at htl-perg.ac.at): Not a valid MD5-CRYPT or PLAIN-MD5 password Nov 28 18:43:15 mail dovecot: auth-worker: Error: smd5_verify(wschoeller at htl-perg.ac.at): SMD5 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha_verify(wschoeller at htl-perg.ac.at): SSHA password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha256_verify(wschoeller at htl-perg.ac.at): SSHA256 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha512_verify(wschoeller at htl-perg.ac.at): SSHA512 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb Nov 28 18:43:15 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): CLEARTEXT(mypassword) != '', try OTP scheme instead Nov 28 18:43:17 mail dovecot: auth: Debug: client out: FAIL 1 user=wschoeller at htl-perg.ac.at Nov 28 18:43:17 mail dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=10.114.57.11, lip=10.114.57.11, secured Dovecot -n: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.36-gentoo-r8 i686 Gentoo Base System release 1.12.14 auth_debug = yes auth_debug_passwords = yes disable_plaintext_auth = no first_valid_uid = 89 listen = * mail_location = maildir:~/.maildir namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . type = private } passdb { args = webmail=127.0.0.1 driver = vpopmail } service auth { user = root } ssl_cert = References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <87sjl7lrom.fsf@algae.riseup.net> Message-ID: <0DD532C1-7E22-49BA-A457-250194CF4828@rocketfish.com> https://github.com/facebook/flashcache/blob/master/doc/flashcache-doc.txt On Nov 28, 2011, at 4:04 PM, Micah Anderson wrote: > Dovecot-GDH writes: > >> If I/O performance is a concern, you may be interested in ZFS and Flashcache. >> >> Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) >> ZFS does run on Linux http://zfs-fuse.net >> >> Flashcache: https://github.com/facebook/flashcache/ > > That site has no information about what flashcache is. > > From dbrooks at mdah.state.ms.us Tue Nov 29 22:46:35 2011 From: dbrooks at mdah.state.ms.us (Donny Brooks) Date: Tue, 29 Nov 2011 14:46:35 -0600 Subject: [Dovecot] Best setup for performance and fault tolerance Message-ID: <6f4f-4ed54480-5-fba0340@98207411> Hello all. I am in need of some guidance. First a little background. Currently our mail server is on physical hardware (Dell server with 2x 2.8GHz Xeon w/ 4GB ram, raid5 array, single gigabit nic) running on Fedora 11 and postfix-2.5.6-3.fc11.x86_64 with dovecot-1.2.11-3.fc11.x86_64. Mailstore is via Maildir format that was converted from mbox about a year ago. This same machine is also our PDC with samba, Master LDAP, slave MySQL replication, primary DNS, and home server for about 20 users. We have approximately 200 end users that have mailboxes on the server ranging from 1KB to 20GB in size. Total mail store is currently at 300GB. About 75 of the users are currently POP access and their mail will be moved to the server soon and setup as IMAP. This is calculated to add roughly another 150GB of mail for a total of 450GB mail store. Being a state agency we have to keep the mail indefinitely for public record reasons. We use a mixture of Thunderbird as an IMAP client and SOGo for web access. Now to the problem: Recently we have been having super slow access to the mail server. Turns out the load was insanely high partially due to the samba home server portion, which is being moved off as we speak, and the other part is due to people searching their mail. Just yesterday one of our users nearly brought the entire agency to its knees by performing a search on her 8GB of mail via IMAP. Since the server is old in both hardware and software I have been tasked with moving it to newer hardware and a newer OS. We currently have 3 virtual servers running Xen and a SAN. The new setup will be placed in the virtual environment. I will probably run Fedora 16 as the OS but am open to Centos, Fedora, or Ubuntu. Now to the question: What is the best way to setup Dovecot so that it is tuned for performance and high available? We have been running with this single point of failure for years so as long as we are moving the mail server we might as well build in some redundancy. To solve the searching problem I thought of maybe setting up some type of indexing. I do kind of want to break the various services out on to separate virtual machines for a little more fault tolerance, but that is not totally necessary. What do you think of things like iRedmail? I see it's usefulness but the not being able to separate services kind of defeats the purpose, plus I want to setup a high available MySQL cluster and possibly OpenLDAP or 389 cluster so iRedMail may not be the best solution. Sorry for the long email but I am trying to get all the information out there at once so it will help get more directed responses in the shortest amount of time. I look forward to any and all input on this matter -- Donny B MDAH From Ralf.Hildebrandt at charite.de Tue Nov 29 23:42:56 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 29 Nov 2011 22:42:56 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders Message-ID: <20111129214256.GL14970@charite.de> While our webmail installation creates "Sent" as a Sent-Items Folder, Apple Mail creates "Sent Messages". Is there a way of "mapping" foldernames e.g. map "Sent Messages" to "Sent" map "Deletes Messages" to "Trash" and so on? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From micah at riseup.net Tue Nov 29 23:50:59 2011 From: micah at riseup.net (micah anderson) Date: Tue, 29 Nov 2011 16:50:59 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <87ehx8gic7.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> Message-ID: <87zkfek37g.fsf@algae.riseup.net> On Wed, 16 Nov 2011 11:09:12 -0500, micah anderson wrote: > On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen wrote: > > On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > > > I really like the feature where you can define quota rules with percents > > > which trigger off of the default values[0] (so you can set the Trash to > > > allow for 10% more of the user's quota for example). > > > > > > What I would really love in dovecot would be for the ability to > > > configure a quota rule for dovecot-lda. I would like to configure things > > > so we don't bounce emails for users until they are well over quota, the > > > IMAP quota plugin is a really great way to notify people that they are > > > over quota because it fails to write to other folders.... that should be > > > enough to get people's attention that they need to deal with things, but > > > bouncing is harsh. > > > > You can give LDA-specific quota settings. Like maybe: > > > > protocol lda { > > plugin { > > quota_rule2 = INBOX:storage=200% > > } > > } I just tried doing this, and I started bouncing mail for everyone because they were over quota (even when they arent), so I am clearly missing something in how I have it configured. This is what I have: namespace { separator = . prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no } plugin { quota = dict:Your mail quota::proxy::quota } dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } plugin { quota_exceeded_message = You are over quota. To avoid losing mail, immediately empty your Trash and Sent folders and delete emails with large attachments. quota_rule = *:bytes=100663296 # in case someone doesn't have sql quota quota_rule2 = Trash:bytes=+20%% # let trash be larger by 20% quota_rule3 = Spam:bytes=+10%% # let spam be larger by 10% quota_rule4 = INBOX.Trash:bytes=+20%% # repeat the above for namespaces quota_rule5 = INBOX.Spam:bytes=+10%% # same quota_rule6 = INBOX.restored:bytes=+500%% # allow for temporary folder } ... protocol lda { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins plugin { quota_rule7 = INBOX:bytes=+10%% } } Maybe I should have that quota_rule7 be *:bytes=+10%% ? thanks for any suggestions for what I am missing here! micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From p at state-of-mind.de Tue Nov 29 23:51:49 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 29 Nov 2011 22:51:49 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <20111129215149.GA29651@state-of-mind.de> * Ralf Hildebrandt : > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? There's an RFC, but it hasn't been adopted yet. Neither by IMAP servers nor by IMAP clients. ;) p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From tss at iki.fi Wed Nov 30 00:17:05 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 30 Nov 2011 00:17:05 +0200 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> On 29.11.2011, at 23.42, Ralf Hildebrandt wrote: > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? With v2.1 it would be somewhat easy to create a plugin that supports aliases, but each client would still see each others' Sent boxes and possibly synchronize everything twice.. From slusarz at curecanti.org Wed Nov 30 00:17:23 2011 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 29 Nov 2011 15:17:23 -0700 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129215149.GA29651@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> Message-ID: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Quoting Patrick Ben Koetter

: > * Ralf Hildebrandt : >> While our webmail installation creates "Sent" as a Sent-Items Folder, >> Apple Mail creates "Sent Messages". >> >> Is there a way of "mapping" foldernames e.g. >> >> map "Sent Messages" to "Sent" >> map "Deletes Messages" to "Trash" >> and so on? > > There's an RFC, but it hasn't been adopted yet. Neither by IMAP > servers nor by > IMAP clients. ;) Not true. We've (IMP) had special-use (RFC 6154) support since August 2010. Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. michael From p at state-of-mind.de Wed Nov 30 00:26:51 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 29 Nov 2011 23:26:51 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Message-ID: <20111129222651.GG10344@state-of-mind.de> * Michael M Slusarz : > Quoting Patrick Ben Koetter

: > > >* Ralf Hildebrandt : > >>While our webmail installation creates "Sent" as a Sent-Items Folder, > >>Apple Mail creates "Sent Messages". > >> > >>Is there a way of "mapping" foldernames e.g. > >> > >>map "Sent Messages" to "Sent" > >>map "Deletes Messages" to "Trash" > >>and so on? > > > >There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >servers nor by > >IMAP clients. ;) > > Not true. We've (IMP) had special-use (RFC 6154) support since August 2010. > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. Very nice. Thanks for the update. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From stan at hardwarefreak.com Wed Nov 30 07:59:33 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 29 Nov 2011 23:59:33 -0600 Subject: [Dovecot] Best setup for performance and fault tolerance In-Reply-To: <6f4f-4ed54480-5-fba0340@98207411> References: <6f4f-4ed54480-5-fba0340@98207411> Message-ID: <4ED5C645.1000605@hardwarefreak.com> On 11/29/2011 2:46 PM, Donny Brooks wrote: > Hello all. I am in need of some guidance. First a little background. Currently our mail server is on physical hardware (Dell server with 2x 2.8GHz Xeon w/ 4GB ram, raid5 array, single gigabit nic) running on Fedora 11 and postfix-2.5.6-3.fc11.x86_64 with dovecot-1.2.11-3.fc11.x86_64. Mailstore is via Maildir format that was converted from mbox about a year ago. This same machine is also our PDC with samba, Master LDAP, slave MySQL replication, primary DNS, and home server for about 20 users. > > We have approximately 200 end users that have mailboxes on the server ranging from 1KB to 20GB in size. Total mail store is currently at 300GB. About 75 of the users are currently POP access and their mail will be moved to the server soon and setup as IMAP. This is calculated to add roughly another 150GB of mail for a total of 450GB mail store. Being a state agency we have to keep the mail indefinitely for public record reasons. We use a mixture of Thunderbird as an IMAP client and SOGo for web access. > > Now to the problem: Recently we have been having super slow access to the mail server. Turns out the load was insanely high partially due to the samba home server portion, which is being moved off as we speak, and the other part is due to people searching their mail. Just yesterday one of our users nearly brought the entire agency to its knees by performing a search on her 8GB of mail via IMAP. > > Since the server is old in both hardware and software I have been tasked with moving it to newer hardware and a newer OS. We currently have 3 virtual servers running Xen and a SAN. The new setup will be placed in the virtual environment. I will probably run Fedora 16 as the OS but am open to Centos, Fedora, or Ubuntu. > > Now to the question: What is the best way to setup Dovecot so that it is tuned for performance and high available? We have been running with this single point of failure for years so as long as we are moving the mail server we might as well build in some redundancy. To solve the searching problem I thought of maybe setting up some type of indexing. I do kind of want to break the various services out on to separate virtual machines for a little more fault tolerance, but that is not totally necessary. > > What do you think of things like iRedmail? I see it's usefulness but the not being able to separate services kind of defeats the purpose, plus I want to setup a high available MySQL cluster and possibly OpenLDAP or 389 cluster so iRedMail may not be the best solution. > > Sorry for the long email but I am trying to get all the information out there at once so it will help get more directed responses in the shortest amount of time. I look forward to any and all input on this matter > -- > Donny B > MDAH Build an Enkive server: http://www.enkive.org/ and configure your SMTP MTAs to transparently copy all email to it (recipient_bcc for example). This fulfills your retention requirements. Since all emails are now archived by Enkive as they arrive, cron a nightly script on the Dovecot server to delete any emails over a week/month/etc old (depending on your short term access needs) from your active Dovecot mailboxes. This drastically reduces your Dovecot storage requirements. Searches will be performed by the Enkive server, removing that load from your Dovecot host. Search interface demo here: http://www.enkive.org/demo In addition your total mail storage (active+archive) requirement will be a fraction of what it is now because Enkive performs deduplication of email content and attachments so you save even more disk space. Enkive can run fine as a VM if you give it the required resources. Carve an appropriately sized LUN off the SAN array for the Enkive server storage. Format it with XFS for best performance. Enkive should be a good fit for your needs. Bear in mind installing/configuring it is not for the faint of heart: http://wiki.enkive.org/index.php/Installation_Instructions http://wiki.enkive.org/index.php/GettingMailIntoEnkive But thankfully administration is relatively easy: http://wiki.enkive.org/index.php/Administrator_Manual -- Stan From robert at schetterer.org Wed Nov 30 08:48:36 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:48:36 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <4ED5D1C4.8010607@schetterer.org> Am 29.11.2011 22:42, schrieb Ralf Hildebrandt: > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? > wish i would know a solution i know newest outlook can configured equal to tb folders default schema , except spam folder -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 08:50:39 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:50:39 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> References: <20111129214256.GL14970@charite.de> <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> Message-ID: <4ED5D23F.2030609@schetterer.org> Am 29.11.2011 23:17, schrieb Timo Sirainen: > On 29.11.2011, at 23.42, Ralf Hildebrandt wrote: > >> While our webmail installation creates "Sent" as a Sent-Items Folder, >> Apple Mail creates "Sent Messages". >> >> Is there a way of "mapping" foldernames e.g. >> >> map "Sent Messages" to "Sent" >> map "Deletes Messages" to "Trash" >> and so on? > > With v2.1 it would be somewhat easy to create a plugin that supports aliases, but each client would still see each others' Sent boxes and possibly synchronize everything twice.. that would be no good solution....in...my eyes > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 08:53:08 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:53:08 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Message-ID: <4ED5D2D4.2090603@schetterer.org> Am 29.11.2011 23:17, schrieb Michael M Slusarz: > Quoting Patrick Ben Koetter

: > >> * Ralf Hildebrandt : >>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>> Apple Mail creates "Sent Messages". >>> >>> Is there a way of "mapping" foldernames e.g. >>> >>> map "Sent Messages" to "Sent" >>> map "Deletes Messages" to "Trash" >>> and so on? >> >> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >> servers nor by >> IMAP clients. ;) > > Not true. We've (IMP) had special-use (RFC 6154) support since August > 2010. > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > > michael > what helps this, tb, outlook, apple mail, winmail and some mobile clients must follow , this is what people use, and it will take years after some of them might upgrade -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From akam at zakamye.ru Wed Nov 30 08:57:57 2011 From: akam at zakamye.ru (=?UTF-8?B?0JDQudC00LDRgCDQmtCw0LzQsNC70L7Qsg==?=) Date: Wed, 30 Nov 2011 10:57:57 +0400 Subject: [Dovecot] second active directory Message-ID: <4ED5D3F5.1020102@zakamye.ru> Hello! There is dovecot authenticating through AD. All is working well, if authentication is just one of AD. But there is another AD, and working with 2 AD's dovecot don't recognizes last AD user's home directory. For example, if user name in the second AD - 1developer and Mail - it-2c at zakamye.ru then dovecot creates folder /var/mail/zakamye.ru/1cdeveloper but it should be /var/mail/zakamye.ru/it-2c. What I do wrong?( And, if I change the order of viewing AD, does not work precisely the second AD: passdb ldap-DUSLYK { driver = ldap args = /etc/dovecot/dovecot-DUSLYK.conf } userdb ldap-DUSLYK { driver = ldap args = /etc/dovecot/dovecot-DUSLYK-users.conf } passdb ldap-DUSL-OFFICE { driver = ldap args = /etc/dovecot/dovecot-DUSL-OFFICE.conf } userdb ldap-DUSL-BASE { driver = ldap args = /etc/dovecot/dovecot-DUSL-OFFICE-users.conf } Sorry for my bad english, but maybe you can understand logs and configs: http://pastebin.com/mtk4XsNE -- ????? ?. ???????, ????????? ????????????? ??? "?? ??????" ?. ???. ?????, ???. ????????? ???.: +7 (8552) 47-01-99 ???. 3025 ???.: +7 (8552) 70-65-15 ???. 280 ???.: +7 (919) 621-28-52 E-mail: akam at zakamye.ru From p at state-of-mind.de Wed Nov 30 10:21:42 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 30 Nov 2011 09:21:42 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5D2D4.2090603@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> Message-ID: <20111130082141.GB1594@state-of-mind.de> * Robert Schetterer : > Am 29.11.2011 23:17, schrieb Michael M Slusarz: > > Quoting Patrick Ben Koetter

: > > > >> * Ralf Hildebrandt : > >>> While our webmail installation creates "Sent" as a Sent-Items Folder, > >>> Apple Mail creates "Sent Messages". > >>> > >>> Is there a way of "mapping" foldernames e.g. > >>> > >>> map "Sent Messages" to "Sent" > >>> map "Deletes Messages" to "Trash" > >>> and so on? > >> > >> There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >> servers nor by > >> IMAP clients. ;) > > > > Not true. We've (IMP) had special-use (RFC 6154) support since August > > 2010. > > > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > > > > michael > > > > what helps this, tb, outlook, apple mail, winmail and some mobile > clients must follow , this is what people use, and it will take years > after some of them might upgrade What helps this == Was hilft es? TB has it on the CR list. Apple has their own X-LIST feature together with google. p at rick > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Wed Nov 30 10:25:46 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 09:25:46 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111130082141.GB1594@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> Message-ID: <4ED5E88A.20205@schetterer.org> Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: > * Robert Schetterer : >> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>> Quoting Patrick Ben Koetter

: >>> >>>> * Ralf Hildebrandt : >>>>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>>>> Apple Mail creates "Sent Messages". >>>>> >>>>> Is there a way of "mapping" foldernames e.g. >>>>> >>>>> map "Sent Messages" to "Sent" >>>>> map "Deletes Messages" to "Trash" >>>>> and so on? >>>> >>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>> servers nor by >>>> IMAP clients. ;) >>> >>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>> 2010. >>> >>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>> >>> michael >>> >> >> what helps this, tb, outlook, apple mail, winmail and some mobile >> clients must follow , this is what people use, and it will take years >> after some of them might upgrade > > What helps this == Was hilft es? > > TB has it on the CR list. > Apple has their own X-LIST feature together with google. cool, *g left another hundred clients however having this widly solved would be a dream why i did take so long for it.........., its a Problem since years > > p at rick > > > > > >> >> -- >> Best Regards >> >> MfG Robert Schetterer >> >> Germany/Munich/Bavaria > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 10:33:24 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 09:33:24 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5E88A.20205@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> Message-ID: <4ED5EA54.7000007@schetterer.org> Am 30.11.2011 09:25, schrieb Robert Schetterer: > Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: >> * Robert Schetterer : >>> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>>> Quoting Patrick Ben Koetter

: >>>> >>>>> * Ralf Hildebrandt : >>>>>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>>>>> Apple Mail creates "Sent Messages". >>>>>> >>>>>> Is there a way of "mapping" foldernames e.g. >>>>>> >>>>>> map "Sent Messages" to "Sent" >>>>>> map "Deletes Messages" to "Trash" >>>>>> and so on? >>>>> >>>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>>> servers nor by >>>>> IMAP clients. ;) >>>> >>>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>>> 2010. >>>> >>>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>>> >>>> michael >>>> >>> >>> what helps this, tb, outlook, apple mail, winmail and some mobile >>> clients must follow , this is what people use, and it will take years >>> after some of them might upgrade >> >> What helps this == Was hilft es? >> >> TB has it on the CR list. >> Apple has their own X-LIST feature together with google. > > cool, *g left another hundred clients > > however having this widly solved would be a dream > why i did take so long for it.........., its a Problem since years hoping big spreaded clients will solve it soon perhaps a chance to block non fixed clients would be nice any chance ?, i see clients anounce their name and versions in debug log > >> >> p at rick >> >> >> >> >> >>> >>> -- >>> Best Regards >>> >>> MfG Robert Schetterer >>> >>> Germany/Munich/Bavaria >> > > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From p at state-of-mind.de Wed Nov 30 10:56:27 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 30 Nov 2011 09:56:27 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5E88A.20205@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> Message-ID: <20111130085626.GD1594@state-of-mind.de> * Robert Schetterer : > Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: > > * Robert Schetterer : > >> Am 29.11.2011 23:17, schrieb Michael M Slusarz: > >>> Quoting Patrick Ben Koetter

: > >>>> > >>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >>>> servers nor by IMAP clients. ;) > >>> > >>> Not true. We've (IMP) had special-use (RFC 6154) support since August > >>> 2010. > >>> > >>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > >>> > >>> michael > >>> > >> > >> what helps this, tb, outlook, apple mail, winmail and some mobile > >> clients must follow , this is what people use, and it will take years > >> after some of them might upgrade > > > > What helps this == Was hilft es? > > > > TB has it on the CR list. > > Apple has their own X-LIST feature together with google. > > cool, *g left another hundred clients > > however having this widly solved would be a dream > why i did take so long for it.........., its a Problem since years Yes, it is a problem, but having a standard doesn't solve it and open source software is not a guarant to have it implemented either, unless you contribute the code yourself or get to sponsor someone to do it. BTW: We'd contribute ? 500 if Timo implemented RFC 6154 in Dovecot 2.1. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Wed Nov 30 11:05:26 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 10:05:26 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111130085626.GD1594@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> <20111130085626.GD1594@state-of-mind.de> Message-ID: <4ED5F1D6.7000501@schetterer.org> Am 30.11.2011 09:56, schrieb Patrick Ben Koetter: > * Robert Schetterer : >> Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: >>> * Robert Schetterer : >>>> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>>>> Quoting Patrick Ben Koetter

: >>>>>> >>>>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>>>> servers nor by IMAP clients. ;) >>>>> >>>>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>>>> 2010. >>>>> >>>>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>>>> >>>>> michael >>>>> >>>> >>>> what helps this, tb, outlook, apple mail, winmail and some mobile >>>> clients must follow , this is what people use, and it will take years >>>> after some of them might upgrade >>> >>> What helps this == Was hilft es? >>> >>> TB has it on the CR list. >>> Apple has their own X-LIST feature together with google. >> >> cool, *g left another hundred clients >> >> however having this widly solved would be a dream >> why i did take so long for it.........., its a Problem since years > > Yes, it is a problem, but having a standard doesn't solve it and open source > software is not a guarant to have it implemented either, unless you contribute > the code yourself or get to sponsor someone to do it. > > BTW: We'd contribute ? 500 if Timo implemented RFC 6154 in Dovecot 2.1. very cool !!!!, as an add on idea, if it would be possible to identificate unfixed clients on the fly and doing some action on it, it might be a solution to allow imap at fixed clients only , others have to use pop3 then, not a solution everywhere but it could be implemented in some places ( i.e mine *g ) > > p at rick > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From info at simonecaruso.com Wed Nov 30 12:33:29 2011 From: info at simonecaruso.com (Simone Caruso) Date: Wed, 30 Nov 2011 11:33:29 +0100 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <4ED60679.1010903@simonecaruso.com> On 28/11/2011 23:14, Marcelo Salhab Brogliato wrote: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. > For imap i need only 1 VIP both for loadbalancing or HA, for SMTP u can use dns failover with multiple mx records. > About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). > I'm using dovecot 1.2.9. I suggest to not use local indexing (for high i/o after failover) , anyway NFS for HA setup works well! If u need a cluster (master/master) consider having an HA loadbancer (dovecot director): -- Simone Caruso IT Consultant +39 349 65 90 805 From akam at zakamye.ru Wed Nov 30 12:39:31 2011 From: akam at zakamye.ru (=?UTF-8?B?0JDQudC00LDRgCDQmtCw0LzQsNC70L7Qsg==?=) Date: Wed, 30 Nov 2011 14:39:31 +0400 Subject: [Dovecot] second active directory In-Reply-To: <4ED5D3F5.1020102@zakamye.ru> References: <4ED5D3F5.1020102@zakamye.ru> Message-ID: <4ED607E3.40805@zakamye.ru> SOLVED. Problem solved. There was double users in both AD domains. 30.11.2011 10:57, ????? ??????? ?????: > Hello! > There is dovecot authenticating through AD. All is working well, if > authentication is just one of AD. > But there is another AD, and working with 2 AD's dovecot don't > recognizes last AD user's home directory. > For example, if user name in the second AD - 1developer and Mail - > it-2c at zakamye.ru then dovecot creates folder > /var/mail/zakamye.ru/1cdeveloper but it should be > /var/mail/zakamye.ru/it-2c. > > What I do wrong?( > > And, if I change the order of viewing AD, does not work precisely the > second AD: > passdb ldap-DUSLYK { > driver = ldap > args = /etc/dovecot/dovecot-DUSLYK.conf > } > userdb ldap-DUSLYK { > driver = ldap > args = /etc/dovecot/dovecot-DUSLYK-users.conf > } > passdb ldap-DUSL-OFFICE { > driver = ldap > args = /etc/dovecot/dovecot-DUSL-OFFICE.conf > } > userdb ldap-DUSL-BASE { > driver = ldap > args = /etc/dovecot/dovecot-DUSL-OFFICE-users.conf > } > > Sorry for my bad english, but maybe you can understand logs and configs: > http://pastebin.com/mtk4XsNE > From heptalium at gmx.de Wed Nov 30 20:04:16 2011 From: heptalium at gmx.de (Jens =?UTF-8?B?TWVpw59uZXI=?=) Date: Wed, 30 Nov 2011 19:04:16 +0100 Subject: [Dovecot] lda/sieve ignores quota from ldap Message-ID: <20111130190416.641ede1f@meissner.pentament.dyndns.org> Hello, I'm using Dovecot 2.0 with Sieve as MDA and Postfix as MTA. The Postfix delivery command is /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT". I have set a default quota with quota_rule = *:storage=1G. Some users should have a larger quota. This is defined in a ldap directory and I have set user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mailQuota=quota_rule=*:storage=%$. The imap_quota plugin shows the right quota for all users but the lda ignores the quota rule from ldap. If a user is over default quota but not over his ldap defined quota I get the following error message: From: Mail Delivery Subsystem Your message to <[?]> was automatically rejected: Quota exceeded (mailbox for user is full) The log file shows the following: Nov 30 18:21:17 [?] dovecot: lda([?]): Error: sieve: msgid=<20111130172116.7F065A07D@[?]>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) Nov 30 18:21:17 [?] dovecot: lda([?]): Error: sieve: script /var/mail/[?]/dovecot.sieve failed with unsuccessful implicit keep (user logfile /var/mail/[?]/dovecot.sieve.log may reveal additional details) Nov 30 18:21:17 [?] dovecot: lda([?]): msgid=<20111130172116.7F065A07D@[?]>: rejected: Quota exceeded (mailbox for user is full) dovecot.sieve.log: error: msgid=<20111130172116.7F065A07D@[?]>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full). What is wrong? What must I do to getting this to work? Thanks in advance. Regards Jens doveconf -n: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.3 nfs mail_location = maildir:/var/mail/%u mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = maildir quota_rule = *:storage=1G sieve = /var/mail/%u/dovecot.sieve sieve_dir = /var/mail/%u/sieve } protocols = " imap sieve pop3" ssl_cert = From terry at cnysupport.com Wed Nov 30 22:36:46 2011 From: terry at cnysupport.com (Terry Carmen) Date: Wed, 30 Nov 2011 15:36:46 -0500 Subject: [Dovecot] MS Exchange IMAP Proxy Message-ID: <20111130153646.Horde.fehRaFeGiNBO1pPe5xWA0vA@www.cnysupport.com> I need to make the IMAP interface of an Exchange 2000 server available on the net, however I would like to give it a little protection, and believe Dovecot's IMAP proxy might be appropriate. Does anybody have a *really simple* config that would allow IMAP pass-through to a single Exchange server? The examples I've seen all assume some sort of load balancing, which isn't an issue here, so I'd like to eliminate the need for an unnecessary database server. Does Dovecot do any cleanup of the IMAP commands that would be useful in protecting Exchange (from this, http://www.cvedetails.com/cve/CVE-2007-0221/ for example), or am I barking up the wrong tree? If Dovecot isn't helpful for this, can anybody point me to a better resource? Thanks! Terry From nam_10 at hotmail.com Tue Nov 1 00:20:56 2011 From: nam_10 at hotmail.com (nuno marques) Date: Mon, 31 Oct 2011 22:20:56 +0000 Subject: [Dovecot] Imap/pop gateway Message-ID: Hello, How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. regards From tss at iki.fi Tue Nov 1 08:24:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Nov 2011 08:24:33 +0200 Subject: [Dovecot] dsync should sync sieve-dirs to! In-Reply-To: <201110290141.41963.p.heinlein@heinlein-support.de> References: <201110290141.41963.p.heinlein@heinlein-support.de> Message-ID: <6589FF32-A0F6-4E75-A6A3-65356164E563@iki.fi> On 29.10.2011, at 2.41, Peer Heinlein wrote: > Having dsync to make backups from existing mail-spaces, it would be nice > to make dsync syncing the sieve-dirs too. -Otherweise backups aren't > complete... dsync was originally meant to be a way to do two-way syncing of mailboxes without losing any changes. That's not really possible to do with Sieve scripts. I guess the best that could be done is to copy the one with the latest timestamp. But if you have multiple Sieve scripts even that doesn't work well, since dsync can't know if a script was added on one side or deleted from another side. Although if scripts were only managed via ManageSieve, the last problem could be solved with some log file. For "dsync backup" I guess it could be useful to have a setting with a list of files/dirs that dsync would always upload to remote.. From tss at iki.fi Tue Nov 1 08:32:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Nov 2011 08:32:47 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: On 30.10.2011, at 13.53, Lukas Wei? wrote: > ive just switched to dovecot imap because dbmail > i used before is not developed any further Oh? It doesn't yet seem dead to me. > and sql storage is difficult > to backup. Ive done that, by setting up dovecot with maildir format and > copy mails from old dbmail to new dovecot imap server with my mail > client (thunderbird). Thunderbird probably changes the INTERNALDATE of the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. From laxlaw at gmx.de Tue Nov 1 10:38:02 2011 From: laxlaw at gmx.de (=?UTF-8?Q?Lukas_Wei=C3=9F?=) Date: Tue, 01 Nov 2011 09:38:02 +0100 Subject: [Dovecot] =?utf-8?q?Bad_creation_timestamp_of_migrated_mails_=28m?= =?utf-8?q?aildir_format=29=3F?= In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: On Tue, 1 Nov 2011 08:32:47 +0200, Timo Sirainen wrote: > On 30.10.2011, at 13.53, Lukas Wei? wrote: > >> ive just switched to dovecot imap because dbmail i used before is not developed any further > Oh? It doesn't yet seem dead to me. > >> and sql storage is difficult to backup. Ive done that, by setting up dovecot with maildir format and copy mails from old dbmail to new dovecot imap server with my mail client (thunderbird). > Thunderbird probably changes the INTERNALDATE of the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. What is INTERNALDATE? The modification timestamp is correct ("ls -l"), and the "Date: ... " line, too. @dbmail: Dead is the wrong designation. But since the last 5 years i only noticed bugfixes, the most "ideas for developement" (STARTTLS for example) mentioned here [1] are still ideas. Links: ------ [1] http://www.dbmail.org/dokuwiki/doku.php/2.3 From taeuber at bbaw.de Tue Nov 1 13:21:49 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Tue, 1 Nov 2011 12:21:49 +0100 Subject: [Dovecot] antispam for dovecot 2.0 Message-ID: <20111101122149.b8028453.taeuber@bbaw.de> Hi! Is there anybody out there who has ported the antispam-storage-1.x.c to antispam-storage-2.0.c? Or is there an alternative to the antispam plugin? Thanks Lars From duihi77 at gmail.com Tue Nov 1 14:34:30 2011 From: duihi77 at gmail.com (Duane Hill) Date: Tue, 1 Nov 2011 07:34:30 -0500 (CDT) Subject: [Dovecot] antispam for dovecot 2.0 In-Reply-To: <20111101122149.b8028453.taeuber@bbaw.de> References: <20111101122149.b8028453.taeuber@bbaw.de> Message-ID: On Tue, 1 Nov 2011, Lars T?uber wrote: > Hi! > > Is there anybody out there who has ported the antispam-storage-1.x.c to > antispam-storage-2.0.c? > > Or is there an alternative to the antispam plugin? Don't know anything about antispam-storage. From searching I assume you mean the antispam plugin for Dovecot 2.x. http://wiki2.dovecot.org/Plugins/Antispam shows where you can get it from and compile. From taeuber at bbaw.de Tue Nov 1 15:58:55 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Tue, 1 Nov 2011 14:58:55 +0100 Subject: [Dovecot] antispam for dovecot 2.0 In-Reply-To: References: <20111101122149.b8028453.taeuber@bbaw.de> Message-ID: <20111101145855.d14e671c.taeuber@bbaw.de> Am Tue, 1 Nov 2011 07:34:30 -0500 (CDT) Duane Hill schrieb: > On Tue, 1 Nov 2011, Lars T?uber wrote: > > > Hi! > > > > Is there anybody out there who has ported the antispam-storage-1.x.c to > > antispam-storage-2.0.c? > > > > Or is there an alternative to the antispam plugin? > > Don't know anything about antispam-storage. From searching I assume you > mean the antispam plugin for Dovecot 2.x. > > http://wiki2.dovecot.org/Plugins/Antispam shows where you can get it from > and compile. Hallo Duane, thank you very much. This is what I was searching for. Thanks Lars From dvstillerfan74 at gmail.com Tue Nov 1 17:55:36 2011 From: dvstillerfan74 at gmail.com (David Varela) Date: Tue, 1 Nov 2011 09:55:36 -0600 Subject: [Dovecot] LDAP w/SASL "Active Directory" authentication failing. Message-ID: Hello, I am running a Dovecot server (version 1.2.17) on FreeBSD 8.2, using LDAP to authenticate Active Directory users. I can successfully bind and authenticate using PLAIN and LDAP without SASL, but obviously passwords for the bind user and the user being authenticated are being passed in plain text. I've attempted to configure my server to us SASL however when I attempt to authenticate a user I see authentication failures. I reviewed the security log on my domain controller and see that the bind user is binding properly, so the issue appears to be orginating from the user authentication, however I cannot determine what the issue is. Here is all the information regarding my configuration, along with the logs from the server: Dovecot -n # 1.2.17: /usr/local/etc/dovecot.conf # OS: FreeBSD 8.2-RELEASE i386 ufs log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot_info.log protocols: imap pop3 imaps pop3s ssl_cert_file: /etc/ssl/dovecot/cert.pem ssl_key_file: /etc/ssl//dovecot/key.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login valid_chroot_dirs: /usr/data/vmail verbose_proctitle: yes first_valid_uid: 1001 last_valid_uid: 1001 first_valid_gid: 1001 last_valid_gid: 1001 mail_privileged_group: mail mail_location: maildir:/usr/data/vmail/%u mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain gssapi ntlm login username_format: %Ln winbind_helper_path: /usr/local/bin/ntlm_auth verbose: yes debug: yes debug_passwords: yes use_winbind: yes passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf userdb: driver: static args: uid=vmail gid=vmail home=/usr/data/vmail/%u socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 dovecot-ldap.conf hosts = 192.168.0.240:3268 dn = cn=PostfixBind,ou=IT,ou=Central Office,ou=LMC,dc=smallmountain,dc=Local dnpass = y0urm0mma sasl_bind = yes #sasl_mech = GSSAPI #sasl_realm = smallmountain.local #sasl_authz_id = debug_level = -1 #auth_bind = yes ldap_version = 3 base = dc=smallmountain,dc=Local deref = never scope = subtree user_filter = (&(objectClass=person)(mail=%u)) pass_attrs = mail=user pass_filter = (&(objectClass=person)(sAMAccountName=%n)) dovecot.log Nov 01 09:09:48 dovecot: Warning: Killed with signal 15 (by pid=99758 uid=0 code=kill) Nov 01 09:09:48 auth(default): Error: ldap_pvt_sasl_getmech Nov 01 09:09:48 auth(default): Error: ldap_search Nov 01 09:09:48 auth(default): Error: put_filter: "(objectclass=*)" Nov 01 09:09:48 auth(default): Error: put_filter: simple Nov 01 09:09:48 auth(default): Error: put_simple_filter: "objectclass=*" Nov 01 09:09:48 auth(default): Error: ldap_build_search_req ATTRS: supportedSASLMechanisms Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_new_connection 1 1 0 Nov 01 09:09:48 auth(default): Error: ldap_int_open_connection Nov 01 09:09:48 auth(default): Error: ldap_connect_to_host: TCP 192.168.0.240:3268 Nov 01 09:09:48 auth(default): Error: ldap_new_socket: 11 Nov 01 09:09:48 auth(default): Error: ldap_prepare_socket: 11 Nov 01 09:09:48 auth(default): Error: ldap_connect_to_host: Trying 192.168.0.240:3268 Nov 01 09:09:48 auth(default): Error: ldap_pvt_connect: fd: 11 tm: -1 async: 0 Nov 01 09:09:48 auth(default): Error: ldap_open_defconn: successful Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 1 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 1, origid 1, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 message type search-entry Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 1, origid 1, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: * msgid 1, type 100 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 1 message type search-result Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 1 Nov 01 09:09:48 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 1, msgid 1) Nov 01 09:09:48 auth(default): Error: adding response ld 0x18611700 msgid 1 type 101: Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: ldap_get_values Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_sasl_interactive_bind: server supports: GSSAPI GSS-SPNEGO EXTERNAL DIGEST-MD5 Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: GSSAPI GSS-SPNEGO EXTERNAL DIGEST-MD5 Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_open: host=adminserver-1.smallmountain.local Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 2 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 2, origid 2, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 2 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 2 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 2 Nov 01 09:09:48 auth(default): Error: res_errno: 14, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 2, msgid 2) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: sasl_client_step: 2 Nov 01 09:09:48 auth(default): Error: sasl_client_step: 1 Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 3 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 3, origid 3, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 3 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 3 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 3 Nov 01 09:09:48 auth(default): Error: res_errno: 14, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 3, msgid 3) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: sasl_client_step: 0 Nov 01 09:09:48 auth(default): Error: ldap_sasl_bind Nov 01 09:09:48 auth(default): Error: ldap_send_initial_request Nov 01 09:09:48 auth(default): Error: ldap_send_server_request Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:09:48 auth(default): Error: ldap_result ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: wait4msg ld 0x18611700 msgid 4 (infinite timeout) Nov 01 09:09:48 auth(default): Error: wait4msg continue ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:09:48 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:09:48 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:09:48 auth(default): Error: last used: Tue Nov 1 09:09:48 2011 Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:09:48 auth(default): Error: * msgid 4, origid 4, status InProgress Nov 01 09:09:48 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:09:48 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:09:48 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:09:48 auth(default): Error: Empty Nov 01 09:09:48 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:09:48 auth(default): Error: ldap_int_select Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 4 all 1 Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 msgid 4 message type bind Nov 01 09:09:48 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:09:48 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: request done: ld 0x18611700 msgid 4 Nov 01 09:09:48 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:09:48 auth(default): Error: ldap_free_request (origid 4, msgid 4) Nov 01 09:09:48 auth(default): Error: ldap_int_sasl_bind: Nov 01 09:09:48 auth(default): Error: ldap_parse_sasl_bind_result Nov 01 09:09:48 auth(default): Error: ldap_parse_result Nov 01 09:09:48 auth(default): Error: ldap_pvt_sasl_generic_install Nov 01 09:09:48 auth(default): Error: ldap_msgfree Nov 01 09:10:13 auth(default): Error: ldap_search Nov 01 09:10:13 auth(default): Error: put_filter: "(&(objectClass=person)(sAMAccountName=davidv))" Nov 01 09:10:13 auth(default): Error: put_filter: AND Nov 01 09:10:13 auth(default): Error: put_filter_list "(objectClass=person)(sAMAccountName=davidv)" Nov 01 09:10:13 auth(default): Error: put_filter: "(objectClass=person)" Nov 01 09:10:13 auth(default): Error: put_filter: simple Nov 01 09:10:13 auth(default): Error: put_simple_filter: "objectClass=person" Nov 01 09:10:13 auth(default): Error: put_filter: "(sAMAccountName=davidv)" Nov 01 09:10:13 auth(default): Error: put_filter: simple Nov 01 09:10:13 auth(default): Error: put_simple_filter: "sAMAccountName=davidv" Nov 01 09:10:13 auth(default): Error: ldap_build_search_req ATTRS: mail Nov 01 09:10:13 auth(default): Error: ldap_send_initial_request Nov 01 09:10:13 auth(default): Error: ldap_send_server_request Nov 01 09:10:13 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:10:13 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:10:13 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:10:13 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:10:13 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:10:13 auth(default): Error: last used: Tue Nov 1 09:10:13 2011 Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:10:13 auth(default): Error: * msgid 5, origid 5, status InProgress Nov 01 09:10:13 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:10:13 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:10:13 auth(default): Error: ldap_int_select Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid 5 message type search-entry Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 msgid 5 message type search-result Nov 01 09:10:13 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:10:13 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 5 Nov 01 09:10:13 auth(default): Error: request done: ld 0x18611700 msgid 5 Nov 01 09:10:13 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:10:13 auth(default): Error: ldap_free_request (origid 5, msgid 5) Nov 01 09:10:13 auth(default): Error: adding response ld 0x18611700 msgid 5 type 101: Nov 01 09:10:13 auth(default): Error: ldap_parse_result Nov 01 09:10:13 auth(default): Error: ldap_first_attribute Nov 01 09:10:13 auth(default): Error: ldap_get_values Nov 01 09:10:13 auth(default): Error: ldap_next_attribute Nov 01 09:10:13 auth(default): Error: ldap_msgfree Nov 01 09:10:13 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:10:13 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:10:13 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:10:13 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:10:13 auth(default): Error: refcnt: 1 status: Connected Nov 01 09:10:13 auth(default): Error: last used: Tue Nov 1 09:10:13 2011 Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 request count 0 (abandoned 0) Nov 01 09:10:13 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:10:13 auth(default): Error: Empty Nov 01 09:10:13 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:10:13 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:10:13 auth(default): Error: ldap_int_select Nov 01 09:13:26 auth(default): Error: ldap_search Nov 01 09:13:26 auth(default): Error: put_filter: "(&(objectClass=person)(sAMAccountName=davidv))" Nov 01 09:13:26 auth(default): Error: put_filter: AND Nov 01 09:13:26 auth(default): Error: put_filter_list "(objectClass=person)(sAMAccountName=davidv)" Nov 01 09:13:26 auth(default): Error: put_filter: "(objectClass=person)" Nov 01 09:13:26 auth(default): Error: put_filter: simple Nov 01 09:13:26 auth(default): Error: put_simple_filter: "objectClass=person" Nov 01 09:13:26 auth(default): Error: put_filter: "(sAMAccountName=davidv)" Nov 01 09:13:26 auth(default): Error: put_filter: simple Nov 01 09:13:26 auth(default): Error: put_simple_filter: "sAMAccountName=davidv" Nov 01 09:13:26 auth(default): Error: ldap_build_search_req ATTRS: mail Nov 01 09:13:26 auth(default): Error: ldap_send_initial_request Nov 01 09:13:26 auth(default): Error: ldap_send_server_request Nov 01 09:13:26 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:13:26 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:13:26 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:13:26 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:13:26 auth(default): Error: refcnt: 2 status: Connected Nov 01 09:13:26 auth(default): Error: last used: Tue Nov 1 09:13:26 2011 Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:13:26 auth(default): Error: * msgid 6, origid 6, status InProgress Nov 01 09:13:26 auth(default): Error: outstanding referrals 0, parent count 0 Nov 01 09:13:26 auth(default): Error: ld 0x18611700 request count 1 (abandoned 0) Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:13:26 auth(default): Error: ldap_int_select Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid 6 message type search-entry Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 msgid 6 message type search-result Nov 01 09:13:26 auth(default): Error: read1msg: ld 0x18611700 0 new referrals Nov 01 09:13:26 auth(default): Error: read1msg: mark request completed, ld 0x18611700 msgid 6 Nov 01 09:13:26 auth(default): Error: request done: ld 0x18611700 msgid 6 Nov 01 09:13:26 auth(default): Error: res_errno: 0, res_error: <>, res_matched: <> Nov 01 09:13:26 auth(default): Error: ldap_free_request (origid 6, msgid 6) Nov 01 09:13:26 auth(default): Error: adding response ld 0x18611700 msgid 6 type 101: Nov 01 09:13:26 auth(default): Error: ldap_parse_result Nov 01 09:13:26 auth(default): Error: ldap_first_attribute Nov 01 09:13:26 auth(default): Error: ldap_get_values Nov 01 09:13:26 auth(default): Error: ldap_next_attribute Nov 01 09:13:26 auth(default): Error: ldap_msgfree Nov 01 09:13:26 auth(default): Error: ldap_result ld 0x18611700 msgid -1 Nov 01 09:13:26 auth(default): Error: wait4msg ld 0x18611700 msgid -1 (timeout 0 usec) Nov 01 09:13:26 auth(default): Error: wait4msg continue ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Connections: Nov 01 09:13:26 auth(default): Error: * host: 192.168.0.240 port: 3268 (default) Nov 01 09:13:26 auth(default): Error: refcnt: 1 status: Connected Nov 01 09:13:26 auth(default): Error: last used: Tue Nov 1 09:13:26 2011 Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Outstanding Requests: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 request count 0 (abandoned 0) Nov 01 09:13:26 auth(default): Error: ** ld 0x18611700 Response Queue: Nov 01 09:13:26 auth(default): Error: Empty Nov 01 09:13:26 auth(default): Error: ld 0x18611700 response count 0 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList ld 0x18611700 msgid -1 all 1 Nov 01 09:13:26 auth(default): Error: ldap_chkResponseList returns ld 0x18611700 NULL Nov 01 09:13:26 auth(default): Error: ldap_int_select dovecot_info.log Nov 01 09:09:48 dovecot: Info: Dovecot v1.2.17 starting up Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99773 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99774 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99775 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99776 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99777 Nov 01 09:09:49 auth(default): Info: new auth connection: pid=99778 Nov 01 09:10:09 auth(default): Info: new auth connection: pid=99782 Nov 01 09:10:13 auth(default): Info: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=993 rport=55010 resp=AGRhdmlkdgBlbGkqNDJ0dW4= Nov 01 09:10:13 auth(default): Info: ldap(davidv,127.0.0.1): pass search: base=dc=smallmountain,dc=Local scope=subtree filter=(&(objectClass=person)(sAMAccountName=davidv)) fields=mail Nov 01 09:10:13 auth(default): Info: auth(davidv,127.0.0.1): username changed davidv -> davidv at smallmountain.net Nov 01 09:10:13 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): result: mail(user)=davidv at smallmountain.net Nov 01 09:10:13 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): No password returned (and no nopassword) Nov 01 09:10:15 auth(default): Info: client out: FAIL 1 user=davidv at smallmountain.net Nov 01 09:13:13 imap-login: Info: Disconnected: Inactivity (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Nov 01 09:13:21 auth(default): Info: new auth connection: pid=99797 Nov 01 09:13:26 auth(default): Info: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=993 rport=61080 resp=AGRhdmlkdgBlbGkqNDJ0dW4= Nov 01 09:13:26 auth(default): Info: ldap(davidv,127.0.0.1): pass search: base=dc=smallmountain,dc=Local scope=subtree filter=(&(objectClass=person)(sAMAccountName=davidv)) fields=mail Nov 01 09:13:26 auth(default): Info: auth(davidv,127.0.0.1): username changed davidv -> davidv at smallmountain.net Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): result: mail(user)=davidv at smallmountain.net Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): No password returned (and no nopassword) Nov 01 09:13:28 auth(default): Info: client out: FAIL 1 user=davidv at smallmountain.net Nov 01 09:13:36 imap-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Your assistance is greatly appreciated, David From leo at strike.wu.ac.at Tue Nov 1 22:12:14 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Tue, 01 Nov 2011 21:12:14 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EAEDDDD.40206@gmx.de> References: <4EAED444.70608@strike.wu.ac.at> <4EAEDDDD.40206@gmx.de> Message-ID: <4EB0529E.2090808@strike.wu.ac.at> On 10/31/2011 06:41 PM, e-frog wrote: > On 31.10.2011 18:00, wrote Alexander 'Leo' Bergolth: >> Have environment based namespace settings been abandoned in 2.0? >> > I think you need to add them to USERDB_KEYS now... > > http://wiki2.dovecot.org/PostLoginScripting#Running_environment I have added NAMESPACE_1 NAMESPACE_1_SEP NAMESPACE_1_TYPE NAMESPACE_1_LIST NAMESPACE_1_PREFIX to USERDB_KEYS. Unfortunately the namespace still isn't used. :-( --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From pgnet.dev+rich-dovecot at gmail.com Tue Nov 1 22:53:47 2011 From: pgnet.dev+rich-dovecot at gmail.com (Rich) Date: Tue, 1 Nov 2011 13:53:47 -0700 Subject: [Dovecot] looking for Dovecot-code + SQL consultants Message-ID: Hi, We're using Dovecot2. Trying, given our own spread-too-thin bandwidth, to make it work within our evolving SQL application environment. When there's a problem, we post to this list (e.g., http://www.dovecot.org/list/dovecot/2011-October/061609.html), but aren't getting any/timely responses. We've decided to look for a consultant (hourly or retainer) that can be available for working with our in-house staff to straighten these issues out -- by helping us identify & fix our own mess, and by working to get fixes pushed to Dovecot project code, where appropriate. If you provide these services, rather than simply deployment or hosting, and are available, please drop me a line *offlist*. We're in the San Francisco area, and local is best, but remote work is certainly an option. Thanks, Rich From ghandidrivesahumvee at rocketfish.com Tue Nov 1 22:57:05 2011 From: ghandidrivesahumvee at rocketfish.com (Dovecot-GDH) Date: Tue, 1 Nov 2011 13:57:05 -0700 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EA916D1.30409@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: If I/O performance is a concern, you may be interested in ZFS and Flashcache. Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) ZFS does run on Linux http://zfs-fuse.net Flashcache: https://github.com/facebook/flashcache/ Both of these techniques can use a pair of SSDs in RAID1 rather than a single SSD. On Oct 27, 2011, at 1:31 AM, Ed W wrote: > On 27/10/2011 03:36, Stan Hoeppner wrote: >> On 10/26/2011 4:13 PM, Patrick Westenberg wrote: >>> Hi all, >>> >>> is anyone on this list who dares/dared to store his index files on a >>> MLC-SSD? >> I have not. But I can tell you that a 32GB Corsair MLC SSD in my >> workstation died after 4 months of laughably light duty. It had nothing >> to do with cell life but low product quality. This was my first foray >> into SSD. The RMA replacement is still kickin after 2 months, >> thankfully. I'm holding my breath... >> >> Scanning the reviews on Newegg shows early MLC SSD failures across most >> brands, early being a year or less. Some models/sizes are worse than >> others. OCZ has a good reputation overall, but reviews show some of >> their models to be grenades. >> >> Thus, if you were to put indexes on SSD, you should strongly consider >> using a mirrored pair. >> > > I don't think you are saying that the advice varies here compared with > HDDs? I do agree that some SSDs are showing very early failures, but > it's only a tweak to the probability parameter compared with any other > storage medium. They ALL fail at some point, and generally well within > the life of the rest of the server. Some kind of failure planning is > necessary > > Caveat the potentially higher failures vs HDDs I don't see any reason > why an SSD shouldn't work well? (even more so if you are using maildir > where indexes can be regenerated). > > More interestingly: for small sizes like 32GB, has anyone played with > the "compressed ram with backing store" thing in newer kernels (that I > forget the name of now). I think it's been marketed for swap files, but > assuming I got the theory it could be used as a ram drive with slow > writeback to permanent storage? > > Good luck > > Ed W From pw at wk-serv.de Tue Nov 1 23:19:11 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 01 Nov 2011 22:19:11 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: <4EB0624F.90307@wk-serv.de> Dovecot-GDH schrieb: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was thinking about a SSD based LUN for the indexes. As I'm using multiple servers this LUN will use OCFS2. From dswartz at druber.com Tue Nov 1 23:40:39 2011 From: dswartz at druber.com (Dan Swartzendruber) Date: Tue, 1 Nov 2011 17:40:39 -0400 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB0624F.90307@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com><4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> Message-ID: <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> I can't imagine running any kind of performance critical app on linux using fuse! There is a native ZFS port going on, but I don't know how stable it is yet. -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Patrick Westenberg Sent: Tuesday, November 01, 2011 5:19 PM To: dovecot at dovecot.org Subject: Re: [Dovecot] Indexes to MLC-SSD Dovecot-GDH schrieb: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was thinking about a SSD based LUN for the indexes. As I'm using multiple servers this LUN will use OCFS2. From janfrode at tanso.net Wed Nov 2 10:57:15 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 2 Nov 2011 09:57:15 +0100 Subject: [Dovecot] doveadm delete-user ? Message-ID: <20111102085715.GA4089@dibs.tanso.net> IMHO it would be great if we could both provision and un-provision users without having the provisioning system mess with files in the filesystem. Provisioning new users works fine since dovecot will create all files/directories on first login, but I haven?t found the tools to un-provison them. I see "doveadm mailbox delete" can delete individual mailboxes, but what about indexes, sieve-scripts and home-directories? Are there any tools to clean out these ? -jf From goetz.reinicke at filmakademie.de Wed Nov 2 15:59:21 2011 From: goetz.reinicke at filmakademie.de (=?ISO-8859-15?Q?G=F6tz_Reinicke?=) Date: Wed, 02 Nov 2011 14:59:21 +0100 Subject: [Dovecot] Upgrading dovecot 1.0.7 -> 2.0.14 / Redhat EL 5.7 Message-ID: <4EB14CB9.9030708@filmakademie.de> Hi, recently I debugged some load and performance and 'log out' problems and found an error in the dovecot logs: Error: pipe() failed: Too many open files Googling around and reading some posts led to the conclusion, that my problem is a bug, which will not be removed. So I'm thinking of upgrading to the latest prebuild rpm from here: http://packages.atrpms.net/dist/el5/dovecot/ As far as I understand the upgrading instructions, I'll be save as we dont use special setting, but I'll check all changes. My question: Dose anybody has done such an upgrade yet? Or are there any suggestion or hints? Thanks a lot and best regards. G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5161 bytes Desc: S/MIME Kryptografische Unterschrift URL: From arif at team.de.inter.net Wed Nov 2 16:28:05 2011 From: arif at team.de.inter.net (Kai Arif) Date: Wed, 02 Nov 2011 15:28:05 +0100 Subject: [Dovecot] Different default realm per listener Message-ID: <4EB15375.30106@team.de.inter.net> Hi all! I want create a large IMAP server setup where dovecot is listening on multiple interfaces and needs a different default realm on each interface. I'm using dovecot 2.0.5 on FreeBSD9 and authenticate against a LDAP database. I found an old mail thread from 2007 that said I'm basically screwed. Has that change since than? Is there any way to use different default realms per listener? Best regards Kai -- Kai Arif - System Administrator Inter.net Germany GmbH Knesebeckstra?e 59-61 10719 Berlin Germany Fon +49 30 25430 0 Fax +49 30 25430 499 arif at team.de.inter.net www.de.inter.net Legal-Information: Inter.net Germany GmbH, HRB #79136, Amtsgericht Berlin Charlottenburg, UST-IdNr:: DE 813 165 159 FA f?r K?rperschaften I Berlin, Gesch?ftsf?hrer: J?rn Lubkoll Zust?ndige Regulierungsbeh?rde: Bundesnetzagentur Chemnitz, Liselotte-Herrmann-Str. 20a, 09127 Chemnitz, Registriernummer: 06/164 From rick at havokmon.com Wed Nov 2 16:37:47 2011 From: rick at havokmon.com (Rick Romero) Date: Wed, 02 Nov 2011 09:37:47 -0500 Subject: [Dovecot] IMP, Dovecot and multiple namespaces Message-ID: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> Hey guys, I'm wondering what the best way is to be backwards compatible with Courier mailbox formats and not duplicate mailbox trees with Dovecot. Is anyone doing this right now? My Dovecot 2.0.13 is configured as follows for Namespaces: namespace { inbox = yes location = prefix = separator = . } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } So I should have an INBOX, plus additional root folders - rather than the Courier default of all folders being subfolders of the INBOX. So when I run LIST in IMAP I get - 08 LIST "" % * LIST (\HasChildren) "." "INBOX" * LIST (\HasChildren) "." "Spam" * LIST (\HasNoChildren) "." "Sent" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "Junk" * LIST (\HasNoChildren) "." "Drafts" Which looks fine. The problem is, IMP (old and new) show two sets of mailboxes (one for each namespace) like so: INBOX Spam Spam.Over10 Sent INBOX.Spam INBOX.Spam.Over10 INBOX.Sent (etc) I assume IMP is walking the INBOX because it says it has Children, and displaying those Children - The Horde team is telling me that's the root of the problem. It makes sense, but I'm not sure what the real solution is. Is it possible to disable Children for a mailbox when an alternate hidden namespace matches the mailbox name? Does that make sense? Or, is there another option I change change in the dovecot config to accomplish the same? Rick From jtam.home at gmail.com Thu Nov 3 02:43:04 2011 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 2 Nov 2011 17:43:04 -0700 (PDT) Subject: [Dovecot] doveadm delete-user ? In-Reply-To: References: Message-ID: Jan-Frode Myklebust writes: > IMHO it would be great if we could both provision and un-provision users > without having the provisioning system mess with files in the > filesystem. Provisioning new users works fine since dovecot will create > all files/directories on first login, but I haven?t found the tools to > un-provison them. > I see "doveadm mailbox delete" can delete individual > mailboxes, but what about indexes, sieve-scripts and home-directories? > Are there any tools to clean out these ? I coundn't find any, so I wrote my own executable cron job that scans the index cache and checks if the corresponding mailbox exists. If not, it echo the commands to remove the stale indices. From there, you can review it, or pipe it into sh. I can post it if there's any interest, but it's not that hard to write. This is only necessary, of course, if you store the indices in a different place than the mail folder. Joseph Tam From scott_the_musician at yahoo.com.au Thu Nov 3 07:31:54 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Wed, 2 Nov 2011 22:31:54 -0700 (PDT) Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? Message-ID: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Hi all, I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. Here's what happens: [root at mail ~]# whereis dovecot dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot ... checking whether to build static libraries... yes dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH to give path to compiled Dovecot sources or to a directory with the installed dovecot-config file. configure: error: dovecot-config not found -- I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: Avelsieve plugin details: backend = ManageSieve ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? thanks in advance! From lists at wildgooses.com Thu Nov 3 10:29:32 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 08:29:32 +0000 Subject: [Dovecot] Thunderbird slow in talking with dovecot IMAP AND to sendmail In-Reply-To: <4EA68C11.5060102@tlinx.org> References: <4EA68C11.5060102@tlinx.org> Message-ID: <4EB250EC.90204@wildgooses.com> On 25/10/2011 11:14, Linda Walsh wrote: > > > I'm trying to find out what's causing this slowdown -- it's > INTOLERABLE.... > > over 1 minute and less than 1% done. (400MB file)... > > After trying 3 times, I gave up and logged in using X to the server > and ran Tbird from there.... > > Mail sent out in < 1 minute, though the copy to dovecot took about 50% > longer. > > So... > > I looked at the network trace. > > and everyfrackin' body was using 4K packet sizes (at the application > level!, the window size on TCP was over 64K...but no one was using > it)....especially galling with my network's MTU at 9K, BTW, because > small packets are really bad on a 1Gb network. > Although larger packets might be helpful, I don't see that you shouldn't be getting much faster speed without it? Even the 64K window, whilst it looks too small, might be ok if your ping times are very low? Something else is limiting your performance I think? Ed W From maria_arrea at gmx.com Thu Nov 3 11:54:24 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 10:54:24 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103095424.67160@gmx.com> Hello. We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 Nov 3 09:23:07 buzon dovecot: imap(delolmo at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/15/77/delolmo/subscriptions Nov 3 09:23:07 buzon dovecot: imap(delolmo at mydomain): Connection closed bytes=0/295 I have been googling but I only see problems with remote NFS, our setup does not use NFS. I give you doveconf -n output & mount options, if more info es needed, please ask. doveconf -n output # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.el5 x86_64 Red Hat Enterprise Linux Server release 5.7 (Tikanga) ext4 auth_cache_negative_ttl = 10 secs auth_cache_size = 10 M auth_cache_ttl = 2 mins auth_master_user_separator = * auth_mechanisms = plain login auth_worker_max_count = 3500 base_dir = /var/run/dovecot/ default_client_limit = 5000 default_process_limit = 6500 disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep delay-newmail tb-lsub-flags lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_fsync = never mail_gid = entrega mail_home = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/ mail_location = mdbox:/buzones/mydomain/%2.26Hn/%2.200Hn/%n:INDEX=/indices_dovecot/indices/%2.26Hn/%2.200Hn/%n mail_max_userip_connections = 15000 mail_plugins = " zlib acl quota autocreate" mail_uid = entrega managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mdbox_rotate_interval = 1 days mdbox_rotate_size = 60 M passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/usuario_maestro.txt driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile autocreate = SPAM autocreate2 = Sent autocreate3 = Drafts autocreate4 = Trash autosubscribe = SPAM autosubscribe2 = Sent autosubscribe3 = Drafts autosubscribe4 = Trash lda_mailbox_autosubscribe = yes quota = dict:Cuota de usuario::file:/buzones/cuotas/%n quota_rule2 = Trash:storage=+10%% quota_warning = storage=90%% aviso_cuota 90 %u sieve = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/dovecot.sieve sieve_dir = /buzones/mydomain/%2.26Hn/%2.200Hn/%n/home_usuario/sieve/ zlib_save = gz zlib_save_level = 9 } pop3_no_flag_updates = yes protocols = pop3 imap sieve service anvil { client_limit = 25000 } service auth { client_limit = 28000 unix_listener auth-master { user = entrega } unix_listener auth-userdb { user = entrega } user = root } service aviso_cuota { executable = script /usr/local/bin/quota-warning.sh unix_listener aviso_cuota { mode = 0666 } user = entrega } service imap-login { executable = /usr/libexec/dovecot/imap-login group = dovenull service_count = 0 } service imap { executable = /usr/libexec/dovecot/imap process_limit = 6000 } service managesieve-login { executable = /usr/libexec/dovecot/managesieve-login inet_listener sieve { port = 2000 } process_limit = 2000 } service managesieve { executable = /usr/libexec/dovecot/managesieve process_limit = 5000 } service pop3-login { executable = /usr/libexec/dovecot/pop3-login process_limit = 4000 service_count = 0 } service pop3 { executable = /usr/libexec/dovecot/pop3 process_limit = 4000 } ssl_ca = References: <20111103095424.67160@gmx.com> Message-ID: <4EB26C2F.1050505@wildgooses.com> Hi > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:30:09 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:30:09 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103103009.67160@gmx.com> We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:25 AM To: dovecot at dovecot.org Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Hi > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W From lists at wildgooses.com Thu Nov 3 12:31:21 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:31:21 +0000 Subject: [Dovecot] Imap/pop gateway In-Reply-To: References: Message-ID: <4EB26D79.1010206@wildgooses.com> On 31/10/2011 22:20, nuno marques wrote: > > > > Hello, > How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. > The question isn't entirely clear, but I *think* you just want to use the normal "proxy" feature of dovecot. This accepts connections on one machine, examines them until the end of the auth stage and passes them onto some other machine based on the results of the auth process Also there are other imap/pop proxies such as nginx That said I'm not sure how much security this really buys you versus port forwarding POP/IMAP ports to your real server? If the proxy machine were to get hacked (over imap?) then the same hack can jump from the proxy to the real server. Also your only exposure in each case is via POP/IMAP, which means you would be mainly chasing buffer overflow vulnerabilities and the like. These can also be mitigated by chrooting the server machine (please consider virtualisation options, it's usually simpler/faster/saner, eg see my favourite: linux-vservers), MAC controls on the dovecot process (grsec/selinux, etc), and compiler extensions (gcc hardened) Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:35:46 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:35:46 +0100 Subject: [Dovecot] Imap/pop gateway Message-ID: <20111103103546.67180@gmx.com> If you are going to use an imap proxy for security reasons, consider using a software DIFFERENT than in your real mailboxes. If you use dovecot in your backend, you could use perdition in the frontend. Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:31 AM To: Dovecot Mailing List Subject: Re: [Dovecot] Imap/pop gateway On 31/10/2011 22:20, nuno marques wrote: > > > > Hello, > How can i make a imap/pop gateway? that is, putting the mailboxes on a server on the internal network and put the gateway in the dmz. > The question isn't entirely clear, but I *think* you just want to use the normal "proxy" feature of dovecot. This accepts connections on one machine, examines them until the end of the auth stage and passes them onto some other machine based on the results of the auth process Also there are other imap/pop proxies such as nginx That said I'm not sure how much security this really buys you versus port forwarding POP/IMAP ports to your real server? If the proxy machine were to get hacked (over imap?) then the same hack can jump from the proxy to the real server. Also your only exposure in each case is via POP/IMAP, which means you would be mainly chasing buffer overflow vulnerabilities and the like. These can also be mitigated by chrooting the server machine (please consider virtualisation options, it's usually simpler/faster/saner, eg see my favourite: linux-vservers), MAC controls on the dovecot process (grsec/selinux, etc), and compiler extensions (gcc hardened) Good luck Ed W From giulio at dsi.unimi.it Thu Nov 3 12:38:17 2011 From: giulio at dsi.unimi.it (Giulio Casella) Date: Thu, 03 Nov 2011 11:38:17 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103103009.67160@gmx.com> References: <20111103103009.67160@gmx.com> Message-ID: <4EB26F19.1080105@dsi.unimi.it> This could be the problem. Double check the time also on your host system(s), not only on guest. Bye, gc Il 03/11/2011 11.30, Maria Arrea ha scritto: > We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:25 AM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > Hi> We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W > -- Giulio Casella giulio at dsi.unimi.it System and network manager Computer Science Dept. - University of Milano From lists at wildgooses.com Thu Nov 3 12:42:36 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:42:36 +0000 Subject: [Dovecot] How can we horizontally scale Dovecot across multiple servers? In-Reply-To: References: <003001cc97a9$addbeb80$0993c280$@mvs.us> <4EAE62AA.6080203@schetterer.org> <000301cc97b1$81100ca0$833025e0$@mvs.us> <4EAE71C1.7060400@schetterer.org> Message-ID: <4EB2701C.6060706@wildgooses.com> On 31/10/2011 11:28, Felipe Scarel wrote: > Quick question about the usage of DRBD: I'm thinking of a setup on my > organization here (15k+ users, 4TB of email data), but I'm holding back on > the clusterization due to the high volume of data. > > Using DRBD would implicate mirroring those 4TB of data across all cluster > nodes? If yes, I might go with a SAN-based solution, though I haven't I don't the technique with DRBD is something like having pairs of machines, each of which is a backup for the other. There were some old notes on the Dovecot website about such a setup? Roughly I seem to recall that each pair of machines ran two virtual machines, each of which ran active on one of the nodes each, but could migrate to the other if needed. Add a bunch of such paired nodes to get to the performance you require and put a dovecot proxy instance in front of the whole lot In contrast the SAN solution uses a clustered filesystem (opinion varies on which performs best) and then in theory every machine has access to every mailbox. In practice access to the SAN is relatively slow compared with local storage, so the technique seems to be to store indexes on the local machine and then using the front end proxy to be somewhat "sticky" in returning users to the same backend node so that the indexes can be re-used and not rebuilt The DRBD solution offers local disk access speed to the node and would on the surface give far faster performance (if disk were the limiting issue). However, it's likely to be more complex to maintain and manage and without buying licences you get only failover between pairs of machines. The SAN solution in theory looks like perfect scale up, big backend and just add more backend IMAP nodes as you need them, and all the clever stuff moves to the frontend load balancer to be "sticky" and obviously that's your main maintenance problem. However, based on evidence from users of big systems, IO is likely to be your main bottleneck and so just theoretically, the SAN will only scale as far as it doesn't run out of IOs... Using local disk for indexes would tend to reduce the amount of IOs needed (from the SAN) very dramatically, but you still have some limit out there and it's a question of whether you will reach it? DRBD has theoretical infinite scale out because each time you add another pair you get more IO as well as more CPU I don't have the fortune to have anything like the volume of users you have so I have no opinion to offer... However, I think the above accurately summarises your options. Others might help clarify the likely bounds on performance of each solution and maintenance headaches (eg some have had problems with maildir mounted on OCFS/GFS2 and fixed that by moving to dbox, etc) Please report on your results! Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 12:49:04 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 11:49:04 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103104904.67150@gmx.com> All the ESXs hosts and all the VM use the same NTP server. Any other idea? Regards Maria ----- Original Message ----- From: Giulio Casella Sent: 11/03/11 11:38 AM To: dovecot at dovecot.org Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube This could be the problem. Double check the time also on your host system(s), not only on guest. Bye, gc Il 03/11/2011 11.30, Maria Arrea ha scritto: > We use ntpd daemon, all our systems are configured equal. Another thing, this is VM on vmware vsphere 4.1 > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:25 AM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > Hi> We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Quick check, but by "NTP" you mean the background daemon and you don't have some cron job running ntpdate or similar every so often? No idea, but since it looks like a clock related curiousity, then knowing if the clock is spot on accurate or drifting would be interesting to know? Simple comparison against other machines over a similar period to you having problems might be accurate enough? Good luck Ed W > -- Giulio Casella giulio at dsi.unimi.it System and network manager Computer Science Dept. - University of Milano From stephan at rename-it.nl Thu Nov 3 12:54:13 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 03 Nov 2011 11:54:13 +0100 Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Message-ID: <4EB272D5.1090403@rename-it.nl> Op 3-11-2011 6:31, Scott Lewis schreef: > Hi all, > > I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. > > Here's what happens: > > [root at mail ~]# whereis dovecot > dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz > > [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot > > ... > > checking whether to build static libraries... yes > dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH > to give path to compiled Dovecot sources or to a directory with the > installed dovecot-config file. configure: error: dovecot-config not found > > -- > > I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I'm not familiar with CentOS, but there usually is a separate package containing the Dovecot development headers and the dovecot-config file you need. By the looks of things, that is not installed at your end. Point the --with-dovecot to wherever the dovecot-config file is installed. Regards, Stephan. From lists at wildgooses.com Thu Nov 3 12:57:37 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 10:57:37 +0000 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103104904.67150@gmx.com> References: <20111103104904.67150@gmx.com> Message-ID: <4EB273A1.4030106@wildgooses.com> On 03/11/2011 10:49, Maria Arrea wrote: > All the ESXs hosts and all the VM use the same NTP server. > > Any other idea? > Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W From maria_arrea at gmx.com Thu Nov 3 13:03:36 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Thu, 03 Nov 2011 12:03:36 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111103110336.67190@gmx.com> We follow the guidelines about timekeeping RHEL in vmware vsphere located here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 These problems happens in peak hours. Any dovecot config parameter I could set to mitigate this problem? Regards Maria ----- Original Message ----- From: Ed W Sent: 11/03/11 11:57 AM To: Maria Arrea, Dovecot Mailing List Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube On 03/11/2011 10:49, Maria Arrea wrote: > All the ESXs hosts and all the VM use the same NTP server. > > Any other idea? > Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W From fbscarel at gmail.com Thu Nov 3 13:32:34 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Thu, 3 Nov 2011 09:32:34 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> Message-ID: I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ users, over 2TB of mail data) with little issues. Dedup and compression disabled, mind that. Dedup especially is a major source of trouble, I wouldn't recommend it for production just yet. Cheers, fbscarel On Tue, Nov 1, 2011 at 19:40, Dan Swartzendruber wrote: > > I can't imagine running any kind of performance critical app on linux using > fuse! There is a native ZFS port going on, but I don't know how stable it > is yet. > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On > Behalf Of Patrick Westenberg > Sent: Tuesday, November 01, 2011 5:19 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] Indexes to MLC-SSD > > Dovecot-GDH schrieb: > > If I/O performance is a concern, you may be interested in ZFS and > Flashcache. > > > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive > Read Cache) > > ZFS does run on Linux http://zfs-fuse.net > > I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was > thinking about a SSD based LUN for the indexes. As I'm using multiple > servers this LUN will use OCFS2. > > From lists at wildgooses.com Thu Nov 3 16:27:16 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 14:27:16 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> Message-ID: <4EB2A4C4.2080100@wildgooses.com> On 03/11/2011 11:32, Felipe Scarel wrote: > I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ > users, over 2TB of mail data) with little issues. Dedup and compression > disabled, mind that. > OT: but what were the rough criteria that led you to using ZFS over say LVM with EXT4/XFS/btrfs? I can think of plenty for/against reasons for each, just wondering what criteria affected *your* situation? I'm guessing some kind of manageability reason is at the core, but perhaps you can expand on how it's all worked out for you? I have a fairly static server setup here so I have been "satisfied" with LVM, software raid and mainly ext4. The main thing I miss is simple to use snapshots Cheers Ed W From lists at wildgooses.com Thu Nov 3 17:21:01 2011 From: lists at wildgooses.com (Ed W) Date: Thu, 03 Nov 2011 15:21:01 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB0624F.90307@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> Message-ID: <4EB2B15D.1010005@wildgooses.com> > I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was > thinking about a SSD based LUN for the indexes. As I'm using multiple > servers this LUN will use OCFS2. Given that the SAN always has the network latency behind it, might you be better to look at putting the SSDs in the frontend machines? Obviously this then needs some way to make users "sticky" to one machine (or some few machines) where the indexes are stored? This seems theoretically likely to give you higher IOPs to the index than having them on the OCFS2 storage? (At a trade off with more complexity for the load balancer front end...) Ed W From alietsantiesteban at gmail.com Thu Nov 3 17:52:24 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Thu, 3 Nov 2011 11:52:24 -0400 Subject: [Dovecot] How to define ldap connection idle Message-ID: I'm having a problem with dovecot ldap connection when ldap server is in another firewall zone, firewall kills the ldap connection after a determined period of inactivity, this is good from the firewall point of view but is bad for dovecot because it never knows the connections has been dropped, this creates longs timeouts in dovecot and finally it reconnects, meanwhile many users fails to authenticate, I have seen this kind of post in the list for a while but can't find a solution for it, so my question is how to define a idle ldap time in dovecot so it can reconnect before the firewall has dropped the connection or just close the connection under inactivity so when a user authenticate doesn't fails for a while until dovecot detects that the connection has hanged. Is this a feature request or there is already a configuration for this??? Thank's in advance and congrats tu Timo for this great app. From pgnet.dev+rich-dovecot at gmail.com Thu Nov 3 18:37:13 2011 From: pgnet.dev+rich-dovecot at gmail.com (Rich) Date: Thu, 3 Nov 2011 09:37:13 -0700 Subject: [Dovecot] looking for Dovecot-code + SQL consultants In-Reply-To: References: Message-ID: Hi, I've already received a number of replies from providers offering to help out. I'll be in touch with each, and am certain we'll be able to find the right solution from among them. Thanks for the responses, Rich On Tue, Nov 1, 2011 at 1:53 PM, Rich wrote: > Hi, > > We're using Dovecot2. ?Trying, given our own spread-too-thin > bandwidth, to make it work within our evolving SQL application > environment. > > When there's a problem, we post to this list (e.g., > http://www.dovecot.org/list/dovecot/2011-October/061609.html), but > aren't getting any/timely responses. > > We've decided to look for a consultant (hourly or retainer) that can > be available for working with our in-house staff to straighten these > issues out -- by helping us identify & fix our own mess, and by > working to get fixes pushed to Dovecot project code, where > appropriate. > > If you provide these services, rather than simply deployment or > hosting, and are available, please drop me a line *offlist*. ?We're in > the San Francisco area, and local is best, but remote work is > certainly an option. > > Thanks, > > Rich > From pw at wk-serv.de Thu Nov 3 18:53:24 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 03 Nov 2011 17:53:24 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2B15D.1010005@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> Message-ID: <4EB2C704.1010107@wk-serv.de> Ed W schrieb: >> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >> thinking about a SSD based LUN for the indexes. As I'm using multiple >> servers this LUN will use OCFS2. > > Given that the SAN always has the network latency behind it, might you > be better to look at putting the SSDs in the frontend machines? > Obviously this then needs some way to make users "sticky" to one machine > (or some few machines) where the indexes are stored? Storing the indexes on several machines? In this case I have to synchronize them. From dswartz at druber.com Thu Nov 3 18:56:45 2011 From: dswartz at druber.com (Dan Swartzendruber) Date: Thu, 03 Nov 2011 12:56:45 -0400 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2C704.1010107@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> Message-ID: <4EB2C7CD.2090707@druber.com> Patrick Westenberg wrote: > Ed W schrieb: > >>> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >>> thinking about a SSD based LUN for the indexes. As I'm using multiple >>> servers this LUN will use OCFS2. >> >> Given that the SAN always has the network latency behind it, might you >> be better to look at putting the SSDs in the frontend machines? >> Obviously this then needs some way to make users "sticky" to one machine >> (or some few machines) where the indexes are stored? > > Storing the indexes on several machines? > In this case I have to synchronize them. > maybe i am missing something. if a client has to fetch the index, the server has to read the index from disk and pass it back. the network latency is unavoidable, but i don't see why putting the fastest possible SSD on the server isn't a win. possibly i am misunderstanding something? From thierry at mailhub.co.za Thu Nov 3 20:13:18 2011 From: thierry at mailhub.co.za (Thierry de Montaudry) Date: Thu, 3 Nov 2011 19:13:18 +0100 Subject: [Dovecot] Restricting IMAP access Message-ID: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Hi list, I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). Anybody has a trick for that? Running dovecot 2.0.13. I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. Regards, Thierry From fbscarel at gmail.com Thu Nov 3 20:24:06 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Thu, 3 Nov 2011 16:24:06 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2A4C4.2080100@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> Message-ID: Reasons to choose ZFS were snapshots, and mainly dedup and compression capabilities. I know, it's ironic since I'm not able to use them now due to severe performance issues with them (mostly dedup) turned on. I do like the emphasis on data integrity and fast on-the-fly configurability of ZFS to an extent, but I wouldn't recommend it highly for new users, especially for production. It works (in fact it's working right now), but has its fair share of troubles. We've started implementations to move our mail system to a more modular enviroment and we'll probably move away from ZFS. Was a nice experiment nonetheless, I learned quite a bit from it. On Thu, Nov 3, 2011 at 12:27, Ed W wrote: > On 03/11/2011 11:32, Felipe Scarel wrote: > > I'm using native ZFS (http://zfsonlinux.org) on production here (15k+ > > users, over 2TB of mail data) with little issues. Dedup and compression > > disabled, mind that. > > > > OT: but what were the rough criteria that led you to using ZFS over say > LVM with EXT4/XFS/btrfs? I can think of plenty for/against reasons for > each, just wondering what criteria affected *your* situation? I'm > guessing some kind of manageability reason is at the core, but perhaps > you can expand on how it's all worked out for you? > > I have a fairly static server setup here so I have been "satisfied" with > LVM, software raid and mainly ext4. The main thing I miss is simple to > use snapshots > > Cheers > > Ed W > From robert at schetterer.org Thu Nov 3 20:46:16 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 03 Nov 2011 19:46:16 +0100 Subject: [Dovecot] Restricting IMAP access In-Reply-To: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> References: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Message-ID: <4EB2E178.8060103@schetterer.org> Am 03.11.2011 19:13, schrieb Thierry de Montaudry: > Hi list, > > I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. > I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). > Anybody has a trick for that? Running dovecot 2.0.13. > > I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. > > Regards, > > Thierry i am shot in time , with a databse this should be get to work i have it as flag for all users, so i can forbid imap to special ones as far i remember ther should be examples on the dovecot site , and it was written about here on the list before -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From simon.brereton at buongiorno.com Thu Nov 3 22:42:40 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 3 Nov 2011 16:42:40 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN Message-ID: Hi Could someone explain to me the difference between LOGIN and PLAIN? I've been googling for a while, but haven't found anything. Thanks. Simon From p at state-of-mind.de Thu Nov 3 23:00:24 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 3 Nov 2011 22:00:24 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <20111103210023.GB2312@state-of-mind.de> * Simon Brereton : > Could someone explain to me the difference between LOGIN and PLAIN? In SMTP these are: Both - are plaintext mechanisms. - base64 encode identification data before they send it over the wire - do not encrypt the indentification data and should therefore only be offered over an encrypted transport layer PLAIN - is an open standard supported by most clients - sends identification data as one string - sends an authentication ID, an authorization ID and the password LOGIN - is a proprietary standard supported by Microsofts clients - sends LOGIN, login name, password and optionally the domain name one after another I guess they are basically the same in IMAP, but others will know better. p at rick > I've been googling for a while, but haven't found anything. > > Thanks. > > Simon -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From stephan at rename-it.nl Thu Nov 3 23:01:17 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 03 Nov 2011 22:01:17 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <4EB3011D.80108@rename-it.nl> On 11/3/2011 9:42 PM, Simon Brereton wrote: > Hi > > Could someone explain to me the difference between LOGIN and PLAIN? > I've been googling for a while, but haven't found anything. The LOGIN SASL mechanism is an obsolete plain text mechanism. It is documented here: http://tools.ietf.org/html/draft-murchison-sasl-login-00 Some clients still support it, but I would not recommend using it when PLAIN or a better SASL mechanism is also available at both ends. The PLAIN mechanism is documented here: http://tools.ietf.org/html/rfc4616 The main technical difference between the two is that the PLAIN mechanism transfers both username and password in a single SASL interaction, where LOGIN needs two. The PLAIN mechanism also provides support for having an authorization id different from the authentication id, allowing for master user login for example. Regards, Stephan. From dovecot.user at seibercom.net Thu Nov 3 23:17:19 2011 From: dovecot.user at seibercom.net (Jerry) Date: Thu, 3 Nov 2011 17:17:19 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: Message-ID: <20111103171719.55e0a7b9@scorpio> On Thu, 3 Nov 2011 16:42:40 -0400 Simon Brereton articulated: > Hi > > Could someone explain to me the difference between LOGIN and PLAIN? > I've been googling for a while, but haven't found anything. You could start here for some basic information: http://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer http://wiki.dovecot.org/Authentication/Mechanisms -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From simon.brereton at buongiorno.com Thu Nov 3 23:28:28 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 3 Nov 2011 17:28:28 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: <4EB3011D.80108@rename-it.nl> References: <4EB3011D.80108@rename-it.nl> Message-ID: On 3 November 2011 17:01, Stephan Bosch wrote: > On 11/3/2011 9:42 PM, Simon Brereton wrote: >> >> Hi >> >> Could someone explain to me the difference between LOGIN and PLAIN? >> I've been googling for a while, but haven't found anything. > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > documented here: > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > Some clients still support it, but I would not recommend using it when PLAIN > or a better SASL mechanism is also available at both ends. The PLAIN > mechanism is documented here: > > http://tools.ietf.org/html/rfc4616 > > The main technical difference between the two is that the PLAIN mechanism > transfers both username and password in a single SASL interaction, where > LOGIN needs two. The PLAIN mechanism also provides support for having an > authorization id different from the authentication id, allowing for master > user login for example. Thanks to both of you. Can I bet that Outlook doesn't support anything but plain? I'm not sure I've ever heard of a client supporting other than Evolution supporting MD5 passwords.. Simon From stan at hardwarefreak.com Fri Nov 4 01:01:29 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 03 Nov 2011 18:01:29 -0500 Subject: [Dovecot] How can we horizontally scale Dovecot across multiple servers? In-Reply-To: <003b01cc9565$c3b86960$4b293c20$@mvs.us> References: <003b01cc9565$c3b86960$4b293c20$@mvs.us> Message-ID: <4EB31D49.9090009@hardwarefreak.com> On 10/28/2011 6:35 AM, Arlin wrote: > How can we horizontally scale Dovecot across multiple servers? Do we require > to install independent instances of Dovecot on each server? The performance limiting factors for IMAP servers are disk IO, memory size, network bandwidth/latency, and processor power, usually in that order. To scale Dovecot horizontally would imply multiple servers each with independent storage handling a subset of the total concurrent user load. > We are planning to use a NAS/SAN device using ZFS or EFS for email storage. > Each logical unit will be of 10TB and similarly as the no: of user increases > we are planning to add multiple 10TB units. This does not describe horizontal maildir performance scaling. This simply describes vertical scaling of the IO, to a point, as the nodes are all attached to one controller with finite IOPS performance, which is dictated by the speed of the IO ASIC and cache memory bandwidth WRT writes, and aggregate spindle IOPS WRT reads. With this setup, you simply gain multi-node redundancy of the front end servers. However, you will *lose* performance compared to a single equivalent large host due to NFS/cluster filesystem overhead, latency, locking, etc, and your management/troubleshooting burden will be increased. If your goal is simply increased performance and storage expandability, go with a single large host system. I'd only go the cluster route if you believe you need the redundancy. -- Stan From stan at hardwarefreak.com Fri Nov 4 04:16:09 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 03 Nov 2011 21:16:09 -0500 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> Message-ID: <4EB34AE9.6070002@hardwarefreak.com> On 11/3/2011 1:24 PM, Felipe Scarel wrote: > Reasons to choose ZFS were snapshots, and mainly dedup and compression > capabilities. I know, it's ironic since I'm not able to use them now due to > severe performance issues with them (mostly dedup) turned on. > > I do like the emphasis on data integrity and fast on-the-fly > configurability of ZFS to an extent, but I wouldn't recommend it highly for > new users, especially for production. It works (in fact it's working right > now), but has its fair share of troubles. > > We've started implementations to move our mail system to a more modular > enviroment and we'll probably move away from ZFS. Was a nice experiment > nonetheless, I learned quite a bit from it. I find this all very interesting... "Please keep in mind the current 0.5.2 stable release does not yet support a mountable filesystem. This functionality is currently available only in the 0.6.0-rc6 release candidate." https://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz "Uploaded October 14, 2011" So in the past ~two weeks, you converted your 15K+ user production server to ZFS on Linux, as an experiment, and have now decided to change to another filesystem solution, a mere two weeks later? Or am I misinterpreting the date given that 0.6.0-rc6 was released? -- Stan From p at state-of-mind.de Fri Nov 4 00:56:11 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 3 Nov 2011 23:56:11 +0100 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: <4EB3011D.80108@rename-it.nl> Message-ID: <20111103225611.GA14437@state-of-mind.de> * Simon Brereton : > On 3 November 2011 17:01, Stephan Bosch wrote: > > On 11/3/2011 9:42 PM, Simon Brereton wrote: > >> > >> Hi > >> > >> Could someone explain to me the difference between LOGIN and PLAIN? > >> I've been googling for a while, but haven't found anything. > > > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > > documented here: > > > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > > > Some clients still support it, but I would not recommend using it when PLAIN > > or a better SASL mechanism is also available at both ends. The PLAIN > > mechanism is documented here: > > > > http://tools.ietf.org/html/rfc4616 > > > > The main technical difference between the two is that the PLAIN mechanism > > transfers both username and password in a single SASL interaction, where > > LOGIN needs two. The PLAIN mechanism also provides support for having an > > authorization id different from the authentication id, allowing for master > > user login for example. > > Thanks to both of you. Can I bet that Outlook doesn't support > anything but plain? Outlook > 2007 LOGIN, NTLM Outlook 2010 > LOGIN, NTLM2, DIGEST-MD5 > I'm not sure I've ever heard of a client supporting other than > Evolution supporting MD5 passwords.. Two come to mind: mutt, Thunderbird However DIGEST-MD5 has been marked deprecated this summer. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From javierdemiguel at us.es Fri Nov 4 11:52:22 2011 From: javierdemiguel at us.es (=?UTF-8?B?SmF2aWVyIGRlIE1pZ3VlbCBSb2Ryw61ndWV6?=) Date: Fri, 04 Nov 2011 10:52:22 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103110336.67190@gmx.com> References: <20111103110336.67190@gmx.com> Message-ID: <4EB3B5D6.10102@us.es> Same problem here, any hint about a fix or workaround? Regards Javier > We follow the guidelines about timekeeping RHEL in vmware vsphere located here > > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427 > > These problems happens in peak hours. Any dovecot config parameter I could set to mitigate this problem? > > Regards > > Maria > > ----- Original Message ----- > From: Ed W > Sent: 11/03/11 11:57 AM > To: Maria Arrea, Dovecot Mailing List > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > On 03/11/2011 10:49, Maria Arrea wrote:> All the ESXs hosts and all the VM use the same NTP server.> > Any other idea?> Doesn't ESX have issues with the time drifting when certain kernel options are set? Something to do with it rescheduling machines and them not counting idle ticks or something..? Does this problem happen during idle hours or peak hours? I should home in on clock problems... Probably vmware related issues to the kernel you are using? Good luck Ed W > From dovecot.user at seibercom.net Fri Nov 4 01:24:43 2011 From: dovecot.user at seibercom.net (Jerry) Date: Thu, 3 Nov 2011 19:24:43 -0400 Subject: [Dovecot] Difference between LOGIN and PLAIN In-Reply-To: References: <4EB3011D.80108@rename-it.nl> Message-ID: <20111103192443.4e239384@scorpio> On Thu, 3 Nov 2011 17:28:28 -0400 Simon Brereton articulated: > On 3 November 2011 17:01, Stephan Bosch wrote: > > On 11/3/2011 9:42 PM, Simon Brereton wrote: > >> > >> Hi > >> > >> Could someone explain to me the difference between LOGIN and PLAIN? > >> I've been googling for a while, but haven't found anything. > > > > The LOGIN SASL mechanism is an obsolete plain text mechanism. It is > > documented here: > > > > http://tools.ietf.org/html/draft-murchison-sasl-login-00 > > > > Some clients still support it, but I would not recommend using it > > when PLAIN or a better SASL mechanism is also available at both > > ends. The PLAIN mechanism is documented here: > > > > http://tools.ietf.org/html/rfc4616 > > > > The main technical difference between the two is that the PLAIN > > mechanism transfers both username and password in a single SASL > > interaction, where LOGIN needs two. The PLAIN mechanism also > > provides support for having an authorization id different from the > > authentication id, allowing for master user login for example. > > Thanks to both of you. Can I bet that Outlook doesn't support > anything but plain? > > I'm not sure I've ever heard of a client supporting other than > Evolution supporting MD5 passwords.. You would lose the bet. In addition, Claws-Mail supports CRAM-MD5. -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From taeuber at bbaw.de Fri Nov 4 13:04:22 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 12:04:22 +0100 Subject: [Dovecot] doveadm altmove -r Message-ID: <20111104120422.85c93004.taeuber@bbaw.de> Hi there, there is the ubuntu version 2.0.13 ov dovecot installed on our server. But in contrast to http://wiki2.dovecot.org/Tools/Doveadm/Altmove the option -r isn't supported in this version. altmove: invalid option -- 'r' doveadm altmove [-u |-A] [-S ] In which version of dovecot is this option supported? Thanks Lars From taeuber at bbaw.de Fri Nov 4 13:57:31 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 12:57:31 +0100 Subject: [Dovecot] doveadm altmove -r In-Reply-To: <20111104120422.85c93004.taeuber@bbaw.de> References: <20111104120422.85c93004.taeuber@bbaw.de> Message-ID: <20111104125731.5f05d91f.taeuber@bbaw.de> Ok, I've found this option to be introduce in version 2.0.14. Maybe someone (with the permissions) should add this info to the wiki page. Best wishes Lars Am Fri, 4 Nov 2011 12:04:22 +0100 Lars T?uber schrieb: > Hi there, > > there is the ubuntu version 2.0.13 ov dovecot installed on our server. > But in contrast to http://wiki2.dovecot.org/Tools/Doveadm/Altmove > the option -r isn't supported in this version. > > altmove: invalid option -- 'r' > doveadm altmove [-u |-A] [-S ] > > In which version of dovecot is this option supported? > > Thanks > Lars From fbscarel at gmail.com Fri Nov 4 13:57:45 2011 From: fbscarel at gmail.com (Felipe Scarel) Date: Fri, 4 Nov 2011 09:57:45 -0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB34AE9.6070002@hardwarefreak.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <2269D2A7C9E54AEDA60CAAA6E29D610D@manticore> <4EB2A4C4.2080100@wildgooses.com> <4EB34AE9.6070002@hardwarefreak.com> Message-ID: I'm using the GIT version, that 0.5 version is quite a bit outdated. I was not all that worried about using ZFS on this experiment because we do have the old mail storage on ext3 synchronized and ready to switch back, and I could disable dedup and compression on-the-fly if needed (which eventually was). On Fri, Nov 4, 2011 at 00:16, Stan Hoeppner wrote: > On 11/3/2011 1:24 PM, Felipe Scarel wrote: > > Reasons to choose ZFS were snapshots, and mainly dedup and compression > > capabilities. I know, it's ironic since I'm not able to use them now due > to > > severe performance issues with them (mostly dedup) turned on. > > > > I do like the emphasis on data integrity and fast on-the-fly > > configurability of ZFS to an extent, but I wouldn't recommend it highly > for > > new users, especially for production. It works (in fact it's working > right > > now), but has its fair share of troubles. > > > > We've started implementations to move our mail system to a more modular > > enviroment and we'll probably move away from ZFS. Was a nice experiment > > nonetheless, I learned quite a bit from it. > > I find this all very interesting... > > "Please keep in mind the current 0.5.2 stable release does not yet > support a mountable filesystem. This functionality is currently > available only in the 0.6.0-rc6 release candidate." > > https://github.com/downloads/zfsonlinux/zfs/zfs-0.6.0-rc6.tar.gz > > "Uploaded October 14, 2011" > > So in the past ~two weeks, you converted your 15K+ user production > server to ZFS on Linux, as an experiment, and have now decided to change > to another filesystem solution, a mere two weeks later? Or am I > misinterpreting the date given that 0.6.0-rc6 was released? > > -- > Stan > From duihi77 at gmail.com Fri Nov 4 14:17:56 2011 From: duihi77 at gmail.com (Duane Hill) Date: Fri, 4 Nov 2011 07:17:56 -0500 (CDT) Subject: [Dovecot] doveadm altmove -r In-Reply-To: <20111104125731.5f05d91f.taeuber@bbaw.de> References: <20111104120422.85c93004.taeuber@bbaw.de> <20111104125731.5f05d91f.taeuber@bbaw.de> Message-ID: On Fri, 4 Nov 2011, Lars T?uber wrote: > Ok, I've found this option to be introduce in version 2.0.14. > Maybe someone (with the permissions) should add this info to the wiki page. It's in the release notes for v2.0.15 (http://www.dovecot.org/doc/NEWS-2.0): v2.0.15 2011-09-16 Timo Sirainen + doveadm altmove: Added -r parameter to move mails back to primary storage. ... From taeuber at bbaw.de Fri Nov 4 14:41:08 2011 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Fri, 4 Nov 2011 13:41:08 +0100 Subject: [Dovecot] doveadm altmove -r In-Reply-To: References: <20111104120422.85c93004.taeuber@bbaw.de> <20111104125731.5f05d91f.taeuber@bbaw.de> Message-ID: <20111104134108.efa5c27e.taeuber@bbaw.de> Am Fri, 4 Nov 2011 07:17:56 -0500 (CDT) Duane Hill schrieb: > On Fri, 4 Nov 2011, Lars T?uber wrote: > > > Ok, I've found this option to be introduce in version 2.0.14. > > Maybe someone (with the permissions) should add this info to the wiki > > page. > > It's in the release notes for v2.0.15 (http://www.dovecot.org/doc/NEWS-2.0): > > v2.0.15 2011-09-16 Timo Sirainen > > + doveadm altmove: Added -r parameter to move mails back to > primary storage. ... Oh yes this was a typo. Lars From leo at strike.wu.ac.at Fri Nov 4 19:36:16 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Fri, 04 Nov 2011 18:36:16 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EB0529E.2090808@strike.wu.ac.at> References: <4EAED444.70608@strike.wu.ac.at> <4EAEDDDD.40206@gmx.de> <4EB0529E.2090808@strike.wu.ac.at> Message-ID: <4EB42290.8060806@strike.wu.ac.at> On 11/01/2011 09:12 PM, Alexander 'Leo' Bergolth wrote: > On 10/31/2011 06:41 PM, e-frog wrote: >> On 31.10.2011 18:00, wrote Alexander 'Leo' Bergolth: >>> Have environment based namespace settings been abandoned in 2.0? >>> >> I think you need to add them to USERDB_KEYS now... >> >> http://wiki2.dovecot.org/PostLoginScripting#Running_environment > > I have added NAMESPACE_1 NAMESPACE_1_SEP NAMESPACE_1_TYPE > NAMESPACE_1_LIST NAMESPACE_1_PREFIX to USERDB_KEYS. > > Unfortunately the namespace still isn't used. :-( Can anyone confirm that setting namespaces via environment isn't supported in 2.0 anymore? Is there another way to dynamically add namespaces? Thanks, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From apm at one.com Fri Nov 4 21:23:36 2011 From: apm at one.com (Peter Mogensen) Date: Fri, 04 Nov 2011 20:23:36 +0100 Subject: [Dovecot] Corrupted transaction log file Message-ID: <4EB43BB8.6090803@one.com> Hi, I'm experiencing a problem I need some pointers to debug. I'm running Dovecot 2.0.15 and have a client which keeps causing log-entries like: Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: indexid changed 1320419300 -> 1320419441 (sync_offset=0) Nov 4 15:10:42 mail dovecot: imap(test at aaaone.net): Error: Corrupted transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: Invalid transaction log size (4476 vs 4360): /mail/3340444/. TestMails/dovecot.index.log (sync_offset=4476) Nov 4 15:10:42 mail dovecot: imap(test at aaaone.net): Disconnected: IMAP session state is inconsistent, please relogin. bytes=549/100474 The client issues a lot of IMAP connections like this: A1_0 LOGIN "test at aaaone.net" "password" A1_1 ENABLE QRESYNC CONDSTORE * OK Waiting for authentication process to respond.. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. A1_0 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in * ENABLED QRESYNC CONDSTORE A1_1 OK Enabled. A2_0 CREATE "INBOX.folder" A2_0 OK Create completed. A3_0 SUBSCRIBE "INBOX.folder" A3_0 OK Subscribe completed. A4_0 APPEND "INBOX.folder" ($CreatedByImapClient-test) {20487+} ... more similar appends. But it only seems to be reproducible, when there's another client involved. Starting up Thunderbird with a login to the account seems to be enough to cause this. (To my knowledge this causes a series of STATUS command and then IDLE). The installation is using Maildir and lock_method = fcntl on local disks. /Peter From tss at iki.fi Fri Nov 4 21:30:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:30:29 +0200 Subject: [Dovecot] Auth Worker failures In-Reply-To: References: <1318948640.21919.103.camel@hurina> Message-ID: <1320435030.21919.142.camel@hurina> On Thu, 2011-10-20 at 16:24 -0400, Simon Brereton wrote: > Oct 20 06:25:14 mail dovecot: auth-worker(default): > sql(simon at example.net,127.0.0.1): CRYPT() != 'RaNDomsTRinG' CRYPT() means that it attempted to log in with an empty password. > I'm still sure this is the webmail trying to log in though.. Yeah, could be. But why it tries it with an empty password, I've no idea. From tss at iki.fi Fri Nov 4 21:34:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:34:03 +0200 Subject: [Dovecot] Accessing a strange mailbox In-Reply-To: <8762jie6ut.fsf@algae.riseup.net> References: <8762jie6ut.fsf@algae.riseup.net> Message-ID: <1320435243.21919.144.camel@hurina> On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote: > I have a user who has a mailbox called: > > A----->B > > It seemed to work in courier fine, they managed to create it, and there > are mails in it. However, dovecot is not letting the user access it, the > IMAP server gives an error. I tried to rename it but I would also get an > error: > > # doveadm mailbox rename 'A----->B' AtoB > doveadm(root): Error: Can't rename mailbox A----->B to AtoB: Mailbox doesn't exist: A----->B Maybe it's not named that? What does it look like in filesystem? Worked fine with me: # doveadm mailbox list 'A*' A----->B # doveadm mailbox rename 'A----->B' AtoB # doveadm mailbox list 'A*' AtoB From tss at iki.fi Fri Nov 4 21:38:27 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:38:27 +0200 Subject: [Dovecot] Getting the SMTP envelope through IMAP? In-Reply-To: <4EA28848.6070303@msgid.danisch.de> References: <4EA19119.4020803@msgid.danisch.de> <20111021180241.GA6924@PC211.ikt.de> <4EA1EB82.10807@msgid.danisch.de> <20111022081510.GA26132@laptop-dg.leere.eu> <4EA28848.6070303@msgid.danisch.de> Message-ID: <1320435508.21919.147.camel@hurina> On Sat, 2011-10-22 at 11:09 +0200, Hadmut Danisch wrote: > And neither the Return-Path nor the Received-Lines reveal the precise > date of delivery as the From line contains. The delivery date in From_ line is available in IMAP INTERNALDATE. The address part isn't available without patching Dovecot (mail_get_special(mail, MAIL_FETCH_FROM_ENVELOPE, &value)) From tss at iki.fi Fri Nov 4 21:43:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:43:31 +0200 Subject: [Dovecot] Dovecot crashes totally In-Reply-To: <4EA317B5.3090209@uni-greifswald.de> References: <4EA317B5.3090209@uni-greifswald.de> Message-ID: <1320435812.21919.150.camel@hurina> On Sat, 2011-10-22 at 21:21 +0200, Gordon Grubert wrote: > Hello, > > our dovecot server crashes totally without any really useful > log messages. The error log can be found in the attachment. > The only way to get dovecot running again is a complete > system restart. How often does it break? If really a "complete system restart" is needed to fix it, it doesn't sound like a Dovecot problem. Check if it's enough to stop dovecot and then make sure there aren't any dovecot processes lying around afterwards. > Oct 11 09:55:23 mailserver2 dovecot: master: Error: service(imap): > Initial status notification not received in 30 seconds, killing the > process > Oct 11 09:56:23 mailserver2 dovecot: imap-login: Error: master(imap): > Auth request timed out (received 0/12 bytes) Kind of looks like auth process is hanging. You could see if stracing it shows anything useful. Also are any errors logged about LDAP? Is LDAP running on the same server? From tss at iki.fi Fri Nov 4 21:47:39 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:47:39 +0200 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <4EA2C213.7030204@mike2k.de> References: <4EA2C213.7030204@mike2k.de> Message-ID: <1320436059.21919.152.camel@hurina> On Sat, 2011-10-22 at 15:16 +0200, Michael Stilkerich wrote: > dovecot: imap(michael): Error: > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file > or directory This is some weird filesystem? fstat() isn't supposed to ever fail with ENOENT. > Another thing that irritates me is that dovecot seems to use the dotlock > locking method, although I explicitly set lock_method to > fcntl (also tried flock, same behavior). The lock_method setting only affects index files. There are several others that always use dotlocks, because they're rarely enough modified that the dotlock performance doesn't matter. That's also why I haven't bothered to set use_excl_lock for them. From tss at iki.fi Fri Nov 4 21:50:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:50:13 +0200 Subject: [Dovecot] iterate_query not working In-Reply-To: <4EA5BBC8.5090901@netexpert.pl> References: <4EA5BBC8.5090901@netexpert.pl> Message-ID: <1320436213.21919.153.camel@hurina> On Mon, 2011-10-24 at 21:26 +0200, Piotr NetExpert wrote: > Hi, > > I have got iterate_query defined in dovecot-sql.conf.ext. However when I > run a command like: > doveadm search -A BEFORE 10d > I receive an error message (Failed to iterate ...) > > There is an error message in a log file too: > Error: Trying to iterate users, but userdbs don't support it dovecot -n output? Maybe you're using only passdb sql, but userdb static? Maybe you have more than one userdb? From piotr-l at netexpert.pl Fri Nov 4 21:53:41 2011 From: piotr-l at netexpert.pl (Piotr NetExpert) Date: Fri, 04 Nov 2011 20:53:41 +0100 Subject: [Dovecot] iterate_query not working In-Reply-To: <1320436213.21919.153.camel@hurina> References: <4EA5BBC8.5090901@netexpert.pl> <1320436213.21919.153.camel@hurina> Message-ID: <4EB442C5.9000709@netexpert.pl> Thanks for a reply. In the meantime I have found the problem. Userdb was static as you say. I believe docs do not tell that iterate query is based on user query. -- Tre?? oryginalnej wiadomo?ci -- > On Mon, 2011-10-24 at 21:26 +0200, Piotr NetExpert wrote: >> Hi, >> >> I have got iterate_query defined in dovecot-sql.conf.ext. However when I >> run a command like: >> doveadm search -A BEFORE 10d >> I receive an error message (Failed to iterate ...) >> >> There is an error message in a log file too: >> Error: Trying to iterate users, but userdbs don't support it > > dovecot -n output? Maybe you're using only passdb sql, but userdb > static? Maybe you have more than one userdb? > > -- pozdrawiam Piotr Szafarczyk http://www.netexpert.pl From tss at iki.fi Fri Nov 4 21:55:00 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 21:55:00 +0200 Subject: [Dovecot] Expire plugin not executing for sieve plugin? In-Reply-To: <4EA5E5DF.1050508@rename-it.nl> References: <4EA03F1C.5010500@druber.com> <4EA5A5E1.3000202@druber.com> <4EA5E5DF.1050508@rename-it.nl> Message-ID: <1320436501.21919.155.camel@hurina> On Tue, 2011-10-25 at 00:25 +0200, Stephan Bosch wrote: > On 10/24/2011 7:52 PM, Dan Swartzendruber wrote: > > > > No ideas? I googled some more and eventually turned up this posting: > > > > http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-01/msg00716.html > > > > > > guy seems to be having the exact problem I am, unfortunately, no-one > > replied to his post either :( > > Hmm, wonder why no one answered that one. In theory, those plugins > should interact gracefully. Right. There shouldn't be anything special with expire plugin and Sieve. > Running dovecot 1.2.12 on ubuntu 10.10. This very likely works fine with v2.0 though, and I don't really have time to debug noncritical v1.x bugs anymore. From tss at iki.fi Fri Nov 4 22:01:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:01:23 +0200 Subject: [Dovecot] SSL renegotiation vulnerability In-Reply-To: References: <23D958A8-91DE-42EA-BDC5-B0E2050EEC9F@iki.fi> <90232072-D5CF-42E7-8B5A-3DFBF75C6B3E@iki.fi> <7CAD9FBB-73BF-4D68-939B-1055AFED4C79@iki.fi> <3E17898B-B716-413F-9582-619B472D7C2B@iki.fi> Message-ID: <1320436883.21919.158.camel@hurina> http://vincent.bernat.im/en/blog/2011-ssl-dos-mitigation.html -> "Things get worse" shows that it's easier to DoS the server with multiple connections than with renegotiations, so I don't know if there's much point in disabling renegotiations. Perhaps Dovecot could allow e.g. one renegotiation per minute, but is that really worth the trouble?.. Perhaps there even are some clients that do renegotiations and Dovecot would break them. From tss at iki.fi Fri Nov 4 22:02:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:02:46 +0200 Subject: [Dovecot] GSSAPI and deny=yes passdb In-Reply-To: <4E8ACB70.7060004@telros.ru> References: <4E8ACB70.7060004@telros.ru> Message-ID: <1320436966.21919.160.camel@hurina> On Tue, 2011-10-04 at 13:01 +0400, Sergey Urushkin wrote: > Hi. Is it possible to use GSSAPI authentication and deny passdb > together? Seems it doesn't work as I expect: GSSAPI doesn't check deny > passdb, so I'm not able to restrict access to GSSAPI-users. GSSAPI doesn't use any passdbs, so it doesn't work. It would be nice to change this some day (so e.g. proxying would also be possible with GSSAPI auth), but for now it's a rather low priority for me. From tss at iki.fi Fri Nov 4 22:04:54 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:04:54 +0200 Subject: [Dovecot] POP server migration and duplicate messages. In-Reply-To: References: Message-ID: <1320437094.21919.162.camel@hurina> On Thu, 2011-10-27 at 12:16 -0700, Ernie Dunbar wrote: > From a > random sampling, the UIDL of the messages checks out between both servers, Completely identical UIDL output with the same order of messages? If so, there's nothing you can do about it from Dovecot's side. > so I really don't understand what's going on here. If this is just a bug > in Evolution, then that means I can just go ahead with the full migration > and hope that people aren't going to be stuck with re-downloading their > entire mailbox which in some cases, means hundreds or thousands of > messages. Some clients redownload mails if you change the hostname. But I don't know why Evolution would redownload only some of the messages but not all. From tss at iki.fi Fri Nov 4 22:08:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:08:59 +0200 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> Message-ID: <1320437339.21919.165.camel@hurina> On Fri, 2011-10-28 at 00:55 +0200, Marco Carcano wrote: > I'm struggling against what seems a really serious trouble: sometimes > mail get lost > > I did all the possible checks, but they seems really lost - I know is > absurd, but it seems so > > it has been noticed by two users on my server: intially I tought that > was their mistake (mail deleted, mail into spam, ecc), but it wasn't Enable mail_log plugin to make sure of this. http://wiki2.dovecot.org/Plugins/MailLog > Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid= >: saved mail to INBOX If Dovecot logs this, then the message definitely was saved to INBOX. > maybe is there some kind of bug in the maildrop version I use? by the > way, I had this issue also before installing amavisd-new Maildrop? From tss at iki.fi Fri Nov 4 22:19:43 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:19:43 +0200 Subject: [Dovecot] doveadm 'proxy list' or 'who' not working ? In-Reply-To: <7A7D318C-0BA3-4FDB-AB7F-F36A540C8472@aquaray.com> References: <7A7D318C-0BA3-4FDB-AB7F-F36A540C8472@aquaray.com> Message-ID: <1320437984.21919.170.camel@hurina> On Fri, 2011-10-28 at 11:49 +0200, de Lafond Guillaume wrote: > Hi, > > I switched from dovecot 1.2.* to 2.0.* for a mail proxy. > I try to play with the new cool utility doveadm, but I don't manage to get results from it. > > # /usr/local/dovecot-2.0.15/bin/doveadm proxy list > username proto src ip dest ip port > # This should work. A quick test shows it working with me. Check if you have "ipc" process running? Check that you have /login/ipc-proxy fifo. Maybe try straceing the imap-login processes and see if any ipc connection arrives there when running the doveadm command. > As 'proxy list' does not return anything, I tried with 'who' > # /usr/local/dovecot-2.0.15/bin/doveadm who -a /var/run/dovecot-socket-auth/anvil > username # proto (pids) (ips) > # This isn't supposed to show anything. Although why do you give such a socket path parameter? From tss at iki.fi Fri Nov 4 22:22:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:22:31 +0200 Subject: [Dovecot] What owner/permissions for virtual homes? In-Reply-To: References: Message-ID: <1320438151.21919.172.camel@hurina> On Fri, 2011-10-28 at 16:04 -0400, sean darcy wrote: > On 10/28/2011 03:34 PM, sean darcy wrote: > > VirtualUsers.Home.txt: > > > > The directory layouts for home and mail directories could look like one of > > these (in the preferred order): > > > > 1. Mail directory under home, for example: home='/var/vmail/domain/user/' > > mail='/var/vmail/domain/user/mail/' > > > > Who should be the owner of these directories, with that permissions? Typically 0700 permissions, owned by the "mail user", whichever you've configured for that. http://wiki2.dovecot.org/UserIds > > Does it make a difference if it's /home/vmail rather than /var/vmail? Either one is fine. > it suggests the following configuration: > > mail_home = /var/vmail/%d/%n > mail_location = maildir:~/mail > > but where does mail_home appear? Also in 10-mail.conf? Doesn't matter. For example there. From tss at iki.fi Fri Nov 4 22:28:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:28:28 +0200 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111025110238.GB8900@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> Message-ID: <1320438508.21919.177.camel@hurina> On Tue, 2011-10-25 at 13:02 +0200, Edgar Fu? wrote: > We have two dovecot 1.2 instances sharing Maildirs on NFS. Indexes are local to the individual servers. > Occasionally (no idea how to trigger this), the Seen flag gets lost on some messages. I've verified that actually the ``S'' is missing from the filename. > I suspect something like server A caching the flags, server B setting Seen, and then server A flushing its cache for another change so overwriting what B changed. That shouldn't happen because of Dovecot's indexes. Dovecot doesn't force Maildir to match its indexes. Whenever doing flag changes, it stores them as "add \Seen flag to UID 1234". Then the 'S' flag is added to the current Maildir filename without losing any other changes. From ms at mike2k.de Fri Nov 4 22:29:16 2011 From: ms at mike2k.de (Michael Stilkerich) Date: Fri, 04 Nov 2011 21:29:16 +0100 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320436059.21919.152.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> Message-ID: <4EB44B1C.90505@mike2k.de> On 04.11.2011 20:47, Timo Sirainen wrote: >> dovecot: imap(michael): Error: >> > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file >> > or directory > This is some weird filesystem? fstat() isn't supposed to ever fail with > ENOENT. Nope, plain ext3. I have these issues since upgrading to Ubuntu 11.10, which was a dovecot upgrade from 1.2.15 to 2.0.13. Currently I use a self-built 2.0.15 dovecot. I'm still getting errors once in a while for the dovecot.index.log file: Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking failed for file /home/dovecot/isa/dovecot.index.log: No such file or directory Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with file /home/dovecot/isa/dovecot.index.log: No such file or directory These errors occur regardless of the lock_method used. And occasionally I also get errors like: Nov 3 19:03:34 keira dovecot: imap(mikey): Error: Corrupted transaction log file /home/dovecot/mikey/dovecot.index.log seq 15: offset points outside file (32708 + 8 > 32708) (sync_offset=32800) I believe the latter was recently reported by someone else on this list as well. The files are there, though I guess that dovecot recreates them when the errors occurs so I don't know what the situation was by the time the error occurred... -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4480 bytes Desc: S/MIME Cryptographic Signature URL: From tss at iki.fi Fri Nov 4 22:30:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:30:13 +0200 Subject: [Dovecot] post login script In-Reply-To: References: Message-ID: <1320438617.21919.179.camel@hurina> On Fri, 2011-10-28 at 12:49 +0200, Osvaldo Alvarez Pozo wrote: > Hello all, > > I am testing postlogin script with dovecot 2. > > I works but i would like that the imap and pop session get executed > even if the post-login script fail. How/why would the script fail? > #/usr/local/bin/postlogin.sh > echo "UPDATE mailbox SET last_login = now() WHERE username = '$USER'" > | mysql -upostlogin -pXXXXXXXX postfixadmin > exec "$@" Looks like it would always exec $@ even if mysql update fails. From tss at iki.fi Fri Nov 4 22:33:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:33:09 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> Message-ID: <1320438789.21919.181.camel@hurina> On Tue, 2011-11-01 at 09:38 +0100, Lukas Wei? wrote: > >> and sql storage is difficult > to backup. Ive done that, by setting up dovecot with maildir format and > copy mails from old dbmail to new dovecot imap server with my mail > client (thunderbird). > > Thunderbird probably changes the INTERNALDATE of > the messages. Copy the mails some other way, like offlineimap or maybe > some other IMAP client. > > What is INTERNALDATE? The modification > timestamp is correct ("ls -l"), and the "Date: ... " line, too. Maildir file's mtime is the INTERNALDATE, yes. Maybe the the problem isn't then any timestamps, but that the messages were copied in reverse order (newest message first), so their IMAP UIDs are reversed. From tss at iki.fi Fri Nov 4 22:38:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:38:08 +0200 Subject: [Dovecot] LDAP w/SASL "Active Directory" authentication failing. In-Reply-To: References: Message-ID: <1320439089.21919.184.camel@hurina> On Tue, 2011-11-01 at 09:55 -0600, David Varela wrote: > I am running a Dovecot server (version 1.2.17) on FreeBSD 8.2, using > LDAP to authenticate Active Directory users. I can successfully bind and > authenticate using PLAIN and LDAP without SASL, but obviously passwords for > the bind user and the user being authenticated are being passed in plain > text. I've attempted to configure my server to us SASL however when I > attempt to authenticate a user I see authentication failures. I reviewed > the security log on my domain controller and see that the bind user is > binding properly, so the issue appears to be orginating from the user > authentication, however I cannot determine what the issue is. Here is all > the information regarding my configuration, along with the logs from the > server: SASL binding currently works only for the initial "ldap admin user" authentication. It doesn't work for individual user authentication requests (auth_bind=yes). > #auth_bind = yes Here you're not even attempting to use auth binds. > pass_attrs = mail=user And you're also not returning a password for user. > Nov 01 09:13:26 auth(default): Info: ldap(davidv at smallmountain.net,127.0.0.1): > No password returned (and no nopassword) So Dovecot has no way of authenticating user. I'd suggest forgetting about SASL and enabling TLS instead. From tss at iki.fi Fri Nov 4 22:41:26 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:41:26 +0200 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <20111102085715.GA4089@dibs.tanso.net> References: <20111102085715.GA4089@dibs.tanso.net> Message-ID: <1320439286.21919.187.camel@hurina> On Wed, 2011-11-02 at 09:57 +0100, Jan-Frode Myklebust wrote: > IMHO it would be great if we could both provision and un-provision users > without having the provisioning system mess with files in the > filesystem. Provisioning new users works fine since dovecot will create > all files/directories on first login, but I haven?t found the tools to > un-provison them. I see "doveadm mailbox delete" can delete individual > mailboxes, but what about indexes, sieve-scripts and home-directories? > Are there any tools to clean out these ? Do you have all of those files in separate directories? Normally I'd think everything is under home dir, so you can just: rm -rf `doveadm user -f home $username` For deleting indexes and other dirs that are not under home.. well.. I guess you'll need some more scripting. Anyway, I don't think there's any good and safe way to just go delete user's directories. Especially home dir. From tss at iki.fi Fri Nov 4 22:44:30 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:44:30 +0200 Subject: [Dovecot] Different default realm per listener In-Reply-To: <4EB15375.30106@team.de.inter.net> References: <4EB15375.30106@team.de.inter.net> Message-ID: <1320439470.21919.190.camel@hurina> On Wed, 2011-11-02 at 15:28 +0100, Kai Arif wrote: > Hi all! > > I want create a large IMAP server setup where dovecot is listening on multiple interfaces and needs > a different default realm on each interface. I'm using dovecot 2.0.5 on FreeBSD9 and authenticate > against a LDAP database. > I found an old mail thread from 2007 that said I'm basically screwed. Has that change since than? Is > there any way to use different default realms per listener? v2.0 can almost do it: local 1.2.3.4 { auth_default_realm = foo.org } local 1.2.3.5 { auth_default_realm = bar.org } Unfortunately that will currently fail, because per-IP auth settings don't work.. It will hopefully be fixed some day. From tss at iki.fi Fri Nov 4 22:47:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:47:57 +0200 Subject: [Dovecot] IMP, Dovecot and multiple namespaces In-Reply-To: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> References: <20111102093747.Horde.zz0wYaPBW7JOsVW77jPxFuo@beta.vfemail.net> Message-ID: <1320439677.21919.192.camel@hurina> On Wed, 2011-11-02 at 09:37 -0500, Rick Romero wrote: > Hey guys, > > I'm wondering what the best way is to be backwards compatible with > Courier mailbox formats and not duplicate mailbox trees with Dovecot. > Is anyone doing this right now? > > My Dovecot 2.0.13 is configured as follows for Namespaces: > > namespace { > inbox = yes > location = > prefix = > separator = . Perhaps use '/' separator. > } > namespace { > hidden = yes > inbox = no > list = no > location = > prefix = INBOX. > separator = . Add "alias_for = " here. I also added it to wiki2. > } > > So I should have an INBOX, plus additional root folders - rather than > the Courier default of all folders being subfolders of the INBOX. > > So when I run LIST in IMAP I get - > 08 LIST "" % > * LIST (\HasChildren) "." "INBOX" INBOX.% now matches both INBOX. namespace's mailboxes, and also the INBOX mailbox's children. If you change the separator to '/', there's a difference between INBOX.% (compatibility namespace) and INBOX/% (INBOX's children). From tss at iki.fi Fri Nov 4 22:59:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 22:59:04 +0200 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111103095424.67160@gmx.com> References: <20111103095424.67160@gmx.com> Message-ID: <1320440344.21919.196.camel@hurina> On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the filesystem operations take more than a second. From tss at iki.fi Fri Nov 4 23:03:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:03:25 +0200 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: References: Message-ID: <1320440605.21919.198.camel@hurina> On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes wrote: > I'm having a problem with dovecot ldap connection when ldap server is in > another firewall zone, firewall kills the ldap connection after a > determined period of inactivity, this is good from the firewall point of > view but is bad for dovecot because it never knows the connections has been > dropped, this creates longs timeouts in dovecot and finally it reconnects, > meanwhile many users fails to authenticate, I have seen this kind of post > in the list for a while but can't find a solution for it, so my question is > how to define a idle ldap time in dovecot so it can reconnect before the > firewall has dropped the connection or just close the connection under > inactivity so when a user authenticate doesn't fails for a while until > dovecot detects that the connection has hanged. Is this a feature request > or there is already a configuration for this??? Can't the firewall be changed to reject the LDAP packets instead of dropping them? Then Dovecot would immediately notice that the connection has died, and with a recent enough version it wouldn't even log an error about it. I guess some kind of an "ldap_idle_disconnect = 30s" setting could be added, but it's not a very high priority for me. From tss at iki.fi Fri Nov 4 23:09:49 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:09:49 +0200 Subject: [Dovecot] Restricting IMAP access In-Reply-To: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> References: <469C3D1B-525E-4660-B1B9-2ED6AEA4CCF3@mailhub.co.za> Message-ID: <1320440989.21919.202.camel@hurina> On Thu, 2011-11-03 at 19:13 +0100, Thierry de Montaudry wrote: > Hi list, > > I have a setup with postfix+dovecot+mysql unser CentOS 5, running 50 odd domains with virtual users. Access is allowed for public POP3, and a webmail on apache+PHP solution through local IMAP. > I'm not gonna give you the long story about the why, but I'm looking for a way to give public IMAP access only to one domain, knowing that users log in with full email (user at domain.tld). > Anybody has a trick for that? Running dovecot 2.0.13. > > I know there should be a way to do it through the database, but quite heavy change on our side for a million odd users. So assuming all users can log in from 127.0.0.1 and domain.com only elsewhere, you can do stuff like: a) Change password_query so that it succeeds only if '%l' = '127.0.0.1' or '%d' = 'domain.com' b) Add a deny=yes passdb with e.g. sqlite backend that basically does the same as a) but reversed. http://wiki2.dovecot.org/Authentication/RestrictAccess has passwd-file example, example config files should have also something. c) Add a new Dovecot proxy facing the public internet where it lets in only those users with '%d'='domain.com' (again maybe sqlite) Possibly other ways too.. From janfrode at tanso.net Fri Nov 4 23:19:31 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 4 Nov 2011 22:19:31 +0100 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <1320439286.21919.187.camel@hurina> References: <20111102085715.GA4089@dibs.tanso.net> <1320439286.21919.187.camel@hurina> Message-ID: <20111104211931.GA5430@dibs.tanso.net> On Fri, Nov 04, 2011 at 10:41:26PM +0200, Timo Sirainen wrote: > On Wed, 2011-11-02 at 09:57 +0100, Jan-Frode Myklebust wrote: > > IMHO it would be great if we could both provision and un-provision users > > without having the provisioning system mess with files in the > > filesystem. Provisioning new users works fine since dovecot will create > > all files/directories on first login, but I haven?t found the tools to > > un-provison them. I see "doveadm mailbox delete" can delete individual > > mailboxes, but what about indexes, sieve-scripts and home-directories? > > Are there any tools to clean out these ? > > Do you have all of those files in separate directories? Normally I'd > think everything is under home dir, so you can just: Unfortunately, for historical reasons, yes. The users doesn't have a home-directory, or the inbox is directly in their home.. so I don't think sieve fits there, and indexes are kept elsewhere for performance reasons. > > rm -rf `doveadm user -f home $username` Ah, great. Looking up the home dir via doveadm helps. > For deleting indexes and other dirs that are not under home.. well.. I > guess you'll need some more scripting. What happens to the indexes when we delete the mailboxes via "doveadm mailbox delete" ? Will that clean up the indexes ? -jf From tss at iki.fi Fri Nov 4 23:18:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:18:56 +0200 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <4EAED444.70608@strike.wu.ac.at> References: <4EAED444.70608@strike.wu.ac.at> Message-ID: <1320441536.21919.206.camel@hurina> On Mon, 2011-10-31 at 18:00 +0100, Alexander 'Leo' Bergolth wrote: > NAMESPACE_2='maildir:/var/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep' > NAMESPACE_2_PREFIX=Shared/spamrep/ > NAMESPACE_2_SEP=/ > NAMESPACE_2_TYPE=public > NAMESPACE_2_LIST=yes The settings are now named. http://wiki2.dovecot.org/MailLocation#Custom_namespace_location shows the basics. So for example if you have: namespace foo { .. } You can change it via: NAMESPACE/FOO/PREFIX=Shared/spamrep/ NAMESPACE/FOO/SEP=/ ..etc.. The '/' character is a bit problematic with shells scripts though, see the wiki link how to do it via "env" at least. Another problem is if you want to add more namespaces than you have configured. IIRC it's possible to simply add: NAMESPACE=foo And more than one is probably either "foo bar" or "foo,bar". If you get something to work, please add example to http://wiki2.dovecot.org/PostLoginScripting :) From tss at iki.fi Fri Nov 4 23:26:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:26:29 +0200 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <4EB43BB8.6090803@one.com> References: <4EB43BB8.6090803@one.com> Message-ID: <1320441989.21919.212.camel@hurina> On Fri, 2011-11-04 at 20:23 +0100, Peter Mogensen wrote: > I'm running Dovecot 2.0.15 and have a client which keeps causing > log-entries like: > > > Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted > transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: > indexid changed 1320419300 -> 1320419441 (sync_offset=0) Session A had TestMails open and created with index file whose ID was 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 17:08:20 EET 2011). Session B came and recreated the index files 141 seconds later with ID 1320419441. Either it didn't see A's original index files for some reason or it simply decided to recreate them for some reason. Either way this shouldn't have happened. Session A then notices that the indexes were recreated, and logs an error. > But it only seems to be reproducible, when there's another client > involved. Starting up Thunderbird with a login to the account seems to > be enough to cause this. (To my knowledge this causes a series of STATUS > command and then IDLE). > > The installation is using Maildir and lock_method = fcntl on local disks. What filesystem? Have changed any service imap {} settings? Can you reproduce this by running stress test with imaptest? http://imapwiki.org/ImapTest For example: imaptest user=testuser pass=testpass host=localhost Or if it's really related to creating the initial files: while imaptest user=testuser pass=testpass host=localhost secs=1; do rm -rf /home/testuser/Maildir done From tss at iki.fi Fri Nov 4 23:27:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:27:46 +0200 Subject: [Dovecot] doveadm delete-user ? In-Reply-To: <20111104211931.GA5430@dibs.tanso.net> References: <20111102085715.GA4089@dibs.tanso.net> <1320439286.21919.187.camel@hurina> <20111104211931.GA5430@dibs.tanso.net> Message-ID: <1320442066.21919.213.camel@hurina> On Fri, 2011-11-04 at 22:19 +0100, Jan-Frode Myklebust wrote: > > For deleting indexes and other dirs that are not under home.. well.. I > > guess you'll need some more scripting. > > What happens to the indexes when we delete the mailboxes via "doveadm > mailbox delete" ? Will that clean up the indexes ? Yes. So you could create a script that lists all mailboxes, deletes them and then rmdirs the index root dir. From tss at iki.fi Fri Nov 4 23:39:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:39:18 +0200 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <4EB44B1C.90505@mike2k.de> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> Message-ID: <1320442758.21919.222.camel@hurina> On Fri, 2011-11-04 at 21:29 +0100, Michael Stilkerich wrote: > On 04.11.2011 20:47, Timo Sirainen wrote: > >> dovecot: imap(michael): Error: > >> > fstat(/home/dovecot/michael/.test/dovecot-acl.lock) failed: No such file > >> > or directory > > This is some weird filesystem? fstat() isn't supposed to ever fail with > > ENOENT. > > Nope, plain ext3. I have these issues since upgrading to Ubuntu 11.10, > which was a dovecot upgrade from 1.2.15 to 2.0.13. Currently I use a > self-built 2.0.15 dovecot. I'm still getting errors once in a while for > the dovecot.index.log file: > > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking > failed for file /home/dovecot/isa/dovecot.index.log: No such file or > directory > Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with > file /home/dovecot/isa/dovecot.index.log: No such file or directory These simply shouldn't happen. I'd say it's a kernel bug. You're running a default Ubuntu kernel? I wonder if other Ubuntu users have this problem. > And occasionally I also get errors like: > > Nov 3 19:03:34 keira dovecot: imap(mikey): Error: Corrupted transaction > log file /home/dovecot/mikey/dovecot.index.log seq 15: offset points > outside file (32708 + 8 > 32708) (sync_offset=32800) > > I believe the latter was recently reported by someone else on this list > as well. The files are there, though I guess that dovecot recreates them > when the errors occurs so I don't know what the situation was by the > time the error occurred... Try if you can reproduce these more easily with imaptest? http://imapwiki.org/ImapTest Be sure to run it against a test account since it trashes the whole mailbox. From tss at iki.fi Fri Nov 4 23:47:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:47:10 +0200 Subject: [Dovecot] Rewrite the ostream output method, to save messages in another directory. In-Reply-To: References: Message-ID: <1320443230.21919.227.camel@hurina> On Mon, 2011-10-31 at 13:25 -0200, Alex Baule wrote: > struct ostream * > o_stream_create_X_list(struct ostream *output, struct istream *input, const > char *type) > { > struct emexis_ostream *X_stream; > struct ostream *new_output; > int fd; > > fd = open("/tmp/email",O_WRONLY); > > new_output = o_stream_create_fd_file(fd, 0, TRUE); > o_stream_cork(new_output); > > emexis_stream = i_new(struct emexis_ostream, 1); > emexis_stream->ostream.sendv = o_stream_X_sendv; > emexis_stream->ostream.flush = o_stream_X_flush; > emexis_stream->ostream.iostream.close = o_stream_X_close; > emexis_stream->output = new_output; > emexis_stream->input = input; > > o_stream_ref(new_output); > > return o_stream_create(&X_stream->ostream); > } This part of the code looks ok. > With this new o_stream, i got this error. > > Oct 27 15:14:24 brc dovecot: imap(alex at exemplo.com.br): Error: > o_stream_send_istream(/storage/gss/emexis/messages/ > exemplo.com.br/alex/Maildir/.Emexis.Ext-Blacklist/tmp/1319735664.M568409P30815.brc) > failed: Bad file descriptor I guess this should call your o_stream_X_sendv(). What do you do in that? I guess you're calling o_stream_sendv() for the new_output stream? Check if it returns -1, if it does then you're somehow breaking the new_output. Maybe you're closing it too early? Try what happens if you create it with: new_output = o_stream_create_fd_file(fd, 0, FALSE); Then it doesn't automatically close the fd, and it's at least clearer if the problem is because the stream is closed too early or if some completely wrong fd is attempted to be used. > There is something wrong Timo ? Like missing something or i need to rewrite > another funcion ? Nope. I think this is completely internal to your code. From apm at one.com Fri Nov 4 23:49:22 2011 From: apm at one.com (Peter Mogensen) Date: Fri, 04 Nov 2011 22:49:22 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EB45DE2.3030909@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: > What filesystem? XFS > Have changed any service imap {} settings? nope. But I have a special auth plugin. I doubt that's the reason though, since it's an entirely different process. Thanx, I'll try gather more info. /Peter From tss at iki.fi Fri Nov 4 23:49:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:49:28 +0200 Subject: [Dovecot] doveadm segfaults on TCP connect - version 2.0.15 In-Reply-To: <3316c960405142bf1d81f1db2b350739.squirrel@webmail.islandemail.com> References: <3316c960405142bf1d81f1db2b350739.squirrel@webmail.islandemail.com> Message-ID: <1320443369.21919.229.camel@hurina> On Wed, 2011-10-19 at 12:40 -0700, ghandidrivesahumvee at rocketfish.com wrote: > I just recently installed Dovecot 2.0.15. Unfortunately, doveadm segfaults > when I attempt to connect to the local dovecot instance. I fixed a lot of bugs related to this for v2.1. I'm not sure if I'll bother backporting them, since it can't work perfectly anyway without breaking other stuff.. Hopefully v2.1.0 won't be too far away now :) From tss at iki.fi Fri Nov 4 23:50:51 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:50:51 +0200 Subject: [Dovecot] Proxy and Virtual IMAP Folders In-Reply-To: References: Message-ID: <1320443452.21919.230.camel@hurina> On Wed, 2011-10-19 at 21:17 -0400, Pedro Paixao wrote: > Is is posible to use the virtual plugin, in a IMAP proxy configuration, in a > way that the virtual plugin searches messages on the real server? > > Here's my scenario: > > [Client]----[Dovecot Proxy]-----[Real Imap Server] > > Client connects to Dovecot Proxy and gets all the folders that are on the > Real Imap Server, plus some virtual ones (generated by Dovecot Proxy) that > search for messages on the real server. In my config the Dovecot Proxy has > no email messages, real folders, or local delivery. Yes, you should be able to do this with v2.1's imapc backend. From tss at iki.fi Fri Nov 4 23:52:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:52:13 +0200 Subject: [Dovecot] imap proxy for Gmail In-Reply-To: References: Message-ID: <1320443533.21919.231.camel@hurina> On Wed, 2011-10-19 at 23:31 -0400, Jumping Mouse wrote: > Hello, b > > I have a situation were I want to be able to give teachers access to their Google apps for education email through Squirrelmail. The network that they are on blocks gmail. I am able to get Squirrelmail to communicate with Gmail via imap. I am looking at getting persistant imap connections to improve the user experience. I have looked at IMAPPROXY but this application is not compatible with the gmail imaps (ssl-encrypted imap over port 993). I recall reading somewhere that Dovecot has a built in imapproxy that could be used for this. Dovecot doesn't provide persistent IMAP proxying. You should be able to combine imapproxy with stunnel to do this. From tss at iki.fi Fri Nov 4 23:56:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:56:04 +0200 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <4E9FEC22.8060105@webfusion.com> References: <4E9FEC22.8060105@webfusion.com> Message-ID: <1320443764.21919.233.camel@hurina> On Thu, 2011-10-20 at 12:38 +0300, Mark Zealey wrote: > I'm currently changing some systems to use the redirector service, which > means that to get local deliveries going I need to get lmtp set up so it > can be redirected as well. This is working fine, however we have a > number of different ports running on our servers depending on which > brand a customer is using. When the time comes to do the user look up, > we use (port, user, domain) to do a unique database lookup which works > fine for everything (proxied pop, imap, lmtp & straight pop, imap) but > not straight lmtp - perhaps because it doesn't have a 2-stage login > process unlike the pop/imap protocols. Looking at the SQL query we are > issuing, the %a port variable is set to 0 even though the connection is > coming in from TCP. Is there a way to change this? We are running > dovecot 2.0.12 however looking through the changelogs I can't see this > would be fixed in the newest version. A quick check in code shows that this is already supposed to work. Set auth_debug=yes and see if LMTP's auth lookup sends lip, rip, lport and rport fields to auth process? From tss at iki.fi Fri Nov 4 23:59:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 04 Nov 2011 23:59:03 +0200 Subject: [Dovecot] lmtp performance In-Reply-To: <201110171642.45954.joseba.torre@ehu.es> References: <4E9BE1FB.6040902@upm.es> <4E9C3C25.7070802@upm.es> <201110171642.45954.joseba.torre@ehu.es> Message-ID: <1320443944.21919.235.camel@hurina> On Mon, 2011-10-17 at 16:42 +0200, Joseba Torre wrote: > > Add to service lmtp: > > process_min_avail = 20 > I had a similar problem, and found the same solution. In my setup, only 1 lmtp > process was available and a 2nd one never was created, so the process was > really busy. With this parameter (I setup it to 10) I have diferent process > and all of them are used. Hmm. I wonder if perhaps it would be better to set: service lmtp { client_limit = 1 } I think I'll make that the default.. From tss at iki.fi Sat Nov 5 00:07:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 00:07:59 +0200 Subject: [Dovecot] [PATCH] Use SSL_MODE_RELEASE_BUFFERS if available to keep memory usage low In-Reply-To: References: Message-ID: <1320444480.21919.236.camel@hurina> On Thu, 2011-10-13 at 16:23 -0300, Cristian Rodr?guez wrote: > Use SSL_MODE_RELEASE_BUFFERS if available to keep memory usage low. Hmm. Committed. Although I wonder how big of a difference it makes. And if it increases CPU usage much? From maria_arrea at gmx.com Sat Nov 5 01:57:11 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Sat, 05 Nov 2011 00:57:11 +0100 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube Message-ID: <20111104235711.67190@gmx.com> Timo, we are not using NFS, we use remote iSCSI volumes with ext4. Regards Maria ----- Original Message ----- From: Timo Sirainen Sent: 11/04/11 09:59 PM To: Maria Arrea Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the filesystem operations take more than a second. From noah-list at enabled.com Sat Nov 5 02:32:09 2011 From: noah-list at enabled.com (Noah) Date: Fri, 04 Nov 2011 17:32:09 -0700 Subject: [Dovecot] dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 Message-ID: <4EB48409.90803@enabled.com> Hi there I keep getting the following errors scrolling in my dovecot logs. I am wondering how I can nail down what is creating that problem. dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 also I have a trash folder with over 100k messages in mbox format. I am wondering if migrating to Maildir format would make it easier and quicker for dovecot to index my mail. I am getting these types of errors from time to time. dovecot: 2011-10-30 20:32:58 Error: IMAP(): Timeout while waiting for lock for transaction log file /home//mail/.imap/INBOX/dovecot.index.log Cheers, Noah From stan at hardwarefreak.com Sat Nov 5 10:26:45 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 05 Nov 2011 03:26:45 -0500 Subject: [Dovecot] dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash UID 42022 got too little data: 90900 vs 1529791 In-Reply-To: <4EB48409.90803@enabled.com> References: <4EB48409.90803@enabled.com> Message-ID: <4EB4F345.80000@hardwarefreak.com> On 11/4/2011 7:32 PM, Noah wrote: > Hi there > > I keep getting the following errors scrolling in my dovecot logs. I am > wondering how I can nail down what is creating that problem. > > dovecot: 2011-11-04 16:58:01 Error: IMAP(): FETCH for mailbox Trash > UID 42022 got too little data: 90900 vs 1529791 What version of Dovecot are you using? > also I have a trash folder with over 100k messages in mbox format. I am > wondering if migrating to Maildir format would make it easier and > quicker for dovecot to index my mail. As I answered on the Postfix list, the solution to this is ditching procmail for dovecot-lda so indexing of new messages is incremental upon delivery, instead of megalithic when your MUA connects to the mailbox. Switching to maildir in this case will simply trade on problem for another. Your memory consumption may go down, but your disk access will skyrocket. > I am getting these types of errors from time to time. > > dovecot: 2011-10-30 20:32:58 Error: IMAP(): Timeout while waiting > for lock for transaction log file > /home//mail/.imap/INBOX/dovecot.index.log It would be really helpful if you would simply post a big dump of your machine specs, software configuration, user base, mailbox sizes, total mail storage space, concurrent user load, etc. You've now posted a total of 3 seemingly different problems on two mailing lists, related to your Dovecot problems. I get the feeling that there's a much larger overall problem here, say something like hardware totally under powered for the workload you're throwing at it, or something to that affect. Or maybe just a really bad system setup. The more information you can provide, the more quickly and thoroughly we can provide a solution or salient advice. -- Stan From laxlaw at gmx.de Sat Nov 5 11:57:19 2011 From: laxlaw at gmx.de (=?UTF-8?Q?Lukas_Wei=C3=9F?=) Date: Sat, 05 Nov 2011 10:57:19 +0100 Subject: [Dovecot] =?utf-8?q?Bad_creation_timestamp_of_migrated_mails_=28m?= =?utf-8?q?aildir_format=29=3F?= In-Reply-To: <1320438789.21919.181.camel@hurina> References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> <1320438789.21919.181.camel@hurina> Message-ID: On Fri, 04 Nov 2011 22:33:09 +0200, Timo Sirainen wrote: > On Tue, 2011-11-01 at 09:38 +0100, Lukas Wei? wrote: >>>> and sql storage is difficult to backup. Ive done that, by setting up dovecot with maildir format and copy mails from old dbmail to new dovecot imap server with my mail client (thunderbird). >> >>> Thunderbird probably changes the INTERNALDATE of >> the messages. Copy the mails some other way, like offlineimap or maybe some other IMAP client. What is INTERNALDATE? The modification timestamp is correct ("ls -l"), and the "Date: ... " line, too. > Maildir file's mtime is the INTERNALDATE, yes. Maybe the the problem isn't then any timestamps, but that the messages were copied in reverse order (newest message first), so their IMAP UIDs are reversed. Possible. How do i avoid this? What is the preffered way to migrate mails from one to another imap server? Offlineimap? Can i do a save sync with that, even if my dovecot has newer mails now? From lists at kokelnet.de Sat Nov 5 16:13:30 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 15:13:30 +0100 Subject: [Dovecot] Dovecot SASL on another machine Message-ID: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> Hello list, is there a way to use Dovecot SASL Authentication for submission on a different machine dovecot runs? e.g.: Dovecot machine <-> postfix machine(for submission) How can I authenticate my senders on a different physical machine against Dovecot SASL? Thanks, Tobias From tss at iki.fi Sat Nov 5 17:43:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 17:43:10 +0200 Subject: [Dovecot] fts-lucene vs SEARCH header Re: 2.1: problems compiling fts-lucene - CLucene git version necessary? In-Reply-To: References: <7C9B9C67-A1FD-472B-8514-BC54F1B01D00@iki.fi> <1316446540.12936.114.camel@hurina> <20110920071245.GA3617217@gabi.sernet.de> Message-ID: <1320507790.21919.237.camel@hurina> On Thu, 2011-09-22 at 13:24 +0200, Lutz Pre?ler wrote: > Next problem :-) side effect: my test setup (some commits ago) with fts = lucene > in plugin section (ftps_lucene plugin without stemmer support): > SEARCH header is not working (empty result sets, no logging). Fixed finally. From tss at iki.fi Sat Nov 5 18:01:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:01:47 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> Message-ID: <1320508907.21919.239.camel@hurina> On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > is there a way to use Dovecot SASL Authentication for submission on a > different machine dovecot runs? With v2.0 you should be able to add inet_listener for auth service, which acts as an auth-client socket. > Dovecot machine <-> postfix machine(for submission) Looking at Postfix v2.8.0 code, looks like Postfix also supports it. From tss at iki.fi Sat Nov 5 18:06:11 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:06:11 +0200 Subject: [Dovecot] Bad creation timestamp of migrated mails (maildir format)? In-Reply-To: References: <995cb6b0e895c72fa16fd7e0f0a4d8e0@weiss.homelinux.com> <1320438789.21919.181.camel@hurina> Message-ID: <1320509172.21919.241.camel@hurina> On Sat, 2011-11-05 at 10:57 +0100, Lukas Wei? wrote: > > Maildir file's mtime is the INTERNALDATE, yes. Maybe the the > problem isn't then any timestamps, but that the messages were copied in > reverse order (newest message first), so their IMAP UIDs are > reversed. > > Possible. How do i avoid this? What is the preffered way to > migrate mails from one to another imap server? Offlineimap? Can i do a > save sync with that, even if my dovecot has newer mails now? If you've already moved the mails, there's not much you can do.. Or if you don't care about losing IMAP UIDs (= clients redownloading mails), you could simply delete dovecot-uidlist and dovecot.index* files and let Dovecot reassign UIDs in the "correct" order. From tss at iki.fi Sat Nov 5 18:15:05 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:15:05 +0200 Subject: [Dovecot] Dot Lock timestmap, users disconnections from roundcube In-Reply-To: <20111104235711.67190@gmx.com> References: <20111104235711.67190@gmx.com> Message-ID: <1320509705.21919.248.camel@hurina> Well, doesn't matter if it's NFS or not. It still looks as if Dovecot process was stuck for 45 seconds, most likely waiting for disk I/O to finish.. What happens is something like: 1. Get the current time ("now") 2. See if lock file exists 3. Create lock file 4. fstat() the created lock file 5. Log a warning if fstat's ctime differs from "now" more than 30 seconds. (Actually I think the 30 seconds threshold is way too generous, it should be less than 1 second usually.) So steps 2 and 3 took 45 seconds to finish. Basically I guess the disk I/O load was very high at that time, or alternatively there was some unintentional delay caused by iSCSI (kernel/network bug/problem). On Sat, 2011-11-05 at 00:57 +0100, Maria Arrea wrote: > Timo, we are not using NFS, we use remote iSCSI volumes with ext4. > > Regards > > Maria > > ----- Original Message ----- > From: Timo Sirainen > Sent: 11/04/11 09:59 PM > To: Maria Arrea > Subject: Re: [Dovecot] Dot Lock timestmap, users disconnections from roundcube > > On Thu, 2011-11-03 at 10:54 +0100, Maria Arrea wrote: > Hello. > > We are running dovecot 2.0.13 with mdbox+zlib on RHEL 5.7 x64, ext4. We use NTP. Indexes are in a iSCSI raid 10, mailboxes in raid5. No NFS. We have detected that sometimes all users get disconnected from roundcube at the same time. In dovecot logs we hundreds of lines like this: > > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Warning: Created dotlock file's timestamp is different than current time (1320308587 vs 1320308542): /buzones/mydomain/03/67/mcrivero/subscriptions I did several fixes related to this, but they were already in v2.0.10. Note the time difference of 45 seconds. > Nov 3 09:23:07 buzon dovecot: imap(mcrivero at mydomain): Connection closed bytes=0/295 The dotlock warning isn't related to this. My guess: NFS was being extremely slow here, some operation took 45 seconds and Roundcube decided to abort before that. The "timestamp is different" check doesn't work 100% correctly if the fil > esystem operations take more than a second. From tss at iki.fi Sat Nov 5 18:23:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:23:14 +0200 Subject: [Dovecot] Attachements with SIS : shauld all the same files share the same inode ? In-Reply-To: <4E9BD704.9080801@univ-nantes.fr> References: <4E9BD704.9080801@univ-nantes.fr> Message-ID: <1320510194.21919.251.camel@hurina> On Mon, 2011-10-17 at 09:19 +0200, Yann Dupont wrote: > 104 instance of -rw------- 104 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-ffd0d1xxxx > > The files are hardlinked, so it seems ok, but I also see 9 others files > in the same directory : > > > -rw------- 1 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-0519df2a94fc9a4e1f4100008a6b99b6 > -rw------- 1 vmail vmail 346584 16 oct. 17:47 > 6e2df299fa96daec9b4735d07c494046429be4d6-1790b92994fc9a4e1a4100008a6b99b6 > > as you can see, same prefix, If I compare them with diff, they are all > the same files. I guess you have "mail_attachment_fs = sis posix" (default)? Sounds like there could be a bug where SIS does comparison of files but for some reason thinks they are different. You could try if you can merge them with: mkdir /tmp/queue touch /tmp/queue/6e2df299fa96daec9b4735d07c494046429be4d6-foo doveadm sis deduplicate /var/attachments /tmp/queue From lists at kokelnet.de Sat Nov 5 18:31:56 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 17:31:56 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320508907.21919.239.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> Message-ID: <91de83aeeb686e189fb47e454ce86a06@hachmer.de> Am 05.11.2011 17:01, schrieb Timo Sirainen: > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: >> is there a way to use Dovecot SASL Authentication for submission on >> a >> different machine dovecot runs? > > With v2.0 you should be able to add inet_listener for auth service, > which acts as an auth-client socket. Thanks for your reply. I think I didn't couch correct. I want a postfix machine to authenticate against Dovecot SASL for submission service which hasn't installed dovecot locally. The postfix machine has to ask my central dovecot which is at the moment dovecot v1.2.15 but will be soon v2. Is this possible, if yes, how? Greetz, Tobias From tss at iki.fi Sat Nov 5 18:35:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:35:01 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <91de83aeeb686e189fb47e454ce86a06@hachmer.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <91de83aeeb686e189fb47e454ce86a06@hachmer.de> Message-ID: <1320510902.21919.253.camel@hurina> On Sat, 2011-11-05 at 17:31 +0100, Tobias Hachmer wrote: > Am 05.11.2011 17:01, schrieb Timo Sirainen: > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > >> is there a way to use Dovecot SASL Authentication for submission on > >> a > >> different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > Thanks for your reply. I think I didn't couch correct. > I want a postfix machine to authenticate against Dovecot SASL for > submission service which hasn't installed dovecot locally. > The postfix machine has to ask my central dovecot which is at the > moment dovecot v1.2.15 but will be soon v2. Yes, this is how I understood it. > Is this possible, if yes, how? With v2.0: service auth { inet_listener { port = 12345 } } Postfix: smtpd_sasl_path = inet:dovecot.foobar.com:12345 From lists at kokelnet.de Sat Nov 5 18:39:50 2011 From: lists at kokelnet.de (Tobias Hachmer) Date: Sat, 05 Nov 2011 17:39:50 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320510902.21919.253.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <91de83aeeb686e189fb47e454ce86a06@hachmer.de> <1320510902.21919.253.camel@hurina> Message-ID: <80202ce555c1e3ca63e1c1ee3e8cd870@hachmer.de> Am 05.11.2011 17:35, schrieb Timo Sirainen: > On Sat, 2011-11-05 at 17:31 +0100, Tobias Hachmer wrote: >> Am 05.11.2011 17:01, schrieb Timo Sirainen: >> > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: >> >> is there a way to use Dovecot SASL Authentication for submission >> on >> >> a >> >> different machine dovecot runs? >> > >> > With v2.0 you should be able to add inet_listener for auth >> service, >> > which acts as an auth-client socket. >> >> Thanks for your reply. I think I didn't couch correct. >> I want a postfix machine to authenticate against Dovecot SASL for >> submission service which hasn't installed dovecot locally. >> The postfix machine has to ask my central dovecot which is at the >> moment dovecot v1.2.15 but will be soon v2. > > Yes, this is how I understood it. OK, sorry. >> Is this possible, if yes, how? > > With v2.0: > > service auth { > inet_listener { > port = 12345 > } > } > > Postfix: > > smtpd_sasl_path = inet:dovecot.foobar.com:12345 Thanks a lot for your quick help!!! Greetz, Tobias From tss at iki.fi Sat Nov 5 18:44:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 18:44:06 +0200 Subject: [Dovecot] Group ACL In-Reply-To: References: Message-ID: <1320511448.21919.257.camel@hurina> On Thu, 2011-10-06 at 08:38 -0700, Daniel L. Miller wrote: > Using Dovecot 2.0, LDAP userdb & passdb, with prefetch: > > hosts = myhost.mydomain > dn = cn=x,cn=y > dnpass = xyz123 > auth_bind = yes > auth_bind_userdn = uid=%n,ou=users,dc=x > ldap_version = 3 > base = ou=Users, dc=x > user_attrs = =home=/var/mail/%d/%n, \ > =mail=mdbox:/var/mail/%d/%n/mdbox, \ > =uid=vmail, \ > =gid=mail Would be good to put acl_groups here also so non-prefetch stuff has access to it also. > user_filter = (&(objectClass=inetOrgPerson)(mail=%u)) > > pass_attrs = mail=user, \ > userPassword=password, \ > =userdb_home=/var/mail/%d/%n, \ > =userdb_mail=mdbox:/var/mail/%d/%n/mdbox, \ > =userdb_uid=vmail, \ > =userdb_gid=mail, \ > =userdb_acl_groups='allshared' I think the problem may simply be those '' characters in there. You're now in "'allshared'" group rather than in "allshared" group.. If that doesn't help: > I have added permissions for "$allshared" to a mailbox's Inbox. It does > not appear in the shared folders list. Other mailboxes, with explicit > permission for a given username, work fine. What am I doing wrong? 1. Make sure that acl_groups setting is enabled for that user by setting mail_debug=yes and looking at the log: imap: Debug: Added userdb setting: plugin/acl_groups=allshared 2. Check with: doveadm acl debug -u user From tss at iki.fi Sat Nov 5 19:20:19 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:20:19 +0200 Subject: [Dovecot] Restarting dovecot periodically fails with "Socket already exists: /var/run/dovecot/anvil" In-Reply-To: <4E97000D.7030908@ubuntu.com> References: <4E97000D.7030908@ubuntu.com> Message-ID: <1320513622.21919.260.camel@hurina> On Thu, 2011-10-13 at 16:13 +0100, James Page wrote: > restarting dovecot regularly fails with version 2.0.13 on Ubuntu 11.10 > due to a running child process exiting after the main dovecot process. > > Oct 13 15:04:58 ubuntu dovecot: master: Error: service(anvil): Socket > already exists: /var/run/dovecot/anvil > This is an Ubuntu specific issue as we use upstart to manage dovecot > which monitors the status of the main dovecot process; the Debian init > script appears to make provision for this happening. http://hg.dovecot.org/dovecot-2.1/rev/6894298ae5fd should apply to v2.0 as well, but since it's a bit large change I don't know if I should backport it, at least without some more testing. From mpapet at yahoo.com Sat Nov 5 19:33:20 2011 From: mpapet at yahoo.com (Michael Papet) Date: Sat, 5 Nov 2011 10:33:20 -0700 (PDT) Subject: [Dovecot] Newbie Question/Feature Request Message-ID: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> Hi, I'm working on a plugin for qpsmtpd (a perl smtp frontend) that delivers mail via dovecot's LDA. I can only get mail delivered by writing the email to file, then calling 'deliver -d someone -f someone-else < /path/to/temp/email' All good, expected behavior. A perl snippet: local $/=undef; open FH, "< ./demo.mail"; my $stringified = ; # this is roughly equivalent to how other LDA queues work. close FH; open (CMD_OUT, " /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom $stringified |"); my $test = ; print $test; The above code errors out because the expected behavior is $stringified is supposed to be a file name. It would be great if deliver could be fed the email as a string so I don't have to add a step of writing to file. Or, maybe there's an undocumented method? Thanks. mpapet From tss at iki.fi Sat Nov 5 19:39:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:39:50 +0200 Subject: [Dovecot] dsync with quotas In-Reply-To: <8762kozbzo.fsf@algae.riseup.net> References: <8762kozbzo.fsf@algae.riseup.net> Message-ID: <1320514790.21919.262.camel@hurina> On Mon, 2011-09-19 at 13:19 -0400, Micah Anderson wrote: > I have been working on converting people from courier maildir -> dovecot > mdbox and during some of the dsync runs I'm seeing the > quota_exceeded_message be printed as an Error: > > dsync(): Error: Can't save message to mailbox INBOX: You are over quota. To avoid losing mail, immediately empty your Trash and Sent folders and \ > delete emails with large attachments. > dsync(): Info: INBOX: Couldn't keep all uids^M > > Its possible the user was over quota on the originating courier side, > but I would still like to migrate their mail proprely to mdbox, but it > seems like being over quota is inhibiting that. Are the quota > calculations including both the maildir files as well as the converted > mdbox files, resulting in a double counting? Depends on how you did the conversion and what quota backend you were using. If they were using different quota files, there shouldn't be double counting. But if for example you did a migration within the same server and quota was set to dict quota, it was possibly being counted double (both thought they owned the quota file). > I'm not entirely sure if the messages above indicate that the migration > failed for that user or not, so I've been manually increasing their > quota, then redoing the dsync mirror until it works properly. > > I wonder if it would be better if I turned off quota entirely during > migration so I don't run into this problem? Added: http://hg.dovecot.org/dovecot-2.0/rev/00157fd523ea http://hg.dovecot.org/dovecot-2.0/rev/0cf3215cdf2d From tss at iki.fi Sat Nov 5 19:43:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 19:43:09 +0200 Subject: [Dovecot] Newbie Question/Feature Request In-Reply-To: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> References: <1320514400.95444.YahooMailClassic@web125407.mail.ne1.yahoo.com> Message-ID: <1320514990.21919.264.camel@hurina> On Sat, 2011-11-05 at 10:33 -0700, Michael Papet wrote: > open (CMD_OUT, " /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom $stringified |"); > my $test = ; > print $test; > > The above code errors out because the expected behavior is $stringified is supposed to be a file name. It would be great if deliver could be fed the email as a string so I don't have to add a step of writing to file. Or, maybe there's an undocumented method? I don't remember exactly how Perl works, and this is untested, but I think it should be something like: open (CMD_OUT, "| /usr/lib/dovecot/deliver -f somedude\@someplace.com -d anotherdude\@destination.dom"); print CMD_OUT $stringified; my $test = ; print $test; The idea is that you write the string to the pipe to deliver that you open, and deliver reads it from stdin. From tss at iki.fi Sat Nov 5 20:00:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 05 Nov 2011 20:00:50 +0200 Subject: [Dovecot] Safely restoring backups In-Reply-To: <87sjn4rg7y.fsf@algae.riseup.net> References: <87sjn4rg7y.fsf@algae.riseup.net> Message-ID: <1320516051.21919.274.camel@hurina> On Fri, 2011-10-07 at 11:09 -0400, Micah Anderson wrote: > I'm trying to determine what the best way to restore mail with mdbox > is. Restoring using maildir was trivial, I just used rsync > --ignore-existing which wrote any mails that were removed and didn't > touch things that already existed[1]. If a mail had changed flag, the maildir file got duplicated, which Dovecot complained about if it noticed it. > With mdbox things have become more > complicated, and I haven't found a way to restore mail that doesn't > result in many message duplicates. Do you need to restore mails so often that this is really a problem? :) > 1. I tried rsyncing the different backup directories back to the mail > storage host, and then doing: > > 'dsync -R backup -u $user mdbox:/path/to/to/daily.1/mdbox'[2] > 'dsync -R backup -u $user mdbox:/path/to/to/daily.2/mdbox' > > This works ok, but it gives duplicates of every mail that already exists > for every daily/weekly I dsync. This is what the rsync --ignore-existing > avoided. This is particularly annoying if I restore the weekly, and > multiple daily directories, because then you get a copy for every one > you restore. I had thought that the individual message's GIDs would keep > them from being duplicated? GUIDs can be used to identify messages, but there's no automatic deduplication. It's fine to e.g. copy a message from INBOX to INBOX, which duplicates it. Dovecot shouldn't prevent that. > Is there a way I can restore things from backup and avoid duplicates? > Is there another method I should try? Here's one way, although somewhat slow (and not fully tested): doveadm -f tab fetch -u user at domain 'mailbox guid' all | sort > guids1 doveadm -f tab fetch -o mdbox:/backups/user -u user at domain 'mailbox-guid guid' all | sort > guids2 diff -u guids1 guids2 | grep '^+[+]' | sed 's/^+//' | awk { system("doveadm import -u user at domain mdbox:/backups/user restored mailbox-guid "$1" guid "$2); } > 3. why doesn't doveadm import have a -s option to subscribe? I suppose it could.. Added to v2.1: http://hg.dovecot.org/dovecot-2.1/rev/afec4ceda8e1 From apm at one.com Sat Nov 5 21:17:04 2011 From: apm at one.com (Peter Mogensen) Date: Sat, 05 Nov 2011 20:17:04 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EB58BB0.2090900@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: >> Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted >> transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: >> indexid changed 1320419300 -> 1320419441 (sync_offset=0) > > Session A had TestMails open and created with index file whose ID was > 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 > 17:08:20 EET 2011). > > Session B came and recreated the index files 141 seconds later with ID > 1320419441. Either it didn't see A's original index files for some > reason or it simply decided to recreate them for some reason. Either way > this shouldn't have happened. > > Session A then notices that the indexes were recreated, and logs an > error. Oh... wait a minute... The timestamp is UTC, so 17:08:20 is about 2:22 before the log line. 2:22 is 142 seconds. So... given that the errors doesn't appear every time the client runs the series of APPEND requests, but (now I come to think of it) probably never the first time he runs it, but the second time - and that he did run the script a few minutes before this log line with out errors, - then... the problem might be that the first run of the script doesn't finish correctly. If session A is the first run of the script, then it should have finished and logged out long before session B. But maybe the problem is the first run not finishing properly. /Peter From p at state-of-mind.de Sat Nov 5 21:43:50 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sat, 5 Nov 2011 20:43:50 +0100 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <1320508907.21919.239.camel@hurina> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> Message-ID: <20111105194350.GA3782@state-of-mind.de> * Timo Sirainen : > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > is there a way to use Dovecot SASL Authentication for submission on a > > different machine dovecot runs? > > With v2.0 you should be able to add inet_listener for auth service, > which acts as an auth-client socket. > > > Dovecot machine <-> postfix machine(for submission) > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. Document it and Wietse will support it on the mailing list. If its not documented support will be declined. Should I? p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From lists at wildgooses.com Sat Nov 5 22:09:38 2011 From: lists at wildgooses.com (Ed W) Date: Sat, 05 Nov 2011 20:09:38 +0000 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2C704.1010107@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> Message-ID: <4EB59802.9040205@wildgooses.com> On 03/11/2011 16:53, Patrick Westenberg wrote: > Ed W schrieb: > >>> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >>> thinking about a SSD based LUN for the indexes. As I'm using multiple >>> servers this LUN will use OCFS2. >> >> Given that the SAN always has the network latency behind it, might you >> be better to look at putting the SSDs in the frontend machines? >> Obviously this then needs some way to make users "sticky" to one machine >> (or some few machines) where the indexes are stored? > > Storing the indexes on several machines? > In this case I have to synchronize them. > See the "sticky" in my reply. You use one of several techniques to ensure that users always end up on the server with the indexes on. That way much of the IO is served from that local machine and you only access the SAN for the (in theory much less frequent) access to the mail files themselves. Clearly if the machine with the indexes on dies then the load balancer needs to pick a new machine and there will be delay/io/etc while the indexes are regenerated. Various techniques could mitigate this... I don't have such a larger system - please ignore all my advice... The basis for the suggestion is that I understand file access (locking in particular) is "expensive" on OCFS2/GFS. Therefore I read here on this list that others have found performance issues accessing maildir over OCFS2? It's also not hard to find benchmarks that show OCFS2/GFS are "fast", but slower than accessing the same storage without using a cluster filesystem - this makes sense. Hence it seems like a trade between convenience of storing everything on a central store and "some" performance improvement from a more complex system... I think if you search on benchmarks of DRBD vs OCFS2 and read here on the list about the "director" and "proxy" services you can see the point? I'm just trying to help you see the effects you might want to measure! (I don't have a system large enough to know much about this stuff from experience...) Good luck! Ed W From dovecot at knutejohnson.com Sun Nov 6 00:15:02 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Sat, 05 Nov 2011 15:15:02 -0700 Subject: [Dovecot] Converting from 1.12 to 2.0 questions about mail location? Message-ID: <4EB5B566.7070808@knutejohnson.com> I'm building a new server and using a newer version of ubuntu server that uses Dovecot 2.0. I thought I would like to use Maildir instead of the mbox that was used in the previous server. What do I have to do to convert my mbox mail to Maildir? Or should I just use mbox and copy it over :-)? Thanks, -- Knute Johnson From stan at hardwarefreak.com Sun Nov 6 00:46:57 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 05 Nov 2011 17:46:57 -0500 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB2B15D.1010005@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> Message-ID: <4EB5BCE1.3020608@hardwarefreak.com> On 11/3/2011 10:21 AM, Ed W wrote: > >> I'm using NexentaStor (Solaris, ZFS) to export iSCSI-LUNs and I was >> thinking about a SSD based LUN for the indexes. As I'm using multiple >> servers this LUN will use OCFS2. > > Given that the SAN always has the network latency behind it, might you > be better to look at putting the SSDs in the frontend machines? The latency of the GbE IP network, iSCSI HBAs, GbE switches, etc, is but a fraction of the overhead of the out of band OCFS metadata exchange between cluster members, and the general overhead of OCFS, or of cluster filesystems in general. > Obviously this then needs some way to make users "sticky" to one machine > (or some few machines) where the indexes are stored? > > This seems theoretically likely to give you higher IOPs to the index > than having them on the OCFS2 storage? (At a trade off with more > complexity for the load balancer front end...) Following this logic, simply using local mechanical disk would yield improvement without the cost of SSDs. Depending on the number of nodes, putting a couple of SSDs in the SAN controller may likely be cheaper overall than adding mech disks to each node, let alone SSDs to each node. The random IO latency of SSD is so considerably lower than mechanical disk, even with the OCFS and iSCSI SAN overhead, overall read/write latency will likely be lower than using local mech disk in the nodes. And you get to retain centralized storage of the indexes, eliminating stickiness complexity issues. Something else to consider is the read/write caching performance of NexentaStor (I've never used it, know nothing about it). If it's very good, and the NexentaStor host has gobs of RAM (think 64-128GB), then adding SSDs for indexes may not improve performance much, if any, depending on the concurrent user load. I've read cases where adding SLC-SSD to high end FC SAN controllers with gobs of writeback cache RAM yielded little benefit with similar random IO workloads, simply because the cache was never taxed enough to force regular flushing. If your cache is large and fast enough to buffer most of your IOPS, then your current spindle speed is already irrelevant. In such a case adding SSD will yield little, or no, advantage. -- Stan From knute2011 at knutejohnson.com Sun Nov 6 01:35:46 2011 From: knute2011 at knutejohnson.com (Knute Johnson) Date: Sat, 05 Nov 2011 16:35:46 -0700 Subject: [Dovecot] Converting from 1.12 to 2.0 questions about mail location? In-Reply-To: <4EB5B566.7070808@knutejohnson.com> References: <4EB5B566.7070808@knutejohnson.com> Message-ID: <4EB5C852.5060301@knutejohnson.com> On 11/5/2011 3:15 PM, Knute Johnson wrote: > I'm building a new server and using a newer version of ubuntu server > that uses Dovecot 2.0. I thought I would like to use Maildir instead of > the mbox that was used in the previous server. What do I have to do to > convert my mbox mail to Maildir? Or should I just use mbox and copy it > over :-)? > > Thanks, > After looking at it some more, I think I'm just going to keep the mbox format mail. Thanks, -- Knute Johnson knute2011 at knutejohnson.com From e-frog at gmx.de Sun Nov 6 18:03:47 2011 From: e-frog at gmx.de (e-frog) Date: Sun, 06 Nov 2011 17:03:47 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub Message-ID: <4EB6AFE3.3020001@gmx.de> Hi Timo, I'm using the virtual plugin and have 2 mailboxes configured in the "virtual" namespace (dovecot -n attached). Using lsub with dovecot 2.0.15 leads to the following result: 1 lsub "" "virtual/*" * LSUB () "/" "virtual/Lists" * LSUB () "/" "virtual/unread" 1 OK Lsub completed. However with dovecot 2.1 (latest hg as of today) and the same config there is a 3rd mailbox "virtual" listed which doesn't exist: 1 lsub "" "virtual/*" * LSUB () "/" "virtual" * LSUB () "/" "virtual/Lists" * LSUB () "/" "virtual/unread" 1 OK Lsub completed. Thunderbird shows the "virtual" mailbox as selectable and not grayed out as with 2.0.15. Clicking on that mailbox gives an error message "Mailbox doesn't exist: virtual". Thanks, e-frog -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf-n.2.0.15.txt URL: From mds at schinhammer.de Mon Nov 7 01:54:28 2011 From: mds at schinhammer.de (Michael Schinhammer) Date: Mon, 07 Nov 2011 00:54:28 +0100 Subject: [Dovecot] Problems with password verification on a BSD box against PAM Message-ID: <4EB71E34.5050706@schinhammer.de> Spam detection software, running on the system "ns.089.at", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hi ML, i want to use my local users database on my Freebsd 8.2 box and got the following error in the maillog. The password is ok and it working on the console. i can establish the ssl connection after login "a login ux password" the error occures i cant login. [...] Content analysis details: (6.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.2 MR_NOT_ATTRIBUTED_IP Beta rule: an non-attributed IPv4 found in headers 0.6 J_CHICKENPOX_65 BODY: 6alpha-pock-5alpha 0.6 J_CHICKENPOX_73 BODY: 7alpha-pock-3alpha 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS 0.0 NO_RDNS2 Sending MTA has no reverse DNS 3.5 TO_NO_BRKTS_DIRECT To: misformatted and direct-to-MX -------------- next part -------------- An embedded message was scrubbed... From: Michael Schinhammer Subject: Problems with password verification on a BSD box against PAM Date: Mon, 07 Nov 2011 00:54:28 +0100 Size: 2066 URL: From pw at wk-serv.de Mon Nov 7 02:08:23 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 07 Nov 2011 01:08:23 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB59802.9040205@wildgooses.com> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> Message-ID: <4EB72177.8090801@wk-serv.de> Ed W schrieb: > See the "sticky" in my reply. You use one of several techniques to > ensure that users always end up on the server with the indexes on. That > way much of the IO is served from that local machine and you only access > the SAN for the (in theory much less frequent) access to the mail files > themselves. I know you can afford that (IMAP) users always end up on one particular server but afaik this only works for incoming IMAP connections. My mail exchangers use dovecot-lda and I think indexes will be written from these servers too or am I wrong with this? Patrick From komodo at uvt.cz Mon Nov 7 12:10:20 2011 From: komodo at uvt.cz (komodo) Date: Mon, 7 Nov 2011 11:10:20 +0100 Subject: [Dovecot] Missing public folder Message-ID: <201111071110.20143.komodo@uvt.cz> Hi i am using dovecot 2.0.13 and i can't see public folder in my mail client. But what is strange, it was working after instalation, and i don't know when it dissapeared. Maybe after some upgrade ? When i enable debug, then i can see in my log that folder exists and no errors. Please where should i find the problem ? Thanks Martin Here is log Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: Namespace : type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: maildir++: root=/home/komodo/Maildir, index=, control=, inbox=/home/komodo/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: Namespace : type=public, prefix=spam_learner., sep=., inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/shared/Maildir Nov 7 11:00:27 OVZ dovecot: imap(komodo): Debug: maildir++: root=/home/shared/Maildir, index=, control=, inbox= Here is my conf # 2.0.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab037.1 x86_64 CentOS release 5.5 (Final) default_client_limit = 1300 disable_plaintext_auth = no hostname = somehost.com mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } namespace { location = maildir:/home/shared/Maildir prefix = spam_learner. separator = . subscriptions = no type = public } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/before.d/ sieve_dir = ~/sieve } protocols = imap pop3 sieve sendmail_path = /usr/sbin/sendmail.exim service imap-login { process_limit = 1000 process_min_avail = 30 } ssl_cert = References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> Message-ID: <4EB7BF9A.9020900@webfusion.com> 04-11-2011 23:56, Timo Sirainen yazm??: > On Thu, 2011-10-20 at 12:38 +0300, Mark Zealey wrote: > >> I'm currently changing some systems to use the redirector service, which >> means that to get local deliveries going I need to get lmtp set up so it >> can be redirected as well. This is working fine, however we have a >> number of different ports running on our servers depending on which >> brand a customer is using. When the time comes to do the user look up, >> we use (port, user, domain) to do a unique database lookup which works >> fine for everything (proxied pop, imap, lmtp& straight pop, imap) but >> not straight lmtp - perhaps because it doesn't have a 2-stage login >> process unlike the pop/imap protocols. Looking at the SQL query we are >> issuing, the %a port variable is set to 0 even though the connection is >> coming in from TCP. Is there a way to change this? We are running >> dovecot 2.0.12 however looking through the changelogs I can't see this >> would be fixed in the newest version. > A quick check in code shows that this is already supposed to work. Set > auth_debug=yes and see if LMTP's auth lookup sends lip, rip, lport and > rport fields to auth process? From my reading of the code in 2.0.15, you pass the local & remote IP in lmtp/commands.c:427 but the struct mail_storage_service_input only has remote_ip and local_ip fields, not port fields (unlike the auth_user_info struct which has both ip's and port entries as well). I think that the mail_storage_service_input struct needs the uint local_port, remote_port fields adding in & appropriate code changes to pass these through? Mark From gordon.grubert at uni-greifswald.de Sun Nov 6 20:56:05 2011 From: gordon.grubert at uni-greifswald.de (Gordon Grubert) Date: Sun, 06 Nov 2011 19:56:05 +0100 Subject: [Dovecot] Dovecot crashes totally In-Reply-To: <1320435812.21919.150.camel@hurina> References: <4EA317B5.3090209@uni-greifswald.de> <1320435812.21919.150.camel@hurina> Message-ID: <4EB6D845.7040208@uni-greifswald.de> On 11/04/2011 08:43 PM, Timo Sirainen wrote: > On Sat, 2011-10-22 at 21:21 +0200, Gordon Grubert wrote: >> Hello, >> >> our dovecot server crashes totally without any really useful >> log messages. The error log can be found in the attachment. >> The only way to get dovecot running again is a complete >> system restart. > > How often does it break? If really a "complete system restart" is needed > to fix it, it doesn't sound like a Dovecot problem. Check if it's enough > to stop dovecot and then make sure there aren't any dovecot processes > lying around afterwards. Currently, the problem occurred three times. The last time some days ago. The last "crash" was in the night and, therefore, we used the chance for a detailed debugging of the system. You could be right, that it's not a dovecot problem. Next to dovecot, we found other processes hanging and could not be killed by "kill -9". Additionally, we found a commonness of all of these processes: They hanged while trying to access the mailbox volume. Therefore, we repaired the filesystem. Now, we're watching the system ... >> Oct 11 09:55:23 mailserver2 dovecot: master: Error: service(imap): >> Initial status notification not received in 30 seconds, killing the >> process >> Oct 11 09:56:23 mailserver2 dovecot: imap-login: Error: master(imap): >> Auth request timed out (received 0/12 bytes) > > Kind of looks like auth process is hanging. You could see if stracing it > shows anything useful. Also are any errors logged about LDAP? Is LDAP > running on the same server? Dovecot authenticates against postfix and postfix has an LDAP connection. The LDAP is running on an external cluster. Here, no errors are reported. We hope, that the filesystem error was the reason for the problem and, that the problem is fixed by repairing it. Best regards, Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5396 bytes Desc: S/MIME Cryptographic Signature URL: From yuichi at sklc.co.jp Mon Nov 7 17:12:33 2011 From: yuichi at sklc.co.jp (Yuuichi Ikeda (SKLC)) Date: Tue, 08 Nov 2011 00:12:33 +0900 Subject: [Dovecot] POP3/IMAPv4 CRAM-MD5 Authentication failed.(Re-post) Message-ID: <20111108001232.2ACB.1986BE3C@sklc.co.jp> Hi, I'm Yuuichi Ikeda from Japan User. OS:Solaris 10 9/10 s10x_u9wos_14a X86 Mem:8GB HDD:3TB gcc:gcc (GCC) 4.1.2 gcc-prefix:/unsupported/gcc Dovecot Version:2.0.15 configure:./configure --prefix=/opt/dovecot_2 --sysconfdir=/opt/dovecot_2/conf --mandir=/opt/man --enable-shared --with-mysql --with-zlib --with-sqlite --with-sql=plugin --with-ssldir=/opt/openssl --with-rundir=/var/run --with-libiconv-prefix=/opt/libiconv Dovecot Configuration > # 2.0.15: /opt/dovecot_2/conf/dovecot/dovecot.conf > # OS: SunOS 5.10 i86pc > auth_debug = yes > auth_mechanisms = cram-md5 > auth_ssl_require_client_cert = yes > auth_ssl_username_from_cert = yes > auth_verbose = yes > base_dir = /var/run/dovecot/ > doveadm_worker_count = 10 > log_path = /var/log/dovecot/dovecot.log > login_greeting = ready. > login_trusted_networks = 192.168.1.0/24 > mail_location = maildir:~/Maildir > passdb { > driver = pam > } > passdb { > args = /opt/dovecot_2/conf/dovecot/passwd > driver = passwd-file > } > plugin { > acl = vfile:/opt/dovecot_2/conf/dovecot/global-acls:cache_secs=300 > acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes > } > protocols = imap pop3 > service auth { > executable = /opt/dovecot_2/libexec/dovecot/auth > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > } > service imap-login { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/imap-login > inet_listener imap { > port = 143 > ssl = no > } > inet_listener imaps { > port = 993 > ssl = yes > } > } > service imap { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/imap > } > service lmtp { > unix_listener lmtp { > mode = 0666 > } > } > service pop3-login { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/pop3-login > inet_listener pop3 { > port = 110 > ssl = no > } > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service pop3 { > executable = /opt/dovecot_2/libexec/dovecot/rawlog /opt/dovecot_2/libexec/dovecot/pop3 > } > ssl_ca = ssl_cert = ssl_key = ssl_verify_client_cert = yes > userdb { > args = blocking=yes > driver = passwd > } > protocol imap { > imap_logout_format = bytes=%i/%o > imap_max_line_length = 64 k > mail_max_userip_connections = 10 > mail_plugins = > } > protocol lda { > hostname = mailsv.sklc.co.jp > info_log_path = /var/log/dovecot/deliver.log > log_path = /var/log/dovecot/deliver.log > mail_plugins = > postmaster_address = postmaster at sklc.co.jp > sendmail_path = /usr/lib/sendmail > } > protocol lmtp { > mail_plugins = > } > protocol pop3 { > mail_plugins = > pop3_save_uidl = yes > pop3_uidl_format = %v-%u > } If it attests by connecting by POP3 or IMAPv4, the following messages will be displayed and attestation will go wrong. > Nov 07 23:12:40 auth: Debug: auth client connected (pid=20018) > Nov 07 23:12:40 auth: Debug: client in: AUTH 1 CRAM-MD5 service=pop3 secured no-penalty lip=192.168.1.1 rip=192.168.1.110 lport=110 rport=57054 > Nov 07 23:12:40 auth: Info: CRAM-MD5(?,192.168.1.110): Client didn't present valid SSL certificate > Nov 07 23:12:40 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate > Nov 07 23:12:40 pop3-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.110, lip=192.168.1.1, secured > Nov 07 23:16:32 auth: Debug: auth client connected (pid=20126) > Nov 07 23:16:32 auth: Debug: client in: AUTH 1 CRAM-MD5 service=imap secured no-penalty lip=192.168.1.1 rip=192.168.1.1 lport=143 rport=58734 > Nov 07 23:16:32 auth: Info: CRAM-MD5(?,192.168.1.1): Client didn't present valid SSL certificate > Nov 07 23:16:32 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate > Nov 07 23:16:32 imap-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.1, lip=192.168.1.1, secured What will you do and will become like this? If some people know ways of coping, please let me know. ============================================================= Information-system part. Sankei-Koumuten Co.,Ltd. Yuuichi Ikeda Mail:yuichi at sklc.co.jp Tel.+81-3-3623-6474 Fax.+81-3-3623-6475 Our company promotes "Team minus 6 percent" jus, Hatena Joined member. LPIC-2 Certified. ============================================================= From micah at riseup.net Mon Nov 7 19:13:07 2011 From: micah at riseup.net (micah anderson) Date: Mon, 07 Nov 2011 12:13:07 -0500 Subject: [Dovecot] Accessing a strange mailbox In-Reply-To: <1320435243.21919.144.camel@hurina> References: <8762jie6ut.fsf@algae.riseup.net> <1320435243.21919.144.camel@hurina> Message-ID: <87aa87onz0.fsf@algae.riseup.net> Hi, thanks for the reply! On Fri, 04 Nov 2011 21:34:03 +0200, Timo Sirainen wrote: > On Fri, 2011-10-21 at 10:50 -0400, Micah Anderson wrote: > > I have a user who has a mailbox called: > > > > A----->B > > > > It seemed to work in courier fine, they managed to create it, and there > > are mails in it. However, dovecot is not letting the user access it, the > > IMAP server gives an error. I tried to rename it but I would also get an > > error: > > > > # doveadm mailbox rename 'A----->B' AtoB > > doveadm(root): Error: Can't rename mailbox A----->B to AtoB: Mailbox doesn't exist: A----->B > > Maybe it's not named that? What does it look like in filesystem? Worked > fine with me: I moved the user from courier maildir to mdbox, and I've still got the maildir folders around, the filesystem shows it as: drwx------ 6 mail mail 4096 Oct 20 07:56 .A----->B I just realized my mistake, my 'doveadm mailbox rename' command wasn't limited to the particular user (I was not passing -u ) so it was looking in the wrong namespace for that user. Once I specified the user, it worked properly. Micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From user+dovecot at localhost.localdomain.org Mon Nov 7 22:13:25 2011 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 07 Nov 2011 21:13:25 +0100 Subject: [Dovecot] POP3/IMAPv4 CRAM-MD5 Authentication failed.(Re-post) In-Reply-To: <20111108001232.2ACB.1986BE3C@sklc.co.jp> References: <20111108001232.2ACB.1986BE3C@sklc.co.jp> Message-ID: <4EB83BE5.5030006@localhost.localdomain.org> On 11/07/2011 04:12 PM Yuuichi Ikeda (SKLC) wrote: > Hi, I'm Yuuichi Ikeda from Japan User. > > > Dovecot Configuration >> ? >> ssl_ca = > ssl_cert = > ssl_key = > ssl_verify_client_cert = yes > ? > If it attests by connecting by POP3 or IMAPv4, the following messages will > be displayed and attestation will go wrong. > >> Nov 07 23:12:40 auth: Info: CRAM-MD5(?,192.168.1.110): Client didn't present valid SSL certificate >> Nov 07 23:12:40 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate >> Nov 07 23:12:40 pop3-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.110, lip=192.168.1.1, secured > >> Nov 07 23:16:32 auth: Info: CRAM-MD5(?,192.168.1.1): Client didn't present valid SSL certificate >> Nov 07 23:16:32 auth: Debug: client out: FAIL 1 reason=Client didn't present valid SSL certificate >> Nov 07 23:16:32 imap-login: Info: Aborted login (cert required, client didn't start TLS): method=CRAM-MD5, rip=192.168.1.1, lip=192.168.1.1, secured > > What will you do and will become like this? If some people know ways of > coping, please let me know. Are you sure you want to verify the client's certificate (ssl_verify_client_cert = yes)? If not, just remove this line and try again. Regards, Pascal -- The trapper recommends today: cafebabe.1131121 at localdomain.org From alietsantiesteban at gmail.com Tue Nov 8 01:02:23 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Mon, 7 Nov 2011 18:02:23 -0500 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: <1320440605.21919.198.camel@hurina> References: <1320440605.21919.198.camel@hurina> Message-ID: We checked with the firewall admins and they can not change the drop action, this model doesn't support reject, only drops, but for testing they disabled the ldap protocol idle timeout wich was set to 30 mins to never so the firewall never drops ldap idle connections, we also verified the clientidletimeout option in Openldap but is set to 0 wich means never close a idle connection. After testing again we see the connection hanging again after user inactivity, we will keep looking for other issues and maybe do some packet captures to see what is really happening. best regards, btw it would be great this ldap_idle_disconnect = 30s 2011/11/4 Timo Sirainen > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes wrote: > > I'm having a problem with dovecot ldap connection when ldap server is in > > another firewall zone, firewall kills the ldap connection after a > > determined period of inactivity, this is good from the firewall point of > > view but is bad for dovecot because it never knows the connections has > been > > dropped, this creates longs timeouts in dovecot and finally it > reconnects, > > meanwhile many users fails to authenticate, I have seen this kind of post > > in the list for a while but can't find a solution for it, so my question > is > > how to define a idle ldap time in dovecot so it can reconnect before the > > firewall has dropped the connection or just close the connection under > > inactivity so when a user authenticate doesn't fails for a while until > > dovecot detects that the connection has hanged. Is this a feature request > > or there is already a configuration for this??? > > Can't the firewall be changed to reject the LDAP packets instead of > dropping them? Then Dovecot would immediately notice that the connection > has died, and with a recent enough version it wouldn't even log an error > about it. > > I guess some kind of an "ldap_idle_disconnect = 30s" setting could be > added, but it's not a very high priority for me. > > > From tss at iki.fi Tue Nov 8 01:11:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 08 Nov 2011 01:11:02 +0200 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: References: <1320440605.21919.198.camel@hurina> Message-ID: <1320707463.21919.337.camel@hurina> If you set openldap server to close idle clients sooner than the connection itself is dropped by firewall (or whatever), then Dovecot sees the disconnection and won't hang. So you could try something like clientidletimeout=5 mins On Mon, 2011-11-07 at 18:02 -0500, Aliet Santiesteban Sifontes wrote: > We checked with the firewall admins and they can not change the drop > action, this model doesn't support reject, only drops, but for testing > they disabled the ldap protocol idle timeout wich was set to 30 mins > to never so the firewall never drops ldap idle connections, we also > verified the clientidletimeout option in Openldap but is set to 0 wich > means never close a idle connection. After testing again we see the > connection hanging again after user inactivity, we will keep looking > for other issues and maybe do some packet captures to see what is > really happening. > best regards, btw it would be great this ldap_idle_disconnect = 30s > > 2011/11/4 Timo Sirainen > > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes > wrote: > > I'm having a problem with dovecot ldap connection when ldap > server is in > > another firewall zone, firewall kills the ldap connection > after a > > determined period of inactivity, this is good from the > firewall point of > > view but is bad for dovecot because it never knows the > connections has been > > dropped, this creates longs timeouts in dovecot and finally > it reconnects, > > meanwhile many users fails to authenticate, I have seen this > kind of post > > in the list for a while but can't find a solution for it, so > my question is > > how to define a idle ldap time in dovecot so it can > reconnect before the > > firewall has dropped the connection or just close the > connection under > > inactivity so when a user authenticate doesn't fails for a > while until > > dovecot detects that the connection has hanged. Is this a > feature request > > or there is already a configuration for this??? > > > Can't the firewall be changed to reject the LDAP packets > instead of > dropping them? Then Dovecot would immediately notice that the > connection > has died, and with a recent enough version it wouldn't even > log an error > about it. > > I guess some kind of an "ldap_idle_disconnect = 30s" setting > could be > added, but it's not a very high priority for me. > > > From alietsantiesteban at gmail.com Tue Nov 8 04:41:12 2011 From: alietsantiesteban at gmail.com (Aliet Santiesteban Sifontes) Date: Mon, 7 Nov 2011 21:41:12 -0500 Subject: [Dovecot] How to define ldap connection idle In-Reply-To: <1320707463.21919.337.camel@hurina> References: <1320440605.21919.198.camel@hurina> <1320707463.21919.337.camel@hurina> Message-ID: We will try this as next step to find a workaround, the problem with client idletimeout=5 mins in openldap server is that is a global server definition and have the net effect of changing replication refreshAndPersit into type refreshOnly which is not a welcome side effect, we will look other options, still the better candidate is ldap_idle_disconnect in dovecot side or any other kind of logic able to detect this kind of problems. best regards 2011/11/7 Timo Sirainen > If you set openldap server to close idle clients sooner than the > connection itself is dropped by firewall (or whatever), then Dovecot > sees the disconnection and won't hang. So you could try something like > clientidletimeout=5 mins > > On Mon, 2011-11-07 at 18:02 -0500, Aliet Santiesteban Sifontes wrote: > > We checked with the firewall admins and they can not change the drop > > action, this model doesn't support reject, only drops, but for testing > > they disabled the ldap protocol idle timeout wich was set to 30 mins > > to never so the firewall never drops ldap idle connections, we also > > verified the clientidletimeout option in Openldap but is set to 0 wich > > means never close a idle connection. After testing again we see the > > connection hanging again after user inactivity, we will keep looking > > for other issues and maybe do some packet captures to see what is > > really happening. > > best regards, btw it would be great this ldap_idle_disconnect = 30s > > > > 2011/11/4 Timo Sirainen > > > > On Thu, 2011-11-03 at 11:52 -0400, Aliet Santiesteban Sifontes > > wrote: > > > I'm having a problem with dovecot ldap connection when ldap > > server is in > > > another firewall zone, firewall kills the ldap connection > > after a > > > determined period of inactivity, this is good from the > > firewall point of > > > view but is bad for dovecot because it never knows the > > connections has been > > > dropped, this creates longs timeouts in dovecot and finally > > it reconnects, > > > meanwhile many users fails to authenticate, I have seen this > > kind of post > > > in the list for a while but can't find a solution for it, so > > my question is > > > how to define a idle ldap time in dovecot so it can > > reconnect before the > > > firewall has dropped the connection or just close the > > connection under > > > inactivity so when a user authenticate doesn't fails for a > > while until > > > dovecot detects that the connection has hanged. Is this a > > feature request > > > or there is already a configuration for this??? > > > > > > Can't the firewall be changed to reject the LDAP packets > > instead of > > dropping them? Then Dovecot would immediately notice that the > > connection > > has died, and with a recent enough version it wouldn't even > > log an error > > about it. > > > > I guess some kind of an "ldap_idle_disconnect = 30s" setting > > could be > > added, but it's not a very high priority for me. > > > > > > > > > From Yann.Dupont at univ-nantes.fr Tue Nov 8 12:05:42 2011 From: Yann.Dupont at univ-nantes.fr (Yann Dupont) Date: Tue, 08 Nov 2011 11:05:42 +0100 Subject: [Dovecot] Attachements with SIS : shauld all the same files share the same inode ? In-Reply-To: <1320510194.21919.251.camel@hurina> References: <4E9BD704.9080801@univ-nantes.fr> <1320510194.21919.251.camel@hurina> Message-ID: <4EB8FEF6.7000208@univ-nantes.fr> Hi timo, thanks for your answer Le 05/11/2011 17:23, Timo Sirainen a ?crit : > On Mon, 2011-10-17 at 09:19 +0200, Yann Dupont wrote: >> 104 instance of -rw------- 104 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-ffd0d1xxxx >> >> The files are hardlinked, so it seems ok, but I also see 9 others files >> in the same directory : >> >> >> -rw------- 1 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-0519df2a94fc9a4e1f4100008a6b99b6 >> -rw------- 1 vmail vmail 346584 16 oct. 17:47 >> 6e2df299fa96daec9b4735d07c494046429be4d6-1790b92994fc9a4e1a4100008a6b99b6 >> >> as you can see, same prefix, If I compare them with diff, they are all >> the same files. > > I guess you have "mail_attachment_fs = sis posix" (default)? Sounds like yes that's the case. > there could be a bug where SIS does comparison of files but for some > reason thinks they are different. > Well I can't try anymore on this, because this particular attachment is not there. In fact this server keeps only 2 weeks of mails for 5000+ accounts (a form of backup for the main servers). I'll take a new one : -> rw------- 103 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c0c2230ecdd1b74ec91600008a6b99b6 -rw------- 103 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c1c1290ecdd1b74ec51600008a6b99b6 -rw------- 1 vmail vmail 571037 7 nov. 13:40 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 So f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 has only one link, but diff f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c1c1290ecdd1b74ec51600008a6b99b6 f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 tell nothing : they are same files. > You could try if you can merge them with: > > mkdir /tmp/queue > touch /tmp/queue/6e2df299fa96daec9b4735d07c494046429be4d6-foo > doveadm sis deduplicate /var/attachments /tmp/queue > > hum I didn't get what you want to do ? Does putting a 0 byte file there force a complete rescan ? Tried it : the 0 byte length /tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo disappear from the /tmp/queue, but f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 still has one link. A strace reveal this stat("/vmail/attachments-deduplic", {st_mode=S_IFDIR|0700, st_size=8192, ...}) = 0 open("/tmp/queue", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 8 getdents(8, /* 3 entries */, 32768) = 112 stat("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 link("/vmail/attachments-deduplic/f7/c9/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo", "/vmail/attachments-deduplic/f7/c9/hashes/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667") = -1 ENOENT (No such file or directory) mkdir("/vmail/attachments-deduplic/f7/c9/hashes", 0700) = -1 EEXIST (File exists) unlink("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-foo") = 0 getdents(8, /* 0 entries */, 32768) = 0 close(8) = 0 Tried with a copy of the file but this doesn't work either : stat("/vmail/attachments-deduplic", {st_mode=S_IFDIR|0700, st_size=8192, ...}) = 0 open("/tmp/queue", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 8 getdents(8, /* 3 entries */, 32768) = 144 stat("/tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6", {st_mode=S_IFREG|0600, st_size=571037, ...}) = 0 write(2, "doveadm(root): Fatal: /tmp/queue"..., 170doveadm(root): Fatal: /tmp/queue/f7c9746ed59fd838cbbfca08cde3c33b6b7ab667-c67f0e29bad1b74e7e1500008a6b99b6 is not a valid sis-queue file, is the queue directory correct? How can I help further ? Thanks, -- Yann Dupont - Service IRTS, DSI Universit? de Nantes Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont at univ-nantes.fr From listas.correo at yahoo.es Tue Nov 8 12:20:22 2011 From: listas.correo at yahoo.es (mailing lists) Date: Tue, 8 Nov 2011 10:20:22 +0000 (GMT) Subject: [Dovecot] Failing to share folders when listescape is enabled (2.0.14) Message-ID: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> Hello, How I can share a folder with dots when the listescape plugin is enabled? In this example user001 is sharing two foders named "docs-abc" and "docs-a.b.c" to user002. The first folder (without dots) is seen by user002 but the second is not found by dovecot because it search a system folder named ".docs-a.b.c" but the system folder is ".docs-a\2eb\2ec" Nov? 8 11:03:52 imap2 dovecot: imap(user002): Debug: acl vfile: file /var/virtual-maildir/user001/.docs-a.b.c/dovecot-acl not found drwx------ 2 vmail vmail 4096 2011-11-08 10:54 cur drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .docs-a\2eb\2ec drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .docs-abc -rw------- 1 vmail vmail?? 20 2011-11-08 10:57 dovecot-acl-list -rw------- 1 vmail vmail? 248 2011-11-08 10:55 dovecot.index.log -rw------- 1 vmail vmail?? 96 2011-11-08 10:54 dovecot.mailbox.log -rw------- 1 vmail vmail?? 51 2011-11-08 10:55 dovecot-uidlist -rw------- 1 vmail vmail??? 8 2011-11-08 10:57 dovecot-uidvalidity -r--r--r-- 1 vmail vmail??? 0 2011-11-08 10:54 dovecot-uidvalidity.4eb8fc5b drwx------ 5 vmail vmail 4096 2011-11-08 10:57 .Drafts -rw------- 1 vmail vmail?? 15 2011-11-08 10:54 maildirsize drwx------ 2 vmail vmail 4096 2011-11-08 10:54 new drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Sent drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Spam -rw------- 1 vmail vmail?? 23 2011-11-08 10:54 subscriptions drwx------ 2 vmail vmail 4096 2011-11-08 10:54 tmp drwx------ 5 vmail vmail 4096 2011-11-08 10:54 .Trash # cat /var/virtual-maildir/user001/subscriptions Trash Sent Drafts Spam docs-abc docs-a\2eb\2ec # telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. . login user001 XXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in . create docs-abc??? . OK Create completed. . create docs-a.b.c . OK Create completed. . setacl docs-abc user002 lrsw . OK Setacl complete. . setacl docs-a.b.c user002 lrsw . OK Setacl complete. . subscribe docs-abc . OK Subscribe completed. . subscribe docs-a.b.c . OK Subscribe completed. . logout * BYE Logging out . OK Logout completed. Connection closed by foreign host. # telnet localhost 143 Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. . login user002 XXXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL RIGHTS=texk] Logged in . subscribe shared/user001/docs-abc . OK Subscribe completed. . subscribe shared/user001/docs-a.b.c . NO Mailbox doesn't exist: shared/user001/docs-a.b.c . logout * BYE Logging out . OK Logout completed. Connection closed by foreign host. ??? /-----------------/ # dovecot -n # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34.7-0.7-xen x86_64 openSUSE 11.3 (x86_64) auth_debug = yes auth_verbose = yes base_dir = /var/run/dovecot/ dict { ? expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_proxy_port = 24244 doveadm_socket_path = localhost:24244 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_debug = yes mail_fsync = always mail_gid = 5000 mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = acl quota notify expire listescape mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { ? inbox = yes ? list = yes ? location = ? prefix = ? separator = / ? subscriptions = yes ? type = private } namespace { ? list = children ? location = maildir:/var/virtual-maildir/%%n:INDEX=~/shared.%%n ? prefix = shared/%%n/ ? separator = / ? subscriptions = no ? type = shared } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } plugin { ? acl = vfile ? acl_shared_dict = file:/var/maildir/vol00/shared-mailboxes ? autocreate = Trash ? autocreate2 = Sent ? autocreate3 = Drafts ? autocreate4 = Spam ? autosubscribe = Trash ? autosubscribe2 = Sent ? autosubscribe3 = Drafts ? autosubscribe4 = Spam ? expire = Trash ? expire2 = Trash/* ? expire5 = Spam ? expire6 = Spam/* ? expire_dict = proxy::expire ? mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append ? mail_log_fields = uid box msgid from subject size vsize flags ? mail_plugins = " mail_log notify" ? quota = maildir:Quota ? quota_exceeded_message = Quota limit reached ? quota_rule2 = Trash:storage=+50M ? quota_warning = storage=95%% quota-warning 95 %u ? quota_warning2 = storage=90%% quota-warning 90 %u ? sieve = ~/.dovecot.sieve ? sieve_dir = ~/sieve } service auth { ? unix_listener auth-userdb { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service dict { ? unix_listener dict { ??? group = vmail ??? mode = 0600 ??? user = vmail ? } } service doveadm { ? inet_listener { ??? port = 24244 ? } } service lmtp { ? inet_listener lmtp { ??? port = 24 ? } ? unix_listener lmtp { ??? group = vmail ??? mode = 0666 ??? user = vmail ? } } service pop3-login { ? inet_listener pop3 { ??? port = 110 ? } } service quota-warning { ? executable = script /etc/dovecot/quota-warning.sh ? unix_listener quota-warning { ??? user = vmail ? } } ssl = no userdb { ? driver = prefetch } userdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } verbose_proctitle = yes protocol lmtp { ? mail_plugins = acl quota notify expire listescape sieve } protocol lda { ? mail_plugins = acl quota notify expire listescape sieve } protocol imap { ? mail_max_userip_connections = 100 ? mail_plugins = acl quota notify expire listescape imap_acl autocreate imap_quota } protocol pop3 { ? pop3_uidl_format = %v-%u } local 10.1.245.101/28/28 { ? doveadm_password = secret } From tss at iki.fi Tue Nov 8 14:26:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 14:26:10 +0200 Subject: [Dovecot] Failing to share folders when listescape is enabled (2.0.14) In-Reply-To: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> References: <1320747622.75891.YahooMailNeo@web29116.mail.ird.yahoo.com> Message-ID: <3F867C40-86D5-4572-A6A7-8B6C685245B9@iki.fi> On 8.11.2011, at 12.20, mailing lists wrote: > How I can share a folder with dots when the listescape plugin is enabled? Sorry, doesn't work in v2.0 and I don't think it's possible to fix it without major changes. Those major changes are done in v2.1 code tree though, and it works there. From p.heinlein at heinlein-support.de Tue Nov 8 15:50:25 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 14:50:25 +0100 Subject: [Dovecot] Performance-Tuning Message-ID: <201111081450.25798.p.heinlein@heinlein-support.de> Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff). I have to change my setup, maybe somebody else here have hard facts: *) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout? *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Thanks for any hints an tricks, Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From ricardo at wenn.com Tue Nov 8 15:59:01 2011 From: ricardo at wenn.com (Ricardo Branco) Date: Tue, 8 Nov 2011 13:59:01 +0000 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <28618879-1320760744-cardhu_decombobulator_blackberry.rim.net-908347421-@b13.c3.bise7.blackberry> What is the setup on the EVA, FC or iSCSI? Sent from my BlackBerry? wireless device -----Original Message----- From: Peer Heinlein Sender: dovecot-bounces at dovecot.org Date: Tue, 8 Nov 2011 14:50:25 To: Subject: [Dovecot] Performance-Tuning Hi, I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. I always wanted to make some mesurements about several influences to the performance (switch to ext4, switch to mdbox), but I never had enough time to do that. At the moment I *need* more speed, we have too much waitI/O on the system and I already used all other performance and tuning-tricks (separated cache, noatime, fsync and all that stuff). I have to change my setup, maybe somebody else here have hard facts: *) Is ext4 faster? How much faster? *) Is it faster because of the ext4 kernel-module (which can be used on ext3 to) or because of the ext4 filesystem layout? *) Is mdbox really faster? I'd like to have mdbox to have better performance in running my backup-processes. But does it bring some performance boosts to? Thanks for any hints an tricks, Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From mstevens at imt-systems.com Tue Nov 8 16:03:07 2011 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 08 Nov 2011 15:03:07 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> On 08.11.2011 14:50, Peer Heinlein wrote: > *) Is mdbox really faster? I'd like to have mdbox to have better > performance > in running my backup-processes. But does it bring some performance > boosts > to? Hi Peer, We have switched our mailbox storage format from maildir to mdbox! Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly. Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4. Best regards, Morten From javierdemiguel at us.es Tue Nov 8 16:15:39 2011 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_de_Miguel_Rodr=EDguez?=) Date: Tue, 08 Nov 2011 15:15:39 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EB9398B.1000904@us.es> We are very happy with mdbox+zlib+ext4 + iSCSI SAN (HP Lefthand in our setup) If you have CPU to spare, consider using zlib with mdbox. You are trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Mdbox has halved our backup windows (2,8 TB uncompressed mailboxes, 2 TB compressed) and backup software is happier because there are few (100.000+ files with mdbox) to backup instead of several millions (Maildir) Regards Javier > Hi, > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. > > I always wanted to make some mesurements about several influences to the > performance (switch to ext4, switch to mdbox), but I never had enough time > to do that. > > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). > > I have to change my setup, maybe somebody else here have hard facts: > > *) Is ext4 faster? How much faster? > *) Is it faster because of the ext4 kernel-module (which can be used on ext3 > to) or because of the ext4 filesystem layout? > > > *) Is mdbox really faster? I'd like to have mdbox to have better performance > in running my backup-processes. But does it bring some performance boosts > to? > > > Thanks for any hints an tricks, > > Peer > > From Ralf.Hildebrandt at charite.de Tue Nov 8 16:16:12 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 8 Nov 2011 15:16:12 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> Message-ID: <20111108141612.GE27296@charite.de> * Morten Stevens : > We have switched our mailbox storage format from maildir to mdbox! I wonder how I can incrementally change over from Maildir to mdbox? I can of course use dsync to mirror Maildir: to mdbox:, but how can I make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or vice versa). That would allow for a smooth transition... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From p.heinlein at heinlein-support.de Tue Nov 8 16:19:05 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:19:05 +0100 Subject: [Dovecot] Multiple Patitions with with mdbox Message-ID: <201111081519.05867.p.heinlein@heinlein-support.de> Having > 10 TByte mailstore filesystem-checks takes too much time. At the moment we have four different partitions, but I don't like to set symlinks or LDAP-flags to sort customers and their domains to there individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate the path automatically. How do you handle >> 10 TB mailstore? I'm very interested in the feature "alternative mailstore" with mdbox, because that makes it very easy to use at least TWO filesystems without any tricky configuration. I think I'd love to have alternative mailstores. Why does dbox doesn't look for its m.*-files in more then two directorys? Sure, looking in 4 directorys would lead to four disc operations, but maybe it could be very helpful. Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Tue Nov 8 16:22:24 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:22:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EB9398B.1000904@us.es> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EB9398B.1000904@us.es> Message-ID: <201111081522.24386.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodr?guez: Hi, > If you have CPU to spare, consider using zlib with mdbox. You are > trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). Hey. This point is great. I hadn't realized that. Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks. > compressed) and backup software is happier because there are few > (100.000+ files with mdbox) to backup instead of several millions > (Maildir) Yes, that#s the main reason why I want to switch to mbox. At the moment our roundtrip-time for the backup is > 24h... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From javierdemiguel at us.es Tue Nov 8 16:29:02 2011 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_de_Miguel_Rodr=EDguez?=) Date: Tue, 08 Nov 2011 15:29:02 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081522.24386.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EB9398B.1000904@us.es> <201111081522.24386.p.heinlein@heinlein-support.de> Message-ID: <4EB93CAE.3080308@us.es> Other important thing to consider is message expunging. With mdbox you are "delaying" the I/O associated with deleting e-mails. We have a nightly cronjob that expunge messages from mdboxes. If you have en EVA (wich one? 4.400? 6.400? ) you also can consider RAID 1+0 or SSD for indexes. Indexes are hammered in mdbox. Regards Javier > Am Dienstag, 8. November 2011, 15:15:39 schrieb Javier de Miguel Rodr?guez: > > > Hi, > >> If you have CPU to spare, consider using zlib with mdbox. You are >> trading CPU power (cheap) to get fewer IOPS (IOPS count is expensive). > Hey. This point is great. I hadn't realized that. > > Sure. zlib will save IOPS and 2x6-CPUs aren't a problem. Good point -thanks. > >> compressed) and backup software is happier because there are few >> (100.000+ files with mdbox) to backup instead of several millions >> (Maildir) > Yes, that#s the main reason why I want to switch to mbox. At the moment our > roundtrip-time for the backup is> 24h... > > > Peer > > From p.heinlein at heinlein-support.de Tue Nov 8 16:34:51 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:34:51 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <201111081534.52068.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: Hi, > I wonder how I can incrementally change over from Maildir to mdbox? If you have double diskspace: Just use "dsync mirror" in the background to prepare the change. After that it's just a short downtime to migrate incremental the last changes, or it's just a question of a short login-script. if [ -d ~/Maildir ] ; then dsync mirror voodoo-magic rm -R ~/Maildr fi > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or > vice versa). I wonder about that problem too. Even the last-last-last-quick sync would be so much IO, that I can't handle it in realtime in the morning at 9 a.m. Looks like a nightly downtime for the last incremental run. It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP attributes. In this case the whole migration process could be split up into groups. Unfortunately we have shared folders and I don't know a way to read the *remote* mailbox-format from LDAP... So having users with maildir and mdbox mixed up will break their shared folders... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From p.heinlein at heinlein-support.de Tue Nov 8 16:36:38 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 15:36:38 +0100 Subject: [Dovecot] dsync and zlib Message-ID: <201111081536.38710.p.heinlein@heinlein-support.de> Will dsync also use zlib-compression when copying mails from one side to another? Wouldn't this be a good way to compress existing mails? http://wiki2.dovecot.org/Plugins/Zlib says, there's no way to do that. But dsync should respect the zlib-Plugin... Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From jester at arcor.de Tue Nov 8 16:42:45 2011 From: jester at arcor.de (Oliver Eales) Date: Tue, 08 Nov 2011 15:42:45 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <4EB93FE5.8020903@arcor.de> On 08.11.2011 15:16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! I never tried it, but it should be possible to provide the mail_location from the user repsoitory (LDAP, SQL, whatever) So you can keep your global config, and use a script to convert one mailbox after another, and add a mail_location extra userdb field in the user repository to overwrite the global setting on a per-user-basis. Regards, Oliver From Ralf.Hildebrandt at charite.de Tue Nov 8 16:44:55 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 8 Nov 2011 15:44:55 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <20111108144455.GG27296@charite.de> * Peer Heinlein : > Am Dienstag, 8. November 2011, 15:16:12 schrieb Ralf Hildebrandt: > > Hi, > > > I wonder how I can incrementally change over from Maildir to mdbox? > > If you have double diskspace: haha :) no. I thought of a per-user migration, that way I don't need extra space. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Nov 8 16:57:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 16:57:42 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <1F29E1C1-7D8F-49F6-AF01-7B50D69F78DB@iki.fi> On 8.11.2011, at 16.34, Peer Heinlein wrote: >> I can of course use dsync to mirror Maildir: to mdbox:, but how can I >> make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or >> vice versa). > > I wonder about that problem too. Even the last-last-last-quick sync would be > so much IO, that I can't handle it in realtime in the morning at 9 a.m. > > Looks like a nightly downtime for the last incremental run. > > It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP > attributes. Easy! > In this case the whole migration process could be split up into > groups. Unfortunately we have shared folders and I don't know a way to read > the *remote* mailbox-format from LDAP... So having users with maildir and > mdbox mixed up will break their shared folders... Not so easy.. Only the home directory can be currently looked up from userdb for shared folders. There is also automatic detection of Maildir and mbox when mail_location isn't set, but no such code for mdbox. It could be added without much trouble though. But for shared folders, assuming you'd want per-user \seen flags, it would also need something like: mail_location = auto::INDEX=~/shared-indexes This "auto" doesn't exist yet either. And then there's the biggest problem: You can't have per-user \seen flags with mdbox, because you can't change the index file path without breaking mdbox. From tss at iki.fi Tue Nov 8 16:59:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 16:59:03 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108141612.GE27296@charite.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> Message-ID: <877AD4C4-F8A0-48F5-812D-5AAC6FCF13C2@iki.fi> On 8.11.2011, at 16.16, Ralf Hildebrandt wrote: > * Morten Stevens : > >> We have switched our mailbox storage format from maildir to mdbox! > > I wonder how I can incrementally change over from Maildir to mdbox? > I can of course use dsync to mirror Maildir: to mdbox:, but how can I > make dovecot look at Maildir FIRST and (if that fails) at mdbox? (or > vice versa). > > That would allow for a smooth transition... If you don't have shared folders (as explained in previous mail) and you can have per-user mail_location in the userdb, this should be pretty easy. The man page for dsync lists the steps that can be used for online migration. From tss at iki.fi Tue Nov 8 17:00:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:00:29 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> Message-ID: On 8.11.2011, at 16.03, Morten Stevens wrote: > We have switched our mailbox storage format from maildir to mdbox! > > Maildir is a disaster. (too many small files) After the migration to mdbox the performance has improved significantly. > > Conclusion: mdbox is great and much better performance than maildir! I would also recommend ext4. You don't happen to have any specific numbers/graphs that can be used to compare maildir vs. mdbox in the same hardware? I'd be interested in seeing those, such as a graph of disk iops spanning a month before/after mdbox switch. From dirk.jahnke-zumbusch at desy.de Tue Nov 8 17:03:24 2011 From: dirk.jahnke-zumbusch at desy.de (Jahnke-Zumbusch, Dirk) Date: Tue, 8 Nov 2011 16:03:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EB93FE5.8020903@arcor.de> References: <201111081450.25798.p.heinlein@heinlein-support.de><7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com><20111108141612.GE27296@charite.de> <4EB93FE5.8020903@arcor.de> Message-ID: <7C0189D0CDB51646B7814F4377D356F501536694@ADXV4.win.desy.de> Hi there, >I never tried it, but it should be possible to provide the mail_location >from the user repsoitory (LDAP, SQL, whatever) Actually this works :-) Our userdb looks similar to: account1:xyz:000:000::/account1s/home/dir::userdb_mail=maildir:/account1s/home/dir/Maildir account2:xyz:000:000::/account2s/home/dir::userdb_mail=mdbox:/ account2s/home/dir http://wiki2.dovecot.org/UserDatabase/ExtraFields Concerning Maildir backups: what about a backup-to-disc-to-tape scheme using snapshots for the "to-disc" part and something like perpetual incrementals afterwards for the "top-tape" (secondary store) ? Regards Dirk -- Dirk Jahnke-Zumbusch Deutsches Elektronen-Synchrotron DESY IT Information Fabrics Member of the Helmholtz Association D-22603 Hamburg Notkestrasse 85 / 22607 Hamburg T: +49-40-899.81760 F: +49-40-899.41760 dirk.jahnke-zumbusch at desy.de >So you can keep your global config, and use a script to convert one >mailbox after another, and add a mail_location extra userdb field in the >user repository to overwrite the global setting on a per-user-basis. > >Regards, >Oliver From tss at iki.fi Tue Nov 8 17:04:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:04:07 +0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: On 8.11.2011, at 15.50, Peer Heinlein wrote: > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). A few more ideas for Maildir if you haven't done yet: - maildir_very_dirty_syncs = yes - pop3_no_flag_updates = yes - Switching to dict file quota instead of Maildir++ quota. From tss at iki.fi Tue Nov 8 17:10:11 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:10:11 +0200 Subject: [Dovecot] dsync and zlib In-Reply-To: <201111081536.38710.p.heinlein@heinlein-support.de> References: <201111081536.38710.p.heinlein@heinlein-support.de> Message-ID: On 8.11.2011, at 16.36, Peer Heinlein wrote: > Will dsync also use zlib-compression when copying mails from one side to > another? As long as the zlib plugin is globally enabled and zlib_save setting is set. > Wouldn't this be a good way to compress existing mails? > > http://wiki2.dovecot.org/Plugins/Zlib > > says, there's no way to do that. But dsync should respect the zlib-Plugin... Well .. You could think of compressing existing mails with dsync the same as "migrating from maildir" to "migrating to maildir with zlib enabled" :) I'll add a note to the wiki page. From tss at iki.fi Tue Nov 8 17:37:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 17:37:01 +0200 Subject: [Dovecot] Multiple Patitions with with mdbox In-Reply-To: <201111081519.05867.p.heinlein@heinlein-support.de> References: <201111081519.05867.p.heinlein@heinlein-support.de> Message-ID: <608C2C35-40AD-41C7-9924-9D4F0BE232A2@iki.fi> On 8.11.2011, at 16.19, Peer Heinlein wrote: > Having > 10 TByte mailstore filesystem-checks takes too much time. > > At the moment we have four different partitions, but I don't like to set > symlinks or LDAP-flags to sort customers and their domains to there > individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate > the path automatically. Why not symlinks? You could use e.g. /mail/%16Hu/%d/%n as part of the path which would give you one hex letter when hashing the username (0..9, a..f). So then you could create symlinks like: /mail/0 -> /storage1 /mail/1 -> /storage2 ..etc.. > I'm very interested in the feature "alternative mailstore" with mdbox, > because that makes it very easy to use at least TWO filesystems without any > tricky configuration. > > I think I'd love to have alternative mailstores. Why does dbox doesn't > look for its m.*-files in more then two directorys? Sure, looking in 4 > directorys would lead to four disc operations, but maybe it could be very > helpful. The current implementation uses a simple message flag to tell when/where to move the mail, so the code would have to get much more complicated with more alt paths. From michael.abbott at apple.com Tue Nov 8 18:43:17 2011 From: michael.abbott at apple.com (Mike Abbott) Date: Tue, 08 Nov 2011 10:43:17 -0600 Subject: [Dovecot] another sizeof tweak Message-ID: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { Should that be sizeof(*old_rec)? From tss at iki.fi Tue Nov 8 18:46:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 8 Nov 2011 18:46:07 +0200 Subject: [Dovecot] another sizeof tweak In-Reply-To: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> References: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Message-ID: On 8.11.2011, at 18.43, Mike Abbott wrote: > Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: > memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { > Should that be sizeof(*old_rec)? Yep, looks like was fixed in v2.1 already. I'll fix it for v2.0 too. From rostetter at mail.utexas.edu Tue Nov 8 18:53:32 2011 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Tue, 08 Nov 2011 10:53:32 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081534.52068.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <7cfecee1f67d695d9bb47b2d108cd02b@imt-systems.com> <20111108141612.GE27296@charite.de> <201111081534.52068.p.heinlein@heinlein-support.de> Message-ID: <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> Quoting Peer Heinlein : > It would be MUCH easier if Dovecot could read maildir: or mdbox: from LDAP > attributes. In this case the whole migration process could be split up into > groups. Unfortunately we have shared folders and I don't know a way to read > the *remote* mailbox-format from LDAP... So having users with maildir and > mdbox mixed up will break their shared folders... May not work for you, but... The way I did this when I migrated was to run two dovecot instances, and have perdition software on a front-end (could be on the same machine instead of a front-end, I just happen to have a front-end machine to do it). Perdition will query ldap for the info per user/connection, and send the connection to the correct dovecot instance based on the ldap lookup. Worked for me, your milage may vary... -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From jeroen at bizworx-it.nl Tue Nov 8 20:34:30 2011 From: jeroen at bizworx-it.nl (Jeroen Grusewski) Date: Tue, 8 Nov 2011 19:34:30 +0100 Subject: [Dovecot] Dovecot alias and mailbox problem Message-ID: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> Hi All, My Postfix / Dovecot / MySQL is running fine but I have the following issue: When I create the following mailboxes test at example.com and test1 at example.com it is working fine. I can create an alias test2 at examples.com => test1 at example.com that is also working without a problem, only when I create test at example.com => test1 at example.com email is only delivered to test1 at .... instead to both test at .... and test1 at .... I believe it is properly just a setting but please help me out here ! Let me know if you need any more information before I post configs etc. Best regards, Jeroen From tom at whyscream.net Tue Nov 8 20:54:46 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Tue, 08 Nov 2011 19:54:46 +0100 Subject: [Dovecot] Dovecot alias and mailbox problem In-Reply-To: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> References: <624AD995-576E-4FD9-A72A-4FD0C613CAD9@bizworx-it.nl> Message-ID: <4EB97AF6.1020606@whyscream.net> On 08-11-11 19:34, Jeroen Grusewski wrote: > > Hi All, > > My Postfix / Dovecot / MySQL is running fine but I have the following issue: > > When I create the following mailboxes test at example.com and test1 at example.com it is working fine. > I can create an alias test2 at examples.com => test1 at example.com that is also working without a problem, > only when I create test at example.com => test1 at example.com email is only delivered to test1 at .... instead to > both test at .... and test1 at .... > > I believe it is properly just a setting but please help me out here ! > > Let me know if you need any more information before I post configs etc. > You probably want to create the alias as test at example.com => test1 at example.com,test at example.com. Depending on how you setup postfix, this should deliver to both mailboxes (and not create an infinite loop). Note that aliasing is handled within Postfix, not in Dovecot, so if you have further questions, you're probably on the wrong list. -- Regards, Tom From p.heinlein at heinlein-support.de Tue Nov 8 21:40:08 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Tue, 8 Nov 2011 20:40:08 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> References: <201111081450.25798.p.heinlein@heinlein-support.de> <201111081534.52068.p.heinlein@heinlein-support.de> <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> Message-ID: <201111082040.08183.p.heinlein@heinlein-support.de> Am Dienstag, 8. November 2011, 17:53:32 schrieb Eric Rostetter: > May not work for you, but... > > The way I did this when I migrated was to run two dovecot instances, and > have perdition software on a front-end (could be on the same machine > instead of a front-end, I just happen to have a front-end machine to do > it). You could do that with Dovecot, too. > Perdition will query ldap for the info per user/connection, and send the > connection to the correct dovecot instance based on the ldap lookup. > Worked for me, your milage may vary... The problem is: You're running in problems with shared folders. You can't read your neighbors storage-engine from ldap. It's easy to read the user's storage engine from ldap. So there's no need to use perdition for that :-) But you can't read or proxy the storage engine from somebody who shared you his folders. That's my problem :-( Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From rostetter at mail.utexas.edu Tue Nov 8 21:50:56 2011 From: rostetter at mail.utexas.edu (Eric Rostetter) Date: Tue, 08 Nov 2011 13:50:56 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111082040.08183.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <201111081534.52068.p.heinlein@heinlein-support.de> <20111108105332.7131495c1d3qw6g4@mail.ph.utexas.edu> <201111082040.08183.p.heinlein@heinlein-support.de> Message-ID: <20111108135056.312858jv1jl7bhgk@mail.ph.utexas.edu> Quoting Peer Heinlein : > The problem is: You're running in problems with shared folders. You can't > read your neighbors storage-engine from ldap. Yes, but I didn't have any shared folders, so it worked. Your milage may vary, as I said... :) -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns! From tss at iki.fi Tue Nov 8 23:09:52 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 08 Nov 2011 23:09:52 +0200 Subject: [Dovecot] another sizeof tweak In-Reply-To: References: <788F26A1-061E-4FF2-9AE5-2D20F9DC1AE8@apple.com> Message-ID: <1320786593.21919.345.camel@hurina> On Tue, 2011-11-08 at 18:46 +0200, Timo Sirainen wrote: > On 8.11.2011, at 18.43, Mike Abbott wrote: > > > Line 776 of dovecot-2.0.15/src/lib-storage/index/maildir/maildir-sync-index.c reads: > > memcmp(old_rec, &new_rec, sizeof(old_rec)) != 0) { > > Should that be sizeof(*old_rec)? > > Yep, looks like was fixed in v2.1 already. I'll fix it for v2.0 too. Actually it doesn't matter in v2.0 since this is mailbox list indexing code, which is forcibly disabled (and broken) in v2.0. From tss at iki.fi Wed Nov 9 00:35:37 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 00:35:37 +0200 Subject: [Dovecot] v2.1.beta1 released Message-ID: <1320791738.21919.350.camel@hurina> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig Here's the first beta release of Dovecot v2.1. This version has already been tested quite a lot, so I'm not expecting any major bugs. So please upgrade and see if you can find any problems. I'm optimistic about getting rc1 released this year and perhaps even v2.1.0. Since alpha2 there have been a lot of fixes, especially to imapc backend. There are probably also some small new features, but nothing huge. As a reminder, here's the largest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now From stephan at rename-it.nl Wed Nov 9 01:37:05 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 09 Nov 2011 00:37:05 +0100 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320791738.21919.350.camel@hurina> References: <1320791738.21919.350.camel@hurina> Message-ID: <4EB9BD21.6060501@rename-it.nl> On 11/8/2011 11:35 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig > > Here's the first beta release of Dovecot v2.1. This version has already > been tested quite a lot, so I'm not expecting any major bugs. So please > upgrade and see if you can find any problems. I'm optimistic about > getting rc1 released this year and perhaps even v2.1.0. > Ok, I guess it is time for a Pigeonhole release for Dovecot v2.1. I'm a bit swamped this week, but next weekend I should be able to compose one. For now you can use the dovecot-2.0-pigeonhole hg and patch it with http://hg.rename-it.nl/pigeonhole-0.2-dovecot-2.1-patches/file/tip/pigeonhole-0.2-dovecot-2.1.patch to make it compile against v2.1. Regards, Stephan. From stan at hardwarefreak.com Wed Nov 9 09:34:04 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 01:34:04 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EBA2CEC.1040701@hardwarefreak.com> On 11/8/2011 7:50 AM, Peer Heinlein wrote: > I have > 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. That's a lot of mail (likely a large user base--not given), on a filesystem not designed for such, on a decent SAN controller--LUN RAID configuration not given. > I always wanted to make some mesurements about several influences to the > performance (switch to ext4, switch to mdbox), but I never had enough time > to do that. If you're going to switch filesystems, for this size dataset and concurrent workload, you're moving in the wrong direction. > At the moment I *need* more speed, we have too much waitI/O on the system > and I already used all other performance and tuning-tricks (separated cache, > noatime, fsync and all that stuff). EXT3/4 are not designed, nor optimized, for high concurrency workloads. > I have to change my setup, maybe somebody else here have hard facts: > > *) Is ext4 faster? How much faster? Simulated maildir workload test on 2.6.35-rc5, 128 threads (No data published for newer kernels): http://btrfs.boxacle.net/repository/raid/2.6.35-rc5/2.6.35-rc5/2.6.35-rc5_Mail_server_simulation._num_threads=128.html As you can see EXT4 shows a small gain over EXT3, ~20%. If you really want high performance it's time to move to XFS, properly configured to match the underlying RAID characteristics of the LUN(s) you're mounting. You'll prefer kernel 2.6.39+, 2.6.36 at minimum, so you get the delayed logging feature (2.6.35 had delayed logging but had problems in other areas). I'll assume with a >10TB mail store that you're seeing greater than 128 concurrent user operations regularly. As you can see from the graph, XFS will give you ~50% greater ops/s than EXT4 and ~90% greater than EXT3--yes, almost double that of EXT3. As the concurrency increases, so will this performance gap, as XFS was designed from day 1 for high concurrency workloads. This is a simulated mail server benchmark. However you should see similar gains with Dovecot. The XFS delayed logging feature will dramatically reduce the number of physical IOs required for journal writes (i.e. metadata IO), as will delayed allocation, a feature of XFS since its inception in 1994. EXT4 was the first of its lineage to gain delayed allocation, some 10+ years later, after Ted T'so studied the XFS code. In short, if you want an 'enterprise caliber' production Linux filesystem tailor made for high IO concurrency, XFS is it. JFS yields similar performance, but hasn't been actively developed for 8 years or so. XFS has substantial ongoing feature and fix development. > *) Is it faster because of the ext4 kernel-module (which can be used on ext3 > to) or because of the ext4 filesystem layout? AIUI, the bulk of the EXT4 performance advantage over EXT3 is the delayed allocation logic. The new EXT4 extent based on disk layout yields little in the way of additional performance, but much in free space management, fragmentation mitigation, etc. > *) Is mdbox really faster? I'd like to have mdbox to have better performance > in running my backup-processes. But does it bring some performance boosts > to? mdbox will substantially decrease physical IOs to your storage back end due to dramatically less metadata operations compared to maildir. You've stated you currently have a storage IOPS bottleneck, so I'd have to assume mdbox will seriously increase your overall performance. Good old mbox will do so as well, but everyone shuns it for various reasons, some valid, some not so valid. If you have an appropriate LUN available (sufficient size and spindle speed/count of member disks), properly create an XFS filesystem on it (read much before creating it), and moved to mdbox atop that, I think you'll be really surprised by how much you gain from simply changing filesystems and mailbox storage formats. If you double the size of the LUN you could potentially carry twice as many users with, fewer IOPS than you're seeing now, on essentially the same hardware platform. -- Stan From ghandidrivesahumvee at rocketfish.com Wed Nov 9 09:40:14 2011 From: ghandidrivesahumvee at rocketfish.com (Dovecot-GDH) Date: Tue, 8 Nov 2011 23:40:14 -0800 Subject: [Dovecot] Multiple Patitions with with mdbox In-Reply-To: <201111081519.05867.p.heinlein@heinlein-support.de> References: <201111081519.05867.p.heinlein@heinlein-support.de> Message-ID: > How do you handle >> 10 TB mailstore? ZFS: no need to fsck. GlusterFS: "always-online". On Nov 8, 2011, at 6:19 AM, Peer Heinlein wrote: > > Having > 10 TByte mailstore filesystem-checks takes too much time. > > At the moment we have four different partitions, but I don't like to set > symlinks or LDAP-flags to sort customers and their domains to there > individual mount-point. I'd like to work with mdbox:/mail/%d/%n to calculate > the path automatically. > > How do you handle >> 10 TB mailstore? > > I'm very interested in the feature "alternative mailstore" with mdbox, > because that makes it very easy to use at least TWO filesystems without any > tricky configuration. > > I think I'd love to have alternative mailstores. Why does dbox doesn't > look for its m.*-files in more then two directorys? Sure, looking in 4 > directorys would lead to four disc operations, but maybe it could be very > helpful. > > Peer > -- > > Heinlein Professional Linux Support GmbH > Linux: Akademie - Support - Hosting > http://www.heinlein-support.de > > Tel: 030/405051-42 > Fax: 030/405051-19 > > Zwangsangaben lt. ?35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From thomas at koch.ro Wed Nov 9 11:29:37 2011 From: thomas at koch.ro (Thomas Koch) Date: Wed, 9 Nov 2011 10:29:37 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT Message-ID: <201111091029.37963.thomas@koch.ro> Hi, somebody from the kolab groupware project recently explained me that the Bus- Factor[1] of around 1 would be one of their primary reasons not to use Dovecot and stick with Cyrus. What do you think about that? Is the bus factor much higher then 1? [1] http://en.wikipedia.org/wiki/Bus_factor Somehow related: Since the free software world[2] slowly converges towards GIT as the "one and only" VCS, have you thought about a switch? It's much more likely that somebody checks out your code, looks into it and provides patches if the VCS is already installed. [2] Android, Debian, Drupal, Eclipse, Fedora, Gnome, KDE, Linux Kernel, Perl, PHP (ongoing), PostgreSQL, Qt, Ruby on Rails, X.org (freedesktop.org) Best regards, Thomas Koch, http://www.koch.ro From sandro.tosi at register.it Wed Nov 9 11:45:12 2011 From: sandro.tosi at register.it (Sandro Tosi) Date: Wed, 09 Nov 2011 10:45:12 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <4EBA4BA8.1080403@register.it> On 11/09/2011 10:29 AM, Thomas Koch wrote: > Somehow related: Since the free software world[2] slowly converges towards GIT > as the "one and only" VCS, have you thought about a switch? It's much more > likely that somebody checks out your code, looks into it and provides patches > if the VCS is already installed. > > [2] Android, Debian, Drupal, Eclipse, Fedora, Gnome, KDE, Linux Kernel, Perl, > PHP (ongoing), PostgreSQL, Qt, Ruby on Rails, X.org (freedesktop.org) I don't know where you inferred that, but for sure Debian is not converging to git; we have that VCS as many others and none is the preferred/superior. Please try to balance what you say with actual facts: backing your reasoning with partial data is misleading for others not knowing the env you're talking about. Regards, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From p.heinlein at heinlein-support.de Wed Nov 9 11:47:04 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Wed, 9 Nov 2011 10:47:04 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT Message-ID: <201111091047.04570.p.heinlein@heinlein-support.de> Am Mittwoch, 9. November 2011, 10:29:37 schrieb Thomas Koch: Hi, > somebody from the kolab groupware project recently explained me that the > Bus- Factor[1] of around 1 would be one of their primary reasons not to > use Dovecot and stick with Cyrus. The problem is: Even it Timo will be hitten by a bus, everybody could stay on the actual Dovecot-version and everything's fine. Cyrus will IMO never reach the level, that Dovecot already has. :-) Peer P.S.: Timo! Don't leave your house. It's too dangerous for you. -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030/405051-42 Fax: 030/405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From david.ocana at bsc.es Wed Nov 9 11:54:28 2011 From: david.ocana at bsc.es (David Ocana) Date: Wed, 09 Nov 2011 10:54:28 +0100 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox Message-ID: <4EBA4DD4.4080505@bsc.es> Hello, I've been trying to set up dovecot 2.0.15, everything seems to work pretty well except for the quota feature. I would like to set a quota limit only for the Inbox folder. I configured two namespaces, according to some posts from Timo Sirainen (http://dovecot.org/list/dovecot/2006-July/014530.html) & (http://www.dovecot.org/list/dovecot/2011-January/056131.html) but I can't configure the quota plugin to act the way I pretend. Here's my namespace configuration: namespace { separator = / prefix = INBOX/ location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n inbox = yes hidden = yes } namespace { separator = / prefix = inbox = no location = mbox:/mail/%d/MAILBOXES/%n:INDEX=/var/dovecot/%d/%n } Quota config: plugin { quota = dirsize:User quota # I've tried with: quota_rule = INBOX:storage=819200K quota_rule = INBOX/*:storage=819200K quota_rule = INBOX/Inbox:storage=819200K # Works with: quota_rule = *:storage=819200K quota_rule = ?:storage=819200K (Same behavior as '*') } Using '*' and '?', is the only way I get it to work, but obviously, quota is calculated summing up all the mailbox folders. I don't know what else to change, any ideas? Is the two namespace configuration correct? Thanks in advance. David -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4439 bytes Desc: S/MIME Cryptographic Signature URL: From thomas at koch.ro Wed Nov 9 12:05:42 2011 From: thomas at koch.ro (Thomas Koch) Date: Wed, 9 Nov 2011 11:05:42 +0100 Subject: [Dovecot] GIT in Debian In-Reply-To: <4EBA4BA8.1080403@register.it> References: <201111091029.37963.thomas@koch.ro> <4EBA4BA8.1080403@register.it> Message-ID: <201111091105.42746.thomas@koch.ro> Hi Sandro, I've been asking myself whether I should include Debian in the list. You're right that there are also other VCS' used in Debian. Zack's statistics say: arch 22 bzr 271 cvs 31 darcs 382 git 5230 hg 63 mtn 13 svn 4843 http://upsilon.cc/~zack/stuff/vcs-usage/ The only open question is, towards which DVCS system the 4843 subversion packages will eventually migrate. I'd bet several rounds of beer on GIT. I believe that the recent migration of 2110 packages from the perl team from svn to git is not yet fully represented in the above numbers. But we shouldn't annoy the dovecot list with Debian details any longer. Regards, Thomas Koch, http://www.koch.ro From adrian.minta at gmail.com Wed Nov 9 14:59:17 2011 From: adrian.minta at gmail.com (Adrian M) Date: Wed, 9 Nov 2011 14:59:17 +0200 Subject: [Dovecot] Quota BUG ? Message-ID: Hello list, I believe I found a bug with quota. The dovecot quota replies with incorrect values when the mailbox is above 2GB. du -ks returns 2005764bytes dovecot IMAP on the same user/directory: 1 getquotaroot inbox * QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1096 2000000)* QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1096 2000000) 1 OK Getquotaroot completed. Is there any way to fix this ? --- linux system: dovecot 1.2.15-7 linux 2.6.32-5-amd64 dovecot.conf: dict { quotadict= mysql:/etc/dovecot/dovecot-dict-quota.conf } plugin { quota = dict:user::proxy::quotadict } dovecot-dict-quota.conf : user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' From maria_arrea at gmx.com Wed Nov 9 15:19:54 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Wed, 09 Nov 2011 14:19:54 +0100 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs Message-ID: <20111109131954.26480@gmx.com> Hello We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy cheap sata-based iscsi storage / sata-NFS. We want to evaluate how much space would be needed in alternate storage BEFORE buying it. I have been reading about doveadm altmove. We want to move ALL messages older than a year to alternate storage, what doveadm-script-fu could I do to identify how much space would go from a user to alternate storage? Regards Maria From tss at iki.fi Wed Nov 9 15:53:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 15:53:08 +0200 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <1320846789.21919.361.camel@hurina> On Wed, 2011-11-09 at 10:29 +0100, Thomas Koch wrote: > somebody from the kolab groupware project recently explained me that the Bus- > Factor[1] of around 1 would be one of their primary reasons not to use Dovecot > and stick with Cyrus. > > What do you think about that? Is the bus factor much higher then 1? I've created a company for Dovecot support, and if all goes well we should have at least one other coder in not too distant future (anyone want a job? :) Also besides me there's already at least Stephan Bosch who has written Dovecot's Sieve/ManageSieve implementations. I don't know if he'd take care of the whole Dovecot if I happened to die right now, but at least he knows the code pretty well. There are also a few big companies that have some people who have done some Dovecot coding. Also the Bus-Factor of Cyrus doesn't seem to be much higher than 1 to me. AFAIK there's only a single person currently developing it actively (plus I guess a few more not-very-active developers from CMU). > Somehow related: Since the free software world[2] slowly converges towards GIT > as the "one and only" VCS, have you thought about a switch? It's much more > likely that somebody checks out your code, looks into it and provides patches > if the VCS is already installed. I'm not as much against git anymore as I was when I switched to hg, but I don't see much benefits in switching to git either. I highly doubt I'd get even a single patch more if I used git instead of hg. The biggest problem with lack of patches is that few people are interested in coding a mail server. You can see the same with all open source IMAP/SMTP servers (and probably commercial ones too). Nearly always there's only a single guy who has written almost all of it. From achekalin at lazurit.com Wed Nov 9 15:57:11 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 09 Nov 2011 16:57:11 +0300 Subject: [Dovecot] Please advise on very fast search Message-ID: <4EBA86B7.2080201@lazurit.com> Hello, I try to create some kind of mail backup system. What I need is system that will store mail for the whole domain, and allow me to restore messages from/to specified email at that domain. The scheme is pretty simple: on our main mail server the SMTP server itself has a rule to send a copy of every message to 'backup at backupserver.host', and the backupserver.host domain is placed nearby on second server. The SMTP on second server do simple 'catchall' redirect of all messages to the single box. There is also a Dovecot that takes care for remote IMAP access to that box. And, finally, I've create some scripts to sort all messages in INBOX to folders named after message's date. So I have a lot of mailboxes inside the catchall box: INBOX 2011.11.03 2011.11.04 2011.11.05 2011.11.06 ...etc... and each folder holds messages for that day. Simply, and works perfectly. The problem is that when my archive become big (several years), it appears to be painful to find specified message(s). When someone suddenly needs to find his/her old message, it is mostly guesses like 'I think the message was between june and july of 2009, or maybe month or two before that', so I need to search all mailboxes (with 1000's messages in each). And it takes really long time. I tried to play with Dovecot indexes, but it won't help too much. The bad part is that I need to search for all emails in each message headers, not only for "From" or "To", since some messages are sent to maillists soe "To" = list address, not person's personal email. Then I tried to index messages on my own, storing info on emails into MySQL database ('email' -> 'mailbox', 'message filename'), but soon I find out that message files can be renamed by Dovecot. Could you please advice me how to speed up message search? Sorry for such a long question, hope you can help! Yours, Alexander Chekalin From tss at iki.fi Wed Nov 9 15:58:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 15:58:42 +0200 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs In-Reply-To: <20111109131954.26480@gmx.com> References: <20111109131954.26480@gmx.com> Message-ID: <1320847122.21919.365.camel@hurina> On Wed, 2011-11-09 at 14:19 +0100, Maria Arrea wrote: > We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy > cheap sata-based iscsi storage / sata-NFS. We want to evaluate how > much space would be needed in alternate storage BEFORE buying it. I > have been reading about doveadm altmove. We want to move ALL messages > older than a year to alternate storage, what doveadm-script-fu could I > do to identify how much space would go from a user to alternate > storage? doveadm -A -f tab fetch size.physical before 1y would print all the message sizes. You could then sum them up with some scripting to produce the total. Unfortunately they are the uncompressed sizes, but maybe simply dividing by 2 would give approximate size? :) (You could get the compressed sizes from dovecot.map.index files with doveadm dump and mapping their contents to the older than 1 year message UIDs, but this gets difficult.) From robert at schetterer.org Wed Nov 9 16:14:21 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 09 Nov 2011 15:14:21 +0100 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBA86B7.2080201@lazurit.com> References: <4EBA86B7.2080201@lazurit.com> Message-ID: <4EBA8ABD.8010804@schetterer.org> Am 09.11.2011 14:57, schrieb Alexander Chekalin: > Hello, > > I try to create some kind of mail backup system. What I need is system > that will store mail for the whole domain, and allow me to restore > messages from/to specified email at that domain. > > The scheme is pretty simple: on our main mail server the SMTP server > itself has a rule to send a copy of every message to > 'backup at backupserver.host', and the backupserver.host domain is placed > nearby on second server. > > The SMTP on second server do simple 'catchall' redirect of all messages > to the single box. There is also a Dovecot that takes care for remote > IMAP access to that box. And, finally, I've create some scripts to sort > all messages in INBOX to folders named after message's date. > > So I have a lot of mailboxes inside the catchall box: > INBOX > 2011.11.03 > 2011.11.04 > 2011.11.05 > 2011.11.06 > ...etc... > > and each folder holds messages for that day. Simply, and works perfectly. > > The problem is that when my archive become big (several years), it > appears to be painful to find specified message(s). When someone > suddenly needs to find his/her old message, it is mostly guesses like 'I > think the message was between june and july of 2009, or maybe month or > two before that', so I need to search all mailboxes (with 1000's > messages in each). And it takes really long time. > > > I tried to play with Dovecot indexes, but it won't help too much. The > bad part is that I need to search for all emails in each message > headers, not only for "From" or "To", since some messages are sent to > maillists soe "To" = list address, not person's personal email. > > Then I tried to index messages on my own, storing info on emails into > MySQL database ('email' -> 'mailbox', 'message filename'), but soon I > find out that message files can be renamed by Dovecot. > > Could you please advice me how to speed up message search? > > > Sorry for such a long question, hope you can help! > > Yours, > Alexander Chekalin > guess youre searching over imap ? perhaps compression will help for speed up, and many other speed related stuff, or you need some other idea of indexing at last if its maildir how fast is "grep" etc...and so on some ideas here http://wiki.dovecot.org/HowTo/ReadOnlyArchive etc anyway , i think you really need another kind of archive solution in Germany there is a law that you need to archive some kind of business mails up to 10 years for finance and other review, so there are a lot of "you can by" solutions now, these have solved the problems you discovered ( indexing etc ) i was shown i.e http://www.bytstormail.de which looked fine to me or perhaps you might have a look http://www.archiveopteryx.org/ here too -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From maria_arrea at gmx.com Wed Nov 9 16:24:14 2011 From: maria_arrea at gmx.com (Maria Arrea) Date: Wed, 09 Nov 2011 15:24:14 +0100 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs Message-ID: <20111109142414.26470@gmx.com> Timo, maybe you have mispelled the command? > We are using dovecot 2.0.15+mdbox+zlib and we are evaluating to buy > cheap sata-based iscsi storage / sata-NFS. We want to evaluate how > much space would be needed in alternate storage BEFORE buying it. I > have been reading about doveadm altmove. We want to move ALL messages > older than a year to alternate storage, what doveadm-script-fu could I > do to identify how much space would go from a user to alternate > storage? doveadm -A -f tab fetch size.physical before 1y would print all the message sizes. You could then sum them up with some scripting to produce the total. Unfortunately they are the uncompressed sizes, but maybe simply dividing by 2 would give approximate size? :) (You could get the compressed sizes from dovecot.map.index files with doveadm dump and mapping their contents to the older than 1 year message UIDs, but this gets difficult.) From tss at iki.fi Wed Nov 9 17:15:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 17:15:23 +0200 Subject: [Dovecot] Doveadm scrip-fu to determine alternate storage needs In-Reply-To: <20111109142414.26470@gmx.com> References: <20111109142414.26470@gmx.com> Message-ID: <1320851723.21919.367.camel@hurina> On Wed, 2011-11-09 at 15:24 +0100, Maria Arrea wrote: > Timo, maybe you have mispelled the command? Minor changes, I tested that this actually works: doveadm -f tab fetch -A size.physical before 365d The -A parameter requires that you use a userdb that supports iteration. With SQL you need to provide the iteration_query. From tss at iki.fi Wed Nov 9 17:17:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 17:17:31 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBA86B7.2080201@lazurit.com> References: <4EBA86B7.2080201@lazurit.com> Message-ID: <1320851851.21919.369.camel@hurina> On Wed, 2011-11-09 at 16:57 +0300, Alexander Chekalin wrote: > The problem is that when my archive become big (several years), it > appears to be painful to find specified message(s). When someone > suddenly needs to find his/her old message, it is mostly guesses like 'I > think the message was between june and july of 2009, or maybe month or > two before that', so I need to search all mailboxes (with 1000's > messages in each). And it takes really long time. > > > I tried to play with Dovecot indexes, but it won't help too much. They'll help with the dates. > The > bad part is that I need to search for all emails in each message > headers, not only for "From" or "To", since some messages are sent to > maillists soe "To" = list address, not person's personal email. Headers only, not message body? Anyway, some of the full text search backends would support searching from both. I'd recommend using either Solr or with Dovecot v2.1 you can also use Lucene: http://wiki2.dovecot.org/Plugins/FTS From odhiambo at gmail.com Wed Nov 9 17:34:45 2011 From: odhiambo at gmail.com (Odhiambo Washington) Date: Wed, 9 Nov 2011 18:34:45 +0300 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320791738.21919.350.camel@hurina> References: <1320791738.21919.350.camel@hurina> Message-ID: On Wed, Nov 9, 2011 at 01:35, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz > http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig > > Here's the first beta release of Dovecot v2.1. This version has already > been tested quite a lot, so I'm not expecting any major bugs. So please > upgrade and see if you can find any problems. I'm optimistic about > getting rc1 released this year and perhaps even v2.1.0. > > Since alpha2 there have been a lot of fixes, especially to imapc > backend. There are probably also some small new features, but nothing > huge. > > As a reminder, here's the largest changes since v2.0: > > * Plugins now use UTF-8 mailbox names rather than mUTF-7: > acl, autocreate, expire, trash, virtual > * auth_username_format default changed to %Lu. If you really want > case sensitive usernames, set it back to empty. > * Solr full text search backend changed to use mailbox GUIDs > instead of > mailbox names, requiring reindexing everything. solr_old backend > can > be used with old indexes to avoid reindexing, but it doesn't > support > some newer features. > > + imapc (= IMAP client) storage allows using a remote IMAP server to > be used as storage. This allows using Dovecot as a smart (caching) > proxy or using dsync to do migration from remote IMAP server. > + Mailbox indexing via queuing indexer service (required for Lucene) > + Lucene full text search (FTS) backend rewritten with support for > different languages > + FTS finally supports "OR" search operation > + FTS supports indexing attachments via external programs > + IMAP FUZZY extension, supported by Lucene and Solr FTS backends > + Mailbox list indexes > + Statistics tracking via stats service. Exported via doveadm stats. > + Autocreate plugin creates/subscribes mailboxes physically only > when > the mailbox is opened for the first time. Mailbox listing shows > the > autocreated mailboxes even if they don't physically exist. > + Password and user databases now support default_fields and > override_fields settings to specify template defaults/overrides. > - listescape plugin works perfectly now > > I have been running the alpha for some time now. I have today tried compiling this -beta, but I saw so many warning about linking libraries, which I ignored, but it did fail with: Making all in wiki make: don't know how to make 2b. Stop *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1/doc. *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1. *** Error code 1 Stop in /usr/home/wash/Tools/Dovecot/dovecot-2.1.beta1. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From achekalin at lazurit.com Wed Nov 9 18:16:39 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 09 Nov 2011 19:16:39 +0300 Subject: [Dovecot] Please advise on very fast search Message-ID: <4EBAA767.6060609@lazurit.com> Thanks, Robert, will take a look at. What I'm afraid for is how database storage should be planned (storage, CPU, RAM, scaling when will be over-filled). When dealing with files (I'm using maildir), it is much easy to understand and to fix just about everything. Adding database involves tune it up too, and I'll have more points of 'tune it a bit' In fact work with Dovecot is pretty nice, but I think I can tune it to work faster. I now run it on FreeBSD (on UFS2), maybe I should change OS + FS, but need to test (really hope ZFS disks on SAS drives will help; still find no benchmarks on such a setup). Will also try to use full text search, but afraid of index size (and I need no search on body, just on headers). Anyway thank your for pointing me in right directions! Yours, Alexander From tss at iki.fi Wed Nov 9 18:37:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 18:37:14 +0200 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: References: <1320791738.21919.350.camel@hurina> Message-ID: <1320856634.21919.371.camel@hurina> On Wed, 2011-11-09 at 18:34 +0300, Odhiambo Washington wrote: > I have today tried compiling this -beta, but I saw so many warning about > linking libraries, which I ignored, Probably plugin warnings which you can safely ignore. > Making all in wiki > make: don't know how to make 2b. Stop Looks like there's some weird file went into wiki docs, which some makes don't like. You could just find the line containing "2b" from doc/wiki/Makefile and delete it. From tss at iki.fi Wed Nov 9 18:40:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 18:40:04 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBAA767.6060609@lazurit.com> References: <4EBAA767.6060609@lazurit.com> Message-ID: <1320856804.21919.372.camel@hurina> On Wed, 2011-11-09 at 19:16 +0300, Alexander Chekalin wrote: > Will also try to use full text search, > but afraid of index size (and I need no search on body, just on headers). It wouldn't be difficult to patch Dovecot to skip indexing message bodies. Of course then you'd need to remember to keep applying the patch when updating. From tss at iki.fi Wed Nov 9 19:56:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 19:56:18 +0200 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <4EBA4DD4.4080505@bsc.es> References: <4EBA4DD4.4080505@bsc.es> Message-ID: <1320861378.21919.376.camel@hurina> On Wed, 2011-11-09 at 10:54 +0100, David Ocana wrote: > I've been trying to set up dovecot 2.0.15, everything seems to work > pretty well except for the quota feature. I would like to set a quota > limit only for the Inbox folder. I configured two namespaces, > according to some posts from Timo Sirainen > > namespace { > separator = / > prefix = INBOX/ > location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n > inbox = yes > hidden = yes > } > > plugin { > quota = dirsize:User quota quota = dirsize:User quota:ns=INBOX/ This limits the quota only to mailboxes in INBOX/ namespace. > # I've tried with: > quota_rule = INBOX:storage=819200K > quota_rule = INBOX/*:storage=819200K > quota_rule = INBOX/Inbox:storage=819200K Quota rules don't work in this way. There are no per-mailbox quotas really, at least in the way you're thinking about. From tss at iki.fi Wed Nov 9 19:59:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 19:59:42 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: References: Message-ID: <1320861585.21919.378.camel@hurina> On Wed, 2011-11-09 at 14:59 +0200, Adrian M wrote: > Hello list, > I believe I found a bug with quota. > The dovecot quota replies with incorrect values when the mailbox is above 2GB. > > du -ks returns 2005764bytes > > dovecot IMAP on the same user/directory: > 1 getquotaroot inbox > * QUOTAROOT "INBOX" "user" > * QUOTA "user" (STORAGE 1096 2000000)* QUOTAROOT "INBOX" "user" > * QUOTA "user" (STORAGE 1096 2000000) > 1 OK Getquotaroot completed. So it wraps. > dovecot.conf: > dict { > quotadict= mysql:/etc/dovecot/dovecot-dict-quota.conf > } > plugin { > quota = dict:user::proxy::quotadict > } > > dovecot-dict-quota.conf : > user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as > mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS > quota_rule FROM mailbox WHERE username = '%u' AND active = '1' This is dovecot-sql.conf, not dovecot-dict-quota.conf (or alternatively you've named them in a "wrong" way). Anyway, here's a guess: the quota is stored in mysql in "integer", which has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also use "integer". Replace that with "bigint" and recalculate quota and it should work. From adrian.minta at gmail.com Wed Nov 9 20:35:42 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 20:35:42 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320861585.21919.378.camel@hurina> References: <1320861585.21919.378.camel@hurina> Message-ID: <4EBAC7FE.5060207@gmail.com> On 11/09/11 19:59, Timo Sirainen wrote: > Anyway, here's a guess: the quota is stored in mysql in "integer", which > has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also > use "integer". Replace that with "bigint" and recalculate quota and it > should work. > Actually the mysql part is OK: quota | bigint(20) As you can see the value readed from mysql ( 2000000bytes) is also OK: * QUOTA "user" (STORAGE 1096 2000000) The problem is the 1096 number calculated internally by dovecot. By tweaking the SQL querry I coud put any value instead of 2000000, but I was unable to change 1096 part. -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From tss at iki.fi Wed Nov 9 20:41:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 20:41:23 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <4EBAC7FE.5060207@gmail.com> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> Message-ID: <1320864083.21919.379.camel@hurina> On Wed, 2011-11-09 at 20:35 +0200, Adrian Minta wrote: > On 11/09/11 19:59, Timo Sirainen wrote: > > Anyway, here's a guess: the quota is stored in mysql in "integer", which > > has a maximum value of 2 GB. Actually looks like Dovecot wiki pages also > > use "integer". Replace that with "bigint" and recalculate quota and it > > should work. > > > Actually the mysql part is OK: > quota | bigint(20) > > As you can see the value readed from mysql ( 2000000bytes) is also OK: 2000000 kilobytes is the quota limit. > * QUOTA "user" (STORAGE 1096 2000000) > > The problem is the 1096 number calculated internally by dovecot. By > tweaking the SQL querry I coud put any value instead of 2000000, but I > was unable to change 1096 part. Is the quota in the database 1096 or >2GB? i.e. is the problem with reading it, or updating it? In general the quota code uses 64bit integers everywhere so this shouldn't be happening. From adrian.minta at gmail.com Wed Nov 9 20:44:31 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 20:44:31 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320864083.21919.379.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> Message-ID: <4EBACA0F.1010200@gmail.com> > Is the quota in the database 1096 or>2GB? i.e. is the problem with > reading it, or updating it? In general the quota code uses 64bit > integers everywhere so this shouldn't be happening. > > In the database quota field is 2048000000. -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From tss at iki.fi Wed Nov 9 20:46:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 09 Nov 2011 20:46:02 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <4EBACA0F.1010200@gmail.com> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> Message-ID: <1320864362.21919.380.camel@hurina> On Wed, 2011-11-09 at 20:44 +0200, Adrian Minta wrote: > > Is the quota in the database 1096 or>2GB? i.e. is the problem with > > reading it, or updating it? In general the quota code uses 64bit > > integers everywhere so this shouldn't be happening. > > > > > In the database quota field is 2048000000. In the dict quota value? That looks more like the quota limit, not the current quota usage. From toddr at cpanel.net Wed Nov 9 20:57:39 2011 From: toddr at cpanel.net (Todd Rinaldo) Date: Wed, 9 Nov 2011 12:57:39 -0600 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <1320846789.21919.361.camel@hurina> References: <201111091029.37963.thomas@koch.ro> <1320846789.21919.361.camel@hurina> Message-ID: <629183B4-622D-4058-B706-D5FFF91FC366@cpanel.net> On Nov 9, 2011, at 7:53 AM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 10:29 +0100, Thomas Koch wrote: > >> Somehow related: Since the free software world[2] slowly converges towards GIT >> as the "one and only" VCS, have you thought about a switch? It's much more >> likely that somebody checks out your code, looks into it and provides patches >> if the VCS is already installed. > > I'm not as much against git anymore as I was when I switched to hg, but > I don't see much benefits in switching to git either. I highly doubt I'd > get even a single patch more if I used git instead of hg. > Hg / Git. Either seems fine to me. My big concern would be that the source history is in multiple accessible places around the world. This is the biggest limitation of SVN to my mind. Could I suggest a google code clone as an additional VCS backup location? From adrian.minta at gmail.com Wed Nov 9 21:01:43 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Wed, 09 Nov 2011 21:01:43 +0200 Subject: [Dovecot] Quota BUG ? In-Reply-To: <1320864362.21919.380.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> <1320864362.21919.380.camel@hurina> Message-ID: <4EBACE17.4070508@gmail.com> On 11/09/11 20:46, Timo Sirainen wrote: > On Wed, 2011-11-09 at 20:44 +0200, Adrian Minta wrote: >>> Is the quota in the database 1096 or>2GB? i.e. is the problem with >>> reading it, or updating it? In general the quota code uses 64bit >>> integers everywhere so this shouldn't be happening. >>> >>> >> In the database quota field is 2048000000. > In the dict quota value? That looks more like the quota limit, not the > current quota usage. In the mailbox table. The 2000000 value commes from: user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', CAST(quota AS CHAR)) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' This value is easy to change by modifying the above mysql query, but the 1096 value I don't know how to change. Maybe to "instruct" dovecot to work internal with Kilobytes ? For instance by altering the query like this: user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', quota, 'K') AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' ... I get ... * QUOTA "user" (STORAGE 1096 2048000000) -- Best regards, Adrian Minta MA3173-RIPE, www.minta.ro From mrvjtod at gmail.com Wed Nov 9 21:04:12 2011 From: mrvjtod at gmail.com (Chris Young) Date: Wed, 9 Nov 2011 14:04:12 -0500 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: References: Message-ID: I am familiar with this article http://wiki.dovecot.org/LDA#multipleuids I want to disable setuid-root, but I don't know the default group and permissions. sudo chgrp root /usr/libexec/dovecot/dovecot-lda sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda but my delivery is still failing Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) Thoughts? Thanks Chris From e-frog at gmx.de Wed Nov 9 21:08:48 2011 From: e-frog at gmx.de (e-frog) Date: Wed, 09 Nov 2011 20:08:48 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <4EB6AFE3.3020001@gmx.de> References: <4EB6AFE3.3020001@gmx.de> Message-ID: <4EBACFC0.6060703@gmx.de> On 06.11.2011 17:03, wrote e-frog: > 1 lsub "" "virtual/*" > * LSUB () "/" "virtual" > * LSUB () "/" "virtual/Lists" > * LSUB () "/" "virtual/unread" > 1 OK Lsub completed. Just tested this again with 2.1beta1 with the same result. From apm at one.com Wed Nov 9 21:25:57 2011 From: apm at one.com (Peter Mogensen) Date: Wed, 09 Nov 2011 20:25:57 +0100 Subject: [Dovecot] Corrupted transaction log file In-Reply-To: <1320441989.21919.212.camel@hurina> References: <4EB43BB8.6090803@one.com> <1320441989.21919.212.camel@hurina> Message-ID: <4EBAD3C5.7030006@one.com> On 2011-11-04 22:26, Timo Sirainen wrote: >> Nov 4 15:10:42 mail dovecot: imap (test at aaaone.net): Error: Corrupted >> transaction log file /mail/3340444/.TestMails/dovecot.index.log seq 2: >> indexid changed 1320419300 -> 1320419441 (sync_offset=0) > > Session A had TestMails open and created with index file whose ID was > 1320419300 (that's also UNIX timestamp of its creation time, Fri Nov 4 > 17:08:20 EET 2011). > > Session B came and recreated the index files 141 seconds later with ID > 1320419441. Either it didn't see A's original index files for some > reason or it simply decided to recreate them for some reason. Either way > this shouldn't have happened. Turns out this is expected to confuse Session A. The client in question sometimes start the session (B) with this command sequence: DELETE folder CREATE folder APPEND... Any Session A having opened "folder" of course would be surprised that there's a new index file (makes we wish for an IMAP5 where folderID!=displayname) This can be reproduced by hand speaking IMAP with two telnets. Only question left is, why does Dovecot end the log sequence by saying: "Disconnected: IMAP session state is inconsistent, please relogin." ... when it is capable of detecting this and returning "BYE folder deleted under us" and logging the same. /Peter From rpalmarin at yahoo.com Wed Nov 9 18:57:44 2011 From: rpalmarin at yahoo.com (rpalmarin) Date: Wed, 9 Nov 2011 16:57:44 +0000 (UTC) Subject: [Dovecot] LDAP expired password References: <116877.53915.qm@web29503.mail.ird.yahoo.com> <57o6m4h3miv8@mids.svenhartge.de> <4D95A998.9080701@noa.gr> <97o72ha3miv8@mids.svenhartge.de> Message-ID: Sven Hartge svenhartge.de> writes: > > Nikolaos Milas noa.gr> wrote: > > On 1/4/2011 11:09 ??, Sven Hartge wrote: > > >> Have a look at the ppolicy slapd.overlay. This will solve your > >> problem. hello all Sorry for the delay in the response I checked the ppolicy overlay but without success. This overlay does not have a single "password expired" attribute to put in the user_filter. currently my dovecot-ldap.conf contain: user_filter (&(|( = uid =% u) (employeeNumber =% u)) (objectClass = inetOrgPerson) (accountStatus = active)) > At my university we introduced our own attribute gifb-status which > contains a "1" if an account is valid, a "0" if it is not (and several > others for different purposes) and our ldap-filters all contain > something like "(&(ou=foobar)(gifb-status=1))". is possible that the only way to do this is to manage a new attribute? how can understand all the people that have configured the mail client to authenticate with imap-dovecot that their passoword has expired? thanks / roby From user+dovecot at localhost.localdomain.org Wed Nov 9 23:41:45 2011 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Wed, 09 Nov 2011 22:41:45 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT In-Reply-To: <201111091029.37963.thomas@koch.ro> References: <201111091029.37963.thomas@koch.ro> Message-ID: <4EBAF399.7090706@localhost.localdomain.org> On 11/09/2011 10:29 AM Thomas Koch wrote: > Hi, > > somebody from the kolab groupware project recently explained me that the Bus- > Factor[1] of around 1 would be one of their primary reasons not to use Dovecot > and stick with Cyrus. And what is their primary reason for the lame excuse? Too many cooks spoil the broth. just my 2? Regards, Pascal -- The trapper recommends today: fabaceae.1131322 at localdomain.org From pch0317 at gmail.com Thu Nov 10 00:33:59 2011 From: pch0317 at gmail.com (pch0317) Date: Wed, 09 Nov 2011 22:33:59 +0000 Subject: [Dovecot] Can't delete sub-folder Message-ID: <4EBAFFD7.8020201@gmail.com> Hi list, I have problem with deleting sub-folder. I use thunderbird 8.0 as a client and dovecot 2.0.9 with imap protocol as a server. I use dovecot with tb-extra-mailbox-sep parameter. I use mbox mailboxes and thunderbird with *not checked* option 'server supports folders that contain sub-folders and messages' so when I create sub-folder I can choose between 'folders only' and 'messages only'. When I try to delete 'parent' folder which contain 'messages only' folder, error message appear: [ALREADYEXISTS] Mailbox has children, delete them first. Ok, so I delete 'message only' folder first and try to delete parent folder, but the same error appear: [ALREADYEXISTS] Mailbox has children, delete them first. When I list mailboxes from console I see inside 'parent' folder ".imap" folder. When I delete .imap folder from console and then delete parent folder from thunderbird, parent folder is deleted correctly. Which application create this ".imap" hidden folder and how solve this problem. Thanks From stan at hardwarefreak.com Thu Nov 10 02:42:33 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 18:42:33 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBAA767.6060609@lazurit.com> References: <4EBAA767.6060609@lazurit.com> Message-ID: <4EBB1DF9.4040208@hardwarefreak.com> On 11/9/2011 10:16 AM, Alexander Chekalin wrote: > Thanks, Robert, > > will take a look at. > > What I'm afraid for is how database storage should be planned (storage, > CPU, RAM, scaling when will be over-filled). When dealing with files > (I'm using maildir) Bingo. ^^^ Maildir is very likely a hug factor in your current slow search time. With a maildir search, every mail file must be opened and searched. How many total mail files are opened for each of your searches? Thousands? Tens of thousands? Maildir causes a massive disk IO bottleneck when searching so many files. Run iostat the next time you do one of these searches, and look at the %iowait value. It will likely be very high. If it is, this confirms maildir is a big part of the problem. mbox, and mdbox, would be many many times faster than maildir WRT searching as the total number of files is lower by orders of magnitude. Switching from maildir to mbox/mdbox shifts the workload burden from the disk subsystem to the processor/memory. And I'm sure as with everyone else on the planet today, you have massive spare CPU cycles, but extremely limited spindle throughput. And as Timo suggested, using one of the indexing search plugins would be much faster yet, as long as you keep the indexes updated. -- Stan From stan at hardwarefreak.com Thu Nov 10 02:50:07 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 09 Nov 2011 18:50:07 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <1320856804.21919.372.camel@hurina> References: <4EBAA767.6060609@lazurit.com> <1320856804.21919.372.camel@hurina> Message-ID: <4EBB1FBF.9070401@hardwarefreak.com> On 11/9/2011 10:40 AM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 19:16 +0300, Alexander Chekalin wrote: >> Will also try to use full text search, >> but afraid of index size (and I need no search on body, just on headers). > > It wouldn't be difficult to patch Dovecot to skip indexing message > bodies. Of course then you'd need to remember to keep applying the patch > when updating. Also keep in mind that, in general, many/most message headers today are often as large, or larger than, the actual message body, especially for list mail. Just take a look at messages from this for evidence. Thus, I'd think that going out of your way to avoid indexing message bodies wouldn't be worth the effort/headaches involved. -- Stan From scott_the_musician at yahoo.com.au Thu Nov 10 06:09:02 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Wed, 9 Nov 2011 20:09:02 -0800 (PST) Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> Message-ID: <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> ----- Forwarded Message ----- From: Scott Lewis To: "dovecot at dovecot.org" Sent: Thursday, 3 November 2011 4:31 PM Subject: patching dovecot for sieve/managesieve support, centos 5.6? Hi all, I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. Here's what happens: [root at mail ~]# whereis dovecot dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot ... checking whether to build static libraries... yes dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH to give path to compiled Dovecot sources or to a directory with the installed dovecot-config file. configure: error: dovecot-config not found -- I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: Avelsieve plugin details: backend = ManageSieve ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? thanks in advance! From achekalin at lazurit.com Thu Nov 10 06:37:05 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Thu, 10 Nov 2011 07:37:05 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB1DF9.4040208@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: Oh, that's the point to consider. But I must confess I'm in love with Maildir for maybe 10 years for that simple fact I can do anything with each and every single message even on disk (=much faster than via IMAP). If I would deal with mbox directly I'd need to parse huge files, brrrr. Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? 10.11.2011, ? 3:42, Stan Hoeppner ???????(?): > On 11/9/2011 10:16 AM, Alexander Chekalin wrote: >> Thanks, Robert, >> >> will take a look at. >> >> What I'm afraid for is how database storage should be planned (storage, >> CPU, RAM, scaling when will be over-filled). When dealing with files >> (I'm using maildir) > > Bingo. ^^^ > > Maildir is very likely a hug factor in your current slow search time. > With a maildir search, every mail file must be opened and searched. How > many total mail files are opened for each of your searches? Thousands? > Tens of thousands? Maildir causes a massive disk IO bottleneck when > searching so many files. Run iostat the next time you do one of these > searches, and look at the %iowait value. It will likely be very high. > If it is, this confirms maildir is a big part of the problem. > > mbox, and mdbox, would be many many times faster than maildir WRT > searching as the total number of files is lower by orders of magnitude. > Switching from maildir to mbox/mdbox shifts the workload burden from > the disk subsystem to the processor/memory. And I'm sure as with > everyone else on the planet today, you have massive spare CPU cycles, > but extremely limited spindle throughput. > > And as Timo suggested, using one of the indexing search plugins would be > much faster yet, as long as you keep the indexes updated. > > -- > Stan From achekalin at lazurit.com Thu Nov 10 07:35:11 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Thu, 10 Nov 2011 08:35:11 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB1DF9.4040208@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EBB628F.8060705@lazurit.com> Hello, Stan, in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway). You see, what I need is not only find message from|to someone on specified date, I also sometime need to restore that message back to user's original box. As far our mailserver and backup-mailserver are different machines, it is a bit tricky to copy messages between it fast enough. Say, if I need to find and restore all mails from user at domain.com within 2009 year, and search yields in some 1000's of messages, then use IMAP to copy it over to another server takes some time - and if you consider both search time and restore/copy time the whole process may take "ages". With maildir I can rsync/scp needed files to another host and that's fast way - that's why I stick with maildir. FTS in my case can help (I can search for user at domain.com, for example), but it also return messages that contains such a string in message body (and that takes index space, too), so I'll need to filter it later, but surely it'll be faster than checking every message in the archive. Yours, Alexander > Maildir is very likely a hug factor in your current slow search time. > With a maildir search, every mail file must be opened and searched. How > many total mail files are opened for each of your searches? Thousands? > Tens of thousands? Maildir causes a massive disk IO bottleneck when > searching so many files. Run iostat the next time you do one of these > searches, and look at the %iowait value. It will likely be very high. > If it is, this confirms maildir is a big part of the problem. > > mbox, and mdbox, would be many many times faster than maildir WRT > searching as the total number of files is lower by orders of magnitude. > Switching from maildir to mbox/mdbox shifts the workload burden from > the disk subsystem to the processor/memory. And I'm sure as with > everyone else on the planet today, you have massive spare CPU cycles, > but extremely limited spindle throughput. > > And as Timo suggested, using one of the indexing search plugins would be > much faster yet, as long as you keep the indexes updated. > -- ? ?????????, ????????? ??????? ??????? ??????????? +7 909 799 2549 achekalin at lazurit.com From research at cruzio.com Thu Nov 10 10:30:31 2011 From: research at cruzio.com (Mark Hanford) Date: Thu, 10 Nov 2011 00:30:31 -0800 Subject: [Dovecot] indexes permissions problem Message-ID: <4EBB8BA7.9070603@cruzio.com> Hey all, I did a search but didn't find the answer to my problem, so here goes. I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box with user home directories NFS mounted), my index files are setup to be in /u/indexes/ On the Dovecot 1.1.1 installation, the perms on the indexes directory is 777 with root:mail ownership. The same thing on the Dovecot 2 / Centos server results in a 'permission denied' error when Dovecot tries to create files. So, I guess my main question is, what perms and ownership should /u/indexes be set to? I've tried several different things before this cry for help... Thanks. Mark From ms at mike2k.de Thu Nov 10 11:19:34 2011 From: ms at mike2k.de (Michael Stilkerich) Date: Thu, 10 Nov 2011 10:19:34 +0100 Subject: [Dovecot] setacl fails - does not find dovecot-acl file In-Reply-To: <1320442758.21919.222.camel@hurina> References: <4EA2C213.7030204@mike2k.de> <1320436059.21919.152.camel@hurina> <4EB44B1C.90505@mike2k.de> <1320442758.21919.222.camel@hurina> Message-ID: Hi, On Nov 4, 2011, at 10:39 PM, Timo Sirainen wrote: >> Nov 4 16:29:03 keira dovecot: imap(isa): Error: fcntl(unlock) locking >> failed for file /home/dovecot/isa/dovecot.index.log: No such file or >> directory >> Nov 4 16:29:03 keira dovecot: imap(isa): Error: fstat() failed with >> file /home/dovecot/isa/dovecot.index.log: No such file or directory > > These simply shouldn't happen. I'd say it's a kernel bug. You're running > a default Ubuntu kernel? I wonder if other Ubuntu users have this > problem. It may be an apparmor issue. I noticed plenty of apparmor log entries on these accesses, though apparmor should only log but not disallow them. I have unloaded the dovecot apparmor profiles and not seen any of these errors since then. I got a new issue, however: I migrated from Maildir to mdbox. Since then, my shared mailboxes don't fully work anymore. I have given another user full rights to a shared mailbox (getacl returns akxeilprwtscd for that folder/user). The user sees the mailbox an can perform some operations including reading and deleting messages on it. If she tries to insert a new message, however, it fails and the error log shows: dovecot: imap(isa): Error: fcntl(write-lock) locking failed for file /home/dovecot/michael/storage/dovecot.map.index.log: Bad file descriptor dovecot: imap(isa): Error: mail_index_wait_lock_fd() failed with file /home/dovecot/michael/storage/dovecot.map.index.log: Bad file descriptor All my mail locations are owned by the respective system user and the mail group, and writeable by both. In particular, I checked that both the storage directory as well as the dovecot.map.index.log are writeable by the mail group. The users are not regular members of the mail group, but my dovecot config contains mail_access_groups = mail Any idea how to resolve this issue? -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4813 bytes Desc: not available URL: From alexdosh at gmail.com Thu Nov 10 11:27:36 2011 From: alexdosh at gmail.com (Alexander KIper) Date: Thu, 10 Nov 2011 13:27:36 +0400 Subject: [Dovecot] Exim thru through Dovecot deliver to spec IMAP-folder Message-ID: Hello All! How can I "post" some mails from Exim trnasport through Dovecot deliver to IMAP MailBox in specific folder (for example: "Junk")? -- With best regards, Alexander KIper! From usenet at schani.com Thu Nov 10 11:40:56 2011 From: usenet at schani.com (wwl) Date: Thu, 10 Nov 2011 10:40:56 +0100 Subject: [Dovecot] Limit the number of pop/imap requests Message-ID: Hello Group, is it possible to Limit the number of pop/imap requests to Dovecot? I read the Wiki and found max_mail_processes = 512 is this the way to limit this ? dovecot --version 1.2.15 Thanx for help Christian From david.ocana at bsc.es Thu Nov 10 13:14:50 2011 From: david.ocana at bsc.es (David Ocana) Date: Thu, 10 Nov 2011 12:14:50 +0100 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <1320861378.21919.376.camel@hurina> References: <4EBA4DD4.4080505@bsc.es> <1320861378.21919.376.camel@hurina> Message-ID: <4EBBB22A.90501@bsc.es> On 09/11/11 18:56, Timo Sirainen wrote: > On Wed, 2011-11-09 at 10:54 +0100, David Ocana wrote: > >> I've been trying to set up dovecot 2.0.15, everything seems to work >> pretty well except for the quota feature. I would like to set a quota >> limit only for the Inbox folder. I configured two namespaces, >> according to some posts from Timo Sirainen >> >> namespace { >> separator = / >> prefix = INBOX/ >> location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n >> inbox = yes >> hidden = yes >> } >> >> plugin { >> quota = dirsize:User quota > > quota = dirsize:User quota:ns=INBOX/ Actually I forgot to mention that I also tried that, but I got the following error: Error: Initialization failed: Failed to initialize quota: Quota root User quota: Unknown parameter for backend dirsize: ns=INBOX/ That's why I was trying to change quota settings by using the quota_rule directive. > > This limits the quota only to mailboxes in INBOX/ namespace. > >> # I've tried with: >> quota_rule = INBOX:storage=819200K >> quota_rule = INBOX/*:storage=819200K >> quota_rule = INBOX/Inbox:storage=819200K > > Quota rules don't work in this way. There are no per-mailbox quotas > really, at least in the way you're thinking about. > I see, I guess they're per-namespace quotas, right? I got the wrong idea after watching the following, which was exactly what I wanted to do :p quota_rule = : May be that, using dirsize backend lets you no other option than calculating quota for the whole user's mailbox? -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4439 bytes Desc: S/MIME Cryptographic Signature URL: From stan at hardwarefreak.com Thu Nov 10 14:46:05 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 06:46:05 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EBBC78D.2060809@hardwarefreak.com> On 11/9/2011 10:37 PM, Alexander Chekalin wrote: > Oh, that's the point to consider. > > But I must confess I'm in love with Maildir for maybe 10 years This love affair may be coming to and end. >...for that simple fact I can do anything with each and every single message even on disk (=much faster than via IMAP). If I would deal with mbox directly I'd need to parse huge files, brrrr. Mbox is an excellent mailbox format for archived mail *because of* the fact that searching it is very fast and the disk subsystem overhead is low. For example, on my decade+ old 550MHz x86 SOHO server with only 384MB RAM and a single 7.2k SATA disk, after dropping caches, we'll search my debian-users mbox archive (my largest) for total message count by searching a known header of every message: -rw------- 1 stan stan 133M Nov 10 06:03 1-Debian-Users ~/mail$ time grep -c Content-Length 1-Debian-Users 22817 real 0m1.731s user 0m0.328s sys 0m0.852s Now let's search for posts from me (after dropping caches again): ~/mail$ time grep -c "From: Stan Hoeppner" 1-Debian-Users 536 real 0m1.657s user 0m0.216s sys 0m0.896s Nested greps will obviously take longer, as will those using perl expressions, but this gives some indication of the kind of speed we're talking about: less than seconds to search 22,000+ messages for a specific single header. So that's ~20 seconds for an mbox containing 220K+ messages, again on 10+ year old hardware. > Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? You should probably take a look at Enkive. I'm not sure what mail storage format it uses, and I've not used it personally, so I can't vouch for its speed, but it's pretty complete feature-wise. Take the test drive--nice search interface. http://www.enkive.org/ -- Stan From stan at hardwarefreak.com Thu Nov 10 15:17:42 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 07:17:42 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EBB628F.8060705@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EBB628F.8060705@lazurit.com> Message-ID: <4EBBCEF6.5040502@hardwarefreak.com> On 11/9/2011 11:35 PM, Alexander Chekalin wrote: > Hello, Stan, > > in fact the only thing I miss even with my current scheme is permanent > ID assigned to the message so I can easily find it despite the IMAP > mailbox it is now (so if someone moved the message from one > mailbox/folder to another, the ID allows to retrieve it fast anyway). > > You see, what I need is not only find message from|to someone on > specified date, I also sometime need to restore that message back to > user's original box. As far our mailserver and backup-mailserver are > different machines, it is a bit tricky to copy messages between it fast > enough. Say, if I need to find and restore all mails from > user at domain.com within 2009 year, and search yields in some 1000's of > messages, then use IMAP to copy it over to another server takes some > time - and if you consider both search time and restore/copy time the > whole process may take "ages". Apparently I didn't fully understand all of your requirements. Moving the archived mail to mbox/mdbox and/or getting a good indexing search engine installed will cut the search time down tremendously. Whether that would make up for the time consumed with an IMAP copy of many emails I don't know. If your servers aren't old and slow, and are not already overloaded, I would think the IMAP message copying over GbE would be pretty quick, even for the 1000 messages scenario. There may be some Dovecot tweaks that might make this copy process faster. Timo would need to chime in on that. Do you perform the IMAP transfers with a GUI IMAP client on your management PC? Or are you using imapsync or some other util directly on the servers? If the former you may be able to tweak your IMAP client to speed up the transfers as well. Try using IMAP and not IMAPS for the transfers. What is the network infrastructure between the servers and your management workstation? Is it all GbE with jumbo frames enabled? > With maildir I can rsync/scp needed files to another host and that's > fast way - that's why I stick with maildir. There is definitely some flexibility here. > FTS in my case can help (I can search for user at domain.com, for example), > but it also return messages that contains such a string in message body > (and that takes index space, too), so I'll need to filter it later, but > surely it'll be faster than checking every message in the archive. Sure. So you're concerned with your poor performance, but also with disk space. Unfortunately there's no free lunch to be had. You'll have to make sacrifices somewhere. You could go with mdbox and use compression, trading that saved space for search index files space. -- Stan From stan at hardwarefreak.com Thu Nov 10 15:19:57 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 10 Nov 2011 07:19:57 -0600 Subject: [Dovecot] Exim thru through Dovecot deliver to spec IMAP-folder In-Reply-To: References: Message-ID: <4EBBCF7D.1080904@hardwarefreak.com> On 11/10/2011 3:27 AM, Alexander KIper wrote: > Hello All! > > How can I "post" some mails from Exim trnasport through Dovecot deliver to > IMAP MailBox in specific folder (for example: "Junk")? Dovecot 1.x: http://wiki.dovecot.org/LDA/Sieve Dovecot 2.x: http://wiki2.dovecot.org/Pigeonhole/Sieve -- Stan From tss at iki.fi Thu Nov 10 17:29:35 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 10 Nov 2011 17:29:35 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: On 10.11.2011, at 6.37, Alexander Chekalin wrote: > Are there any ways I can search or parse mboxes or mdboxes not directly and not with IMAP (I'm afraid it slooow in dump parsing)? See doveadm fetch / doveadm search. > in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway). Dovecot has message GUIDs (with maildir it's filename), but there's no quick lookup for them, even though doveadm can fetch them easily: doveadm fetch text guid 12312312 From adrian.minta at gmail.com Thu Nov 10 18:03:14 2011 From: adrian.minta at gmail.com (Adrian Minta) Date: Thu, 10 Nov 2011 18:03:14 +0200 Subject: [Dovecot] Quota BUG - fixed In-Reply-To: <1320864362.21919.380.camel@hurina> References: <1320861585.21919.378.camel@hurina> <4EBAC7FE.5060207@gmail.com> <1320864083.21919.379.camel@hurina> <4EBACA0F.1010200@gmail.com> <1320864362.21919.380.camel@hurina> Message-ID: <4EBBF5C2.7020403@gmail.com> After some deep investigations I manage to solve the problem. I was only reading quota in user_querry. Now I read it in user_querry and in password_query and all seems fine: --dovecot-sql.conf--- user_query = SELECT '/home/%d/%n' as home, 'maildir:/home/%d/%n' as mail, 150 AS uid, 8 AS gid, CONCAT('*:bytes=', quota) AS quota_rule FROM mailbox WHERE username = '%u' AND active = '1' password_query = SELECT username as user, password, '/home/%d/%n' as userdb_home, 'maildir:/home/%d/%n' as userdb_mail, 150 as userdb_uid, 8 as userdb_gid, CONCAT('*:bytes=', quota) as userdb_quota_rule FROM mailbox WHERE username = '%u' AND active = '1' --dovecot.conf--- plugin { quota = dict:user::proxy::quotadict quota_rule2 = Trash:storage=10%% quota_rule3 = SPAM:storage=10%% } the result is fine now: 2 getquotaroot inbox * QUOTAROOT "INBOX" "user" * QUOTA "user" (STORAGE 1997999 2000000) 2 OK Getquotaroot completed. Only one "cosmetic" bug remains when an empty mailbox appear as a small negative number in quota2 table, but this is fixable in postfixadmin. -- Best regards, Adrian Minta MA3173-RIPE,www.minta.ro From micah at riseup.net Thu Nov 10 20:59:32 2011 From: micah at riseup.net (Micah Anderson) Date: Thu, 10 Nov 2011 13:59:32 -0500 Subject: [Dovecot] dovecot-lda quota rule Message-ID: <874nybztuz.fsf@algae.riseup.net> I really like the feature where you can define quota rules with percents which trigger off of the default values[0] (so you can set the Trash to allow for 10% more of the user's quota for example). What I would really love in dovecot would be for the ability to configure a quota rule for dovecot-lda. I would like to configure things so we don't bounce emails for users until they are well over quota, the IMAP quota plugin is a really great way to notify people that they are over quota because it fails to write to other folders.... that should be enough to get people's attention that they need to deal with things, but bouncing is harsh. Is there a way to do this now that I haven't seen? thanks! micah 0. http://wiki2.dovecot.org/Quota/Configuration -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From carloswill at gmail.com Thu Nov 10 21:17:15 2011 From: carloswill at gmail.com (Carlos Mennens) Date: Thu, 10 Nov 2011 14:17:15 -0500 Subject: [Dovecot] TLS Authentication Confusion Message-ID: I asked a user today to make sure his incoming and outgoing email was using TLS. He told me it wasn't possible because my Dovecot / Postfix daemons were only listening on TCP 25 & 143 according to a port scan he did. He told me the only way I could enable encrypted secure sessions between the client & server is to enable port 993 (IMAPs). I told him that TLS is supported on my mail server over the default ports TCP 25 / 143 and that many consider IMAPs to be legacy. I sent him a telnet session of my PC communicating with my server & it shows TLS is available. I just wanted to be sure I was correct with the information above or am I completely wrong and I do indeed need TCP port 993? I know this is the Dovecot mailing list but since Dovecot and Postfix both use and support TLS in their configuration files, I figured I would ask here for your help! carloss at pc1:~$ telnet mail.holyghost.org 25 Trying 192.168.4.100... Connected to mail.holyghost.org. Escape character is '^]'. 220 mail.holyghost.org ESMTP Postfix EHLO pc1.holyghost.org 250-mail.holyghost.org 250-PIPELINING 250-SIZE 20480000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN Below is a snip from my mail logs showing TLS: Nov 9 10:26:39 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=::ffff:192.168.4.100, lip=::ffff:192.168.4.100, TLS The above snip from my log means that I'm connecting to Dovecot via TLS, correct? From dick at fouter.net Thu Nov 10 21:28:55 2011 From: dick at fouter.net (Dick Middleton) Date: Thu, 10 Nov 2011 19:28:55 +0000 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: References: Message-ID: <4EBC25F7.1040700@fouter.net> On 11/10/11 19:17, Carlos Mennens wrote: > I asked a user today to make sure his incoming and outgoing email was > using TLS. He told me it wasn't possible because my Dovecot / Postfix > daemons were only listening on TCP 25 & 143 according to a port scan > he did. He told me the only way I could enable encrypted secure > sessions between the client & server is to enable port 993 (IMAPs). Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same port as plain. The difference between SSH and TLS is that with SSH the encryption is set up before any application communication takes place. i.e all application packets are contained in the encrypted payload. With TLS the application starts communication and then the application sets up encryption of its payload. Dick From stephan at rename-it.nl Thu Nov 10 21:37:56 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 10 Nov 2011 20:37:56 +0100 Subject: [Dovecot] patching dovecot for sieve/managesieve support, centos 5.6? In-Reply-To: <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> References: <1320298314.47419.YahooMailNeo@web130114.mail.mud.yahoo.com> <1320898142.23172.YahooMailNeo@web130112.mail.mud.yahoo.com> Message-ID: <4EBC2814.2090200@rename-it.nl> This mail was answered before. Don't repost your question unless you have acted on the information provided, got new information or have additional questions. Re-posting the exact same message makes no sense. Regards, Stephan. On 11/10/2011 5:09 AM, Scott Lewis wrote: > > > ----- Forwarded Message ----- > From: Scott Lewis > To: "dovecot at dovecot.org" > Sent: Thursday, 3 November 2011 4:31 PM > Subject: patching dovecot for sieve/managesieve support, centos 5.6? > > > Hi all, > > I am having real trouble when attempting to patch dovecot 1.2 to include the Pidgeonhole sieve support on my CentOS 5.6 x64 mail server. I am relatively new to the programming side of linux, but I am not having a lot of luck when trying to get this thing to compile. > > Here's what happens: > > [root at mail ~]# whereis dovecot > dovecot: /usr/sbin/dovecot /etc/dovecot.conf /usr/lib/dovecot /usr/libexec/dovecot /usr/share/man/man8/dovecot.8.gz > > [root at mail dovecot-1.2-sieve-0.1.19]# ./configure --with-dovecot=/usr/lib/dovecot > > ... > > checking whether to build static libraries... yes > dovecot-config not found from /usr/lib/dovecot, use --with-dovecot=PATH > to give path to compiled Dovecot sources or to a directory with the > installed dovecot-config file. configure: error: > dovecot-config not found > > -- > > I get this message regardless of whether I set --with-dovecot as /usr/sbin/dovecot, or /etc, or /usr/libexec/dovecot. > > I have SquirrelMail 1.4.22 running, and the avelsieve front-end seems happy enough. when I visit https://mail.mydomain.com/src/configtest.php, I get: > > Avelsieve plugin details: backend = ManageSieve > ERROR: I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=Managesieve) is bad? > > thanks in advance! From frank at moltke28.B.Shuttle.DE Thu Nov 10 21:56:50 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Thu, 10 Nov 2011 20:56:50 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC25F7.1040700@fouter.net> References: <4EBC25F7.1040700@fouter.net> Message-ID: (auto-added) On Thu, 10 Nov 2011 19:28:55 +0000 Dick Middleton wrote: > On 11/10/11 19:17, Carlos Mennens wrote: > > I asked a user today to make sure his incoming and outgoing email was > > using TLS. He told me it wasn't possible because my Dovecot / Postfix > > daemons were only listening on TCP 25 & 143 according to a port scan > > he did. He told me the only way I could enable encrypted secure > > sessions between the client & server is to enable port 993 (IMAPs). > > Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same > port as plain. > > The difference between SSH and TLS is that with SSH the encryption is set up > before any application communication takes place. i.e all application packets > are contained in the encrypted payload. With TLS the application starts > communication and then the application sets up encryption of its payload. :%s/SSH/SSL/g --Frank From tom at whyscream.net Thu Nov 10 22:11:01 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 10 Nov 2011 21:11:01 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC25F7.1040700@fouter.net> References: <4EBC25F7.1040700@fouter.net> Message-ID: <4EBC2FD5.2060602@whyscream.net> On 10-11-11 20:28, Dick Middleton wrote: > On 11/10/11 19:17, Carlos Mennens wrote: >> I asked a user today to make sure his incoming and outgoing email was >> using TLS. He told me it wasn't possible because my Dovecot / Postfix >> daemons were only listening on TCP 25 & 143 according to a port scan >> he did. He told me the only way I could enable encrypted secure >> sessions between the client & server is to enable port 993 (IMAPs). > > Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same > port as plain. > > The difference between SSH and TLS is that with SSH the encryption is set up > before any application communication takes place. i.e all application packets > are contained in the encrypted payload. With TLS the application starts > communication and then the application sets up encryption of its payload. > You're contributing to the confusion. SSL and TLS are practically the same, just another name for the same beast. The only difference is that SSL is the old name, and newer versions of the standard are labeled TLS. The term SSH is not in the scope of this question. There are 2 ways of using SSL/TLS to encrypt sessions: 1) Setup a dedicated port where a SSL/TLS session can be setup before the actual data is transferred. This is what happens for IMAPS/993 and SMTPS/465. 2) Extend an existing protocol to enable SSL/TLS during an open session. This is called STARTTLS in several protocols, SMTP and IMAP being among them. And this is what happens on SMTP/25, Submission/587 and IMAP/143. Note that although the second option is *named* STARTTLS, you probably could implement any server to *use* SSL 1.0 for the actual encryption (not recommended though). The OP is offering STARTTLS for both services, which is good. -- Regards, Tom From noeldude at gmail.com Thu Nov 10 22:21:49 2011 From: noeldude at gmail.com (Noel) Date: Thu, 10 Nov 2011 14:21:49 -0600 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBC2FD5.2060602@whyscream.net> References: <4EBC25F7.1040700@fouter.net> <4EBC2FD5.2060602@whyscream.net> Message-ID: <4EBC325D.60404@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11/10/2011 2:11 PM, Tom Hendrikx wrote: > On 10-11-11 20:28, Dick Middleton wrote: >> On 11/10/11 19:17, Carlos Mennens wrote: >>> I asked a user today to make sure his incoming and outgoing email was >>> using TLS. He told me it wasn't possible because my Dovecot / Postfix >>> daemons were only listening on TCP 25 & 143 according to a port scan >>> he did. He told me the only way I could enable encrypted secure >>> sessions between the client & server is to enable port 993 (IMAPs). >> >> Yes you are right. Port 993 is for IMAPS (SSH). TLS is normally on the same >> port as plain. >> >> The difference between SSH and TLS is that with SSH the encryption is set up >> before any application communication takes place. i.e all application packets >> are contained in the encrypted payload. With TLS the application starts >> communication and then the application sets up encryption of its payload. >> > > You're contributing to the confusion. > > SSL and TLS are practically the same, just another name for the same > beast. The only difference is that SSL is the old name, and newer > versions of the standard are labeled TLS. The term SSH is not in the > scope of this question. > > There are 2 ways of using SSL/TLS to encrypt sessions: > > 1) Setup a dedicated port where a SSL/TLS session can be setup before > the actual data is transferred. This is what happens for IMAPS/993 and > SMTPS/465. > > 2) Extend an existing protocol to enable SSL/TLS during an open session. > This is called STARTTLS in several protocols, SMTP and IMAP being among > them. And this is what happens on SMTP/25, Submission/587 and IMAP/143. > > Note that although the second option is *named* STARTTLS, you probably > could implement any server to *use* SSL 1.0 for the actual encryption > (not recommended though). > > The OP is offering STARTTLS for both services, which is good. > > -- > Regards, > Tom The confusion is caused by the way some client software differentiate these services in their configuration, often referring to wrappermode smtps/imaps as "SSL", and STARTTLS as "TLS". -- Noel Jones -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOvDJcAAoJEHIluGOd3V4F6foH/16+xq91/j4hgXufdnAsxwW1 N2ZXf1fby7TjR4BpaYNdH6PsN5/UqFSZItVYkeDXWgGG/wYCTRC+LHdks/EeQKgR 1ondUL2iorQ7bGy25m3526DGShFmcEh7P+Z6WWwdFeOTLBS57LIgwvFHBg4niYHq 3ZbPOjzI+d7kbz8tT8ATb+Ju+uJlV2rpbZKHQ90qlOR9tRl6bUOEeW32yPf5hjpI gs89o66Ud+mb9kkH9vgrhnutxsWjVxWNWM1ba43S1bh4Jg9YneIdsHdQVQSPrFUz EPy5Tgz3b+LZC6lwe6czFrhYgv/GUiJutS34qRHLSMAQGY+fgOcZBSZQHKP7NC4= =TdNE -----END PGP SIGNATURE----- From sven at svenhartge.de Thu Nov 10 23:15:05 2011 From: sven at svenhartge.de (Sven Hartge) Date: Thu, 10 Nov 2011 22:15:05 +0100 Subject: [Dovecot] LDAP expired password References: <116877.53915.qm@web29503.mail.ird.yahoo.com> <57o6m4h3miv8@mids.svenhartge.de> <4D95A998.9080701@noa.gr> <97o72ha3miv8@mids.svenhartge.de> Message-ID: rpalmarin wrote: > Sven Hartge svenhartge.de> writes: >> Nikolaos Milas noa.gr> wrote: >>> On 1/4/2011 11:09 ??, Sven Hartge wrote: >>>> Have a look at the ppolicy slapd.overlay. This will solve your >>>> problem. > Sorry for the delay in the response I checked the ppolicy overlay but > without success. This overlay does not have a single "password > expired" attribute to put in the user_filter. I think you misunderstood the usage of the overlay. There is _no_ additional attribute to check. With ppolicy any authentication will fail if some previously defined conditions are met (or no longer met) like the max age of a password. Documentation is contained in "man slapo-ppolicy", which as bit hard to understand, I must admit. Also look at http://www.openldap.org/doc/admin24/overlays.html "12.10 Password Policies" has a nice example. With this overlay you don't need any additional attributes and no maintenance or houskeeping script to invalidate expired passwords. >> At my university we introduced our own attribute gifb-status which >> contains a "1" if an account is valid, a "0" if it is not (and >> several others for different purposes) and our ldap-filters all >> contain something like "(&(ou=foobar)(gifb-status=1))". > is possible that the only way to do this is to manage a new attribute? > how can understand all the people that have configured the mail > client to authenticate with imap-dovecot that their passoword has > expired? Well, either way (using ppolicy or an additional attribute): they will call the support desk, if they are unable to understand the message from their mail client. No way to fix _this_ problem, I am afraid ;) S? -- Sigmentation fault. Core dumped. From frank at moltke28.B.Shuttle.DE Fri Nov 11 13:01:05 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 12:01:05 +0100 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop Message-ID: (auto-added) Hi, running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA I've found a strange error in log which I've nerver observed before when startung dovecot: auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop Dovecot seems to work normal despite the error message. Any enlightment welcome. --Frank Elsner From tss at iki.fi Fri Nov 11 13:10:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 11 Nov 2011 13:10:57 +0200 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop In-Reply-To: (auto-added) References: (auto-added) Message-ID: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> On 11.11.2011, at 13.01, Frank Elsner wrote: > running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA .. > auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop authdb_imap is from Dovecot v2.1. It doesn't exist in v2.0.14.. From peter at newton.cx Fri Nov 11 09:41:21 2011 From: peter at newton.cx (Peter Williams) Date: Thu, 10 Nov 2011 23:41:21 -0800 Subject: [Dovecot] Invalid "doveadm search" queries not rejected Message-ID: <1320997281.2261.116.camel@iota.localdomain> Hello, I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS 6). According to the doveadm-search-query(7) manual page, a "mailbox" specification can only come at the beginning of a search query. However, if I put a mailbox specification in the middle of a query, not only do I not get an error message, I get surprising results: $ doveadm search mailbox A |wc -l 6979 $ doveadm search mailbox B |wc -l 206 $ doveadm search mailbox A OR mailbox B |wc -l 42492 If it became possible to do boolean logic on mailbox queries, that'd be cool. But barring that, these invalid queries should be rejected. Relatedly, if I can only do simple mailbox filtering, does anyone know of a clever way to efficiently emulate more complex boolean logic? Please CC me on any replies as I'm not subscribed to the list. Thanks, Peter -- Peter Williams / peter at newton.cx From frank at moltke28.B.Shuttle.DE Fri Nov 11 16:28:44 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 15:28:44 +0100 Subject: [Dovecot] undefined symbol: o_stream_switch_ioloop In-Reply-To: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> References: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> Message-ID: (auto-added) On Fri, 11 Nov 2011 13:10:57 +0200 Timo Sirainen wrote: > On 11.11.2011, at 13.01, Frank Elsner wrote: > > > running dovecot-2.0.14 under Fedora 14 with kernel 2.6.35.14-103.fc14.i686.PA > .. > > auth-worker: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop > > authdb_imap is from Dovecot v2.1. It doesn't exist in v2.0.14.. Nov 11 15:22:13 seymour dovecot: master: Dovecot v2.0.15 starting up (core dumps disabled) Nov 11 15:22:13 seymour dovecot: auth: Error: dlopen(/usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so) failed: /usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so: undefined symbol: o_stream_switch_ioloop This is definitly version 2.0.15 starting. But ... /usr/local/dovecot/lib/dovecot/auth has date Nov 9 10:17 usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so has date Nov 9 10:17 This was the time I've tested dovecot-2.1. --Frank Elsner From frank at moltke28.B.Shuttle.DE Fri Nov 11 16:37:17 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 11 Nov 2011 15:37:17 +0100 Subject: [Dovecot] Re(2): undefined symbol: o_stream_switch_ioloop In-Reply-To: References: <4A275E86-F697-463F-9F9A-35CF728B454C@iki.fi> Message-ID: (auto-added) On Fri, 11 Nov 2011 15:28:44 +0100 Frank Elsner wrote: [ ... ] > But ... > > /usr/local/dovecot/lib/dovecot/auth has date Nov 9 10:17 > usr/local/dovecot/lib/dovecot/auth/libauthdb_imap.so has date Nov 9 10:17 > > This was the time I've tested dovecot-2.1. After removing all files from Nov 9 10:17 in /usr/local/dovecot/lib/dovecot version 2.0.15 start without the error message :-) --Frank Elsner From dovecot-20110531 at billmail.scconsult.com Fri Nov 11 18:00:47 2011 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Fri, 11 Nov 2011 11:00:47 -0500 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: References: Message-ID: <4EBD46AF.40507@billmail.scconsult.com> Carlos Mennens wrote, On 11/10/11 2:17 PM: > I asked a user today to make sure his incoming and outgoing email was > using TLS. He told me it wasn't possible because my Dovecot / Postfix > daemons were only listening on TCP 25& 143 according to a port scan > he did. He told me the only way I could enable encrypted secure > sessions between the client& server is to enable port 993 (IMAPs). I > told him that TLS is supported on my mail server over the default > ports TCP 25 / 143 and that many consider IMAPs to be legacy. I sent > him a telnet session of my PC communicating with my server& it shows > TLS is available. I just wanted to be sure I was correct with the > information above or am I completely wrong and I do indeed need TCP > port 993? You are correct. To convince your user, point him at http://en.wikipedia.org/wiki/STARTTLS which has links to the relevant RFC's. > > I know this is the Dovecot mailing list but since Dovecot and Postfix > both use and support TLS in their configuration files, I figured I > would ask here for your help! > > carloss at pc1:~$ telnet mail.holyghost.org 25 > Trying 192.168.4.100... > Connected to mail.holyghost.org. > Escape character is '^]'. > 220 mail.holyghost.org ESMTP Postfix > EHLO pc1.holyghost.org > 250-mail.holyghost.org > 250-PIPELINING > 250-SIZE 20480000 > 250-VRFY > 250-ETRN > 250-STARTTLS > 250-ENHANCEDSTATUSCODES > 250-8BITMIME > 250 DSN > > Below is a snip from my mail logs showing TLS: > > Nov 9 10:26:39 mail dovecot: imap-login: Login: user=, > method=PLAIN, rip=::ffff:192.168.4.100, lip=::ffff:192.168.4.100, TLS > > The above snip from my log means that I'm connecting to Dovecot via > TLS, correct? Yes From james.page at ubuntu.com Fri Nov 11 18:32:32 2011 From: james.page at ubuntu.com (James Page) Date: Fri, 11 Nov 2011 16:32:32 +0000 Subject: [Dovecot] Restarting dovecot periodically fails with "Socket already exists: /var/run/dovecot/anvil" In-Reply-To: <1320513622.21919.260.camel@hurina> References: <4E97000D.7030908@ubuntu.com> <1320513622.21919.260.camel@hurina> Message-ID: <4EBD4E20.50800@ubuntu.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 05/11/11 17:20, Timo Sirainen wrote: > http://hg.dovecot.org/dovecot-2.1/rev/6894298ae5fd should apply to > v2.0 as well, but since it's a bit large change I don't know if I > should backport it, at least without some more testing. Thanks for this pointer Timo I've patched this into dovecot in the latest Ubuntu development release - it appears to resolve the issue and we are a good 6 months from release so plenty of time to a) flush out any problems or b) upgrade to 2.1. Cheers James - -- James Page Ubuntu Core Developer -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJOvU4gAAoJEL/srsug59jD7fYQAIy+wnPiYuv58wazO23V4nyA ut2D6iaChsNG7XrySNtgY/RajOu9O8dIOnLuLTD9DmxbRZEaH2Tzd/7n6I0P3i7t h3PyI3tW7GPFyII/VOnAsTjfmncjF/ug7ohyUw2CYc1z2D7lQE0zTFlQeEWCqDCz mxKEHg37TvPlRZzfSvzgjn2hoTJ/vmhQPM11I8uEVYOnWWwahCZxkA2rgBmkjtEG mhCZfgoaJ6BZ5fH0BbjChN5Vr4zuQVEws8fkbGX/7/JPUxKHkz7jdxhVuZr7YN5o xy0HfGRr4sqa1v6qqTAFvFmZu6n3V6dVvn64SkjDLPECnOH80HxPzwMKN9+/cnYi d54/xIYH5N0pPhnSah+/caYCJp5FqDeAmim7xpMLExMzRdqvfbaI7ggGILecN3cH iJiWyj32jCArlAFRFKbxrin8VMRFhU4i+S7zigDp3jFuMotMnL5e1Wk+qrDjKRvt XhairFpT9po5rSvJp/J6qJxWrb+1vznBQkKgl91o570AjBBbRfmYvhxtnU5j2FZ9 MwA8pAD9+n7dc5aGY3o8ABa+5/0wqkAt8qGQ8HYuq+I86NwUbjTadvFLZcfy1Vy5 OOSoMPeruz2tQTCb0/PIHM35MFCmm45Lo0ZamqFmCi4iRKcKwx4tVesJHZY51lGu f29+i9iDlLmofPvhUQkF =hTb+ -----END PGP SIGNATURE----- From dovecot at knutejohnson.com Fri Nov 11 18:33:09 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Fri, 11 Nov 2011 08:33:09 -0800 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBD46AF.40507@billmail.scconsult.com> References: <4EBD46AF.40507@billmail.scconsult.com> Message-ID: <4EBD4E45.4080402@knutejohnson.com> So is a STARTTLS on port 143 session somehow superior to using SSL on port 993? Can you still limit connections on port 143 to require STARTTLS? Thanks, -- Knute Johnson From tkrah at fachschaft.imn.htwk-leipzig.de Fri Nov 11 20:43:22 2011 From: tkrah at fachschaft.imn.htwk-leipzig.de (Torsten Krah) Date: Fri, 11 Nov 2011 19:43:22 +0100 Subject: [Dovecot] TLS Authentication Confusion In-Reply-To: <4EBD4E45.4080402@knutejohnson.com> References: <4EBD46AF.40507@billmail.scconsult.com> <4EBD4E45.4080402@knutejohnson.com> Message-ID: <1321037002.2772.1.camel@sf045> Am Freitag, den 11.11.2011, 08:33 -0800 schrieb Knute Johnson: > So is a STARTTLS on port 143 session somehow superior to using SSL on > port 993? Can you still limit connections on port 143 to require STARTTLS? Yes you can disable authentication until starttls is issued. > > Thanks, > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5253 bytes Desc: not available URL: From bambero at gmail.com Sat Nov 12 06:32:31 2011 From: bambero at gmail.com (Bambero) Date: Sat, 12 Nov 2011 05:32:31 +0100 Subject: [Dovecot] submission_host problem Message-ID: Hello, I configured dovecot to use submission smtp host becouse of chroot. submission_host = 127.0.0.1 Unfortunatelly: Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next input="MAIL FROM:\\r\\n" Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next input="MAIL FROM:\\r\\n" Nov 12 05:11:15 myhost dovecot: lda(alias at apisoft.pl): Error: smtp(127.0.0.1): RCPT TO failed: 554 SMTP synchronization error Nov 12 05:11:15 myhost dovecot: lda(alias at apisoft.pl): Error: sieve: msgid=: failed to redirect message to (refer to server log for more information) Does anyone know how to fix it ? Regards, Bambero From rajeev at teamsg.in Sat Nov 12 09:28:42 2011 From: rajeev at teamsg.in (Rajeev Sharma) Date: Sat, 12 Nov 2011 12:58:42 +0530 Subject: [Dovecot] Migration From Courier Imap. Message-ID: <653a51f07131e61c52bf983359f4710c@192.168.0.251> Sir, I am using vpopmail,qmail along with courier-imap 4.0 i want to migrate from courier to dovecot but i have 100 shared folder under /home/vpopmail/domains/domain.com/Malidir and user shared them on the basis of rights assign. I am very much clear how to migrate from courier to dovecot but i am unable how to migrate shared folder and assign rights. Rajeev Sharma. -- Rajeev Sharma From mnbbrown at gmail.com Sun Nov 13 11:49:21 2011 From: mnbbrown at gmail.com (Matthew Brown) Date: Sun, 13 Nov 2011 19:49:21 +1000 Subject: [Dovecot] Dovecot v2.0.13 unix_listeners Message-ID: <604BA92C-C1D5-4694-B17E-E678C5FFFC0D@gmail.com> Could somebody be able to explain to me what the various unix_listeners do in the Dovecot 2.x configuration (specifically in 10-master.conf). Currently, for postfix to use for local delivery, I have: service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } and for auth I have: service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0666 user = vmail } } So what does each one specifically do? Also, does somebody know of a resource that can explain the mode setting? I couldn't seem to find anything on the wiki.. Sorry if these questions seem simple, Im still trying to get a grips with dovecots workings. MDA: Dovecot v2.0.13 MTA: SMTP User & Virtual Domain Management: MySQL Cheers, Matthew From scott_the_musician at yahoo.com.au Sun Nov 13 12:40:20 2011 From: scott_the_musician at yahoo.com.au (Scott Lewis) Date: Sun, 13 Nov 2011 02:40:20 -0800 (PST) Subject: [Dovecot] [re] patching dovecot for managesieve/sieve Message-ID: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> My apologies for the repost yesterday; it was not intentional. I was attempting to post a followup from my mobile, when something went awry with the editing options on my phone. Anyway.. ?I have managed to download and compile my own patched dovecot/dovecot-sieve/dovecot-managesieve modules from scratch, with Thanks to this page hardc0l2e.wordpress.com/2010/02/26/managesieve-with-dovecot-on-centos-5-4/ But when I run http://mydomain.com/src/configtest.php , I am still getting the following rather non-descriptive error: "I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=ManageSieve) is bad?" Where else can I look to resolve this? should managesieve show up as its own /etc/init.d service? Or does dovecot simply control it? How can I confirm that deliver is being used as my LDA? Thanks again, Scott From andreas at kado-web.de Sun Nov 13 16:48:49 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Sun, 13 Nov 2011 15:48:49 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive Message-ID: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> Hi List, i am trying to restore mails from an backup-archive which was made with rsnapshot. All the mails in the archive look similar like the following: -rw------- 2 1014 1015 3308 8. Aug 13:42 1312803768.V809I266a00aM505178.server:2,ST Now I copy the files from the archive into the Maildir/cur directory of the user. Then I set the permissions etc. and log in with the user account via Squirrelmail. The restored mails aren't in the INBOX, and when I go into the Maildir/cur dir of the user, the restored mails are gone. Possibly deleted after the login. Had anyone some similar experiences ? Any hints ? Kind regards Andreas From bambero at gmail.com Sun Nov 13 18:25:55 2011 From: bambero at gmail.com (Bambero) Date: Sun, 13 Nov 2011 17:25:55 +0100 Subject: [Dovecot] Chroot Message-ID: Hello, I have successfully setup dovecot with chroot but there are two things yet 1. submission_host problem (I posted message 2 days ago) 2. Problem with acl's and sharing: all maildirs are located in: /var/mail/jdoe at example.com/Maildir so: mail_chroot=/var/mail I setup acls with: 10-mail.conf ... namespace { type = private separator = / prefix =. inbox = yes hidden = no list = yes subscriptions = yes } namespace { type = shared separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = yes list = yes } ... 90-acl.conf ... plugin { acl = vfile:cache_secs=5 } plugin { acl_shared_dict = file:/%i.db } ... Setting acl is no problem there are dovecot-acl dovecot-acl-list in Maildir. and dictionary /var/mail/xxx.db contains list od shares I can also see shared folder in my IMAP client but when trying to subscribe i got error: Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No such file or directory Error: Namespace 'shared/': Could not lookup home for user jdoe at example.com /var/dovecot/auth-userdb is not accessible because chroot. But why userdb is necessary in this case ? I hope there is a solution without disabling chroot. Below my basic configuration. Regards, Bambero # 2.0.13: /etc/dovecot/dovecot.conf # OS: OpenBSD 5.0 i386 auth_mechanisms = plain login digest-md5 cram-md5 apop default_internal_user = _dovecot default_login_user = _dovenull first_valid_gid = 1000 first_valid_uid = 1000 last_valid_gid = 10000 last_valid_uid = 10000 mail_chroot = /var/mail mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date imapflags notify mbox_write_locks = fcntl mmap_disable = yes passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir:User quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_global_path = /etc/sieve/default.sieve } protocols = imap pop3 sieve sieve service auth { unix_listener auth-client { mode = 0660 user = _exim } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/ca-bundle.crt ssl_cert = References: <604BA92C-C1D5-4694-B17E-E678C5FFFC0D@gmail.com> Message-ID: <20111113163741.Horde.s9ZFZpLnE6FOwDilUtzVjrA@kishi.patrickdk.com> Heh? they pretty much do exactly as described. the service lmtp is how you inject email into dovecot using the lmtp protocol. the service auth is how you can authenicate user/password against dovecot, normally used in postfix for sasl auth the mode setting is just that, the file mode to set for that file, 0666, all access, 0660 only user/group access. Quoting Matthew Brown : > Could somebody be able to explain to me what the various > unix_listeners do in the Dovecot 2.x configuration (specifically in > 10-master.conf). > > Currently, for postfix to use for local delivery, I have: > > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > } > > and for auth I have: > > service auth { > > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > > unix_listener auth-userdb { > mode = 0666 > user = vmail > } > } > > So what does each one specifically do? > > Also, does somebody know of a resource that can explain the mode > setting? I couldn't seem to find anything on the wiki.. > Sorry if these questions seem simple, Im still trying to get a grips > with dovecots workings. > > > MDA: Dovecot v2.0.13 > MTA: SMTP > User & Virtual Domain Management: MySQL > > Cheers, > Matthew From georg.martius at web.de Sun Nov 13 23:57:46 2011 From: georg.martius at web.de (Georg Martius) Date: Sun, 13 Nov 2011 22:57:46 +0100 Subject: [Dovecot] metadata plugin problems Message-ID: <3455528.juJhefTHFy@chugul> Hi, I have trouble with the metadata plugin. I compiled dovecot 2.0.15 from source and also the plugin [2] separately. I work on debian/lenny. I had some trouble compiling it (autoconf somehow didn't insert the DOVECOT_IMAP_INCLUDE variable, I have not idea why) Anyway, I did as described in [1]. Do I have to create: the file /var/lib/dovecot/shared-metadata ? The server starts without errors, but I get for instance the following error message: "Failed to write some annotations for 'Contacts' on the IMAP server. SetMetaData failed, server replied: A000076 NO Parsing entry failed" from my KMail. How can I test that the plugin is actually active and working? Any help would be appreciated. Regards, Georg [1]http://www.mail-archive.com/dovecot at dovecot.org/msg38887.html [2] http://hg.dovecot.org/dovecot-metadata-plugin From dovecot-20110531 at billmail.scconsult.com Mon Nov 14 00:27:02 2011 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Sun, 13 Nov 2011 17:27:02 -0500 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> References: <0M3wNK-1R7FQp2mfa-00rmSi@mrelayeu.kundenserver.de> Message-ID: <4EC04436.3090905@billmail.scconsult.com> Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From pw at wk-serv.de Mon Nov 14 01:31:24 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 14 Nov 2011 00:31:24 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111081450.25798.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> Message-ID: <4EC0534C.8070409@wk-serv.de> Peer Heinlein schrieb: > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on HP EVA. You have 11 TB of mails on a non cluster filesystem? Is it only accessed from one server or how does it work? From andreas at kado-web.de Mon Nov 14 08:09:01 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Mon, 14 Nov 2011 07:09:01 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <4EC04436.3090905@billmail.scconsult.com> Message-ID: <0MHQzX-1RdoD93uYl-00Dcge@mrelayeu.kundenserver.de> Thanks for that information bill. The T was the point which was important for me, because the user didn't tell me that the messages were already marked for deletion. - Anyway, in outlook the mails were visible (marked for deletion). -----Urspr?ngliche Nachricht----- Von: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Bill Cole Gesendet: Sonntag, 13. November 2011 23:27 An: Dovecot Mailing List Betreff: Re: [Dovecot] Restoring Mails from Backup Archive Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From andreas at kado-web.de Mon Nov 14 08:09:20 2011 From: andreas at kado-web.de (Andreas Cieslak) Date: Mon, 14 Nov 2011 07:09:20 +0100 Subject: [Dovecot] Restoring Mails from Backup Archive In-Reply-To: <4EC04436.3090905@billmail.scconsult.com> Message-ID: <0M2TGj-1QaLdl2jLB-00sOsn@mrelayeu.kundenserver.de> Thanks for that information bill. The T was the point which was important for me, because the user didn't tell me that the messages were already marked for deletion. - Anyway, in outlook the mails were visible (marked for deletion). -----Urspr?ngliche Nachricht----- Von: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] Im Auftrag von Bill Cole Gesendet: Sonntag, 13. November 2011 23:27 An: Dovecot Mailing List Betreff: Re: [Dovecot] Restoring Mails from Backup Archive Andreas Cieslak wrote, On 11/13/11 9:48 AM: > Hi List, > > > > i am trying to restore mails from an backup-archive which was made with > rsnapshot. > > > > All the mails in the archive look similar like the following: > > > > -rw------- 2 1014 1015 3308 8. Aug 13:42 > 1312803768.V809I266a00aM505178.server:2,ST The portion of the filename after the ':' is a message status structure. In that structure, the letter 'T' means that the message has been marked for deletion by the user. As a result, most IMAP clients will not show the message to the user and an "EXPUNGE" command on the mailbox will delete the message. From p.heinlein at heinlein-support.de Mon Nov 14 11:34:02 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Mon, 14 Nov 2011 10:34:02 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EC0534C.8070409@wk-serv.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> Message-ID: <201111141034.03060.p.heinlein@heinlein-support.de> Am Montag, 14. November 2011, 00:31:24 schrieb Patrick Westenberg: > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > HP EVA. > > You have 11 TB of mails on a non cluster filesystem? Yes. I don't believe a clustered filesystem would have more performance and would be more rock solid. I don't have a problem on my frontend server. Why should I have two or more of them? I have a problem in my backend. My SAN has too much to do. Why should a cluster filesystem be better for my SAN? > Is it only accessed from one server or how does it work? Yes. peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From arif at team.de.inter.net Mon Nov 14 11:51:20 2011 From: arif at team.de.inter.net (Kai Arif) Date: Mon, 14 Nov 2011 10:51:20 +0100 Subject: [Dovecot] patch for UID 0 problem Message-ID: <4EC0E498.7000900@team.de.inter.net> Hi all, attached you'll find a patch for cmd-thread.c which addresses a problem with certain clients in conjunction with the use of the UID THREAD REFS command: TAG5 UID THREAD REFS us-ascii SINCE 14-May-2011 * THREAD (0)(246)(247)(248)(249)(250)(252)(253)(254)(255)(256)(257)(258)(259)(260)(261)(262)(263)(264)(265)(266)(267)(268) TAG5 OK Thread completed. The first UID delivered (0) is invalid, some clients - @mail for instance - then try to fetch the invalid UID 0 subsequently and run into problems. The patch actually disables the return of (0) and seems to work for us. Please review and - if OK - incorporate it in the next version. Best Regards Kai -- Kai Arif - System Administrator Inter.net Germany GmbH Knesebeckstra?e 59-61 10719 Berlin Germany Fon +49 30 25430 0 Fax +49 30 25430 499 arif at team.de.inter.net www.de.inter.net Legal-Information: Inter.net Germany GmbH, HRB #79136, Amtsgericht Berlin Charlottenburg, UST-IdNr:: DE 813 165 159 FA f?r K?rperschaften I Berlin, Gesch?ftsf?hrer: J?rn Lubkoll Zust?ndige Regulierungsbeh?rde: Bundesnetzagentur Chemnitz, Liselotte-Herrmann-Str. 20a, 09127 Chemnitz, Registriernummer: 06/164 -------------- next part -------------- A non-text attachment was scrubbed... Name: cmd-thread.c.diff Type: text/x-diff Size: 644 bytes Desc: not available URL: From stephan at rename-it.nl Mon Nov 14 12:06:22 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 14 Nov 2011 11:06:22 +0100 Subject: [Dovecot] [re] patching dovecot for managesieve/sieve In-Reply-To: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> References: <1321180820.61715.YahooMailNeo@web130121.mail.mud.yahoo.com> Message-ID: <4EC0E81E.2090401@rename-it.nl> On 13-11-2011 11:40, Scott Lewis wrote: > My apologies for the repost yesterday; it was not intentional. I was attempting to post a followup from my mobile, when something went awry with the editing options on my phone. > Anyway.. > > I have managed to download and compile my own patched dovecot/dovecot-sieve/dovecot-managesieve modules from scratch, with Thanks to this page > > hardc0l2e.wordpress.com/2010/02/26/managesieve-with-dovecot-on-centos-5-4/ > > But when I run http://mydomain.com/src/configtest.php , I am still getting the following rather non-descriptive error: > > "I could not determine the capabilities for Sieve Mail Filtering. Perhaps connectivity with ManageSieve server (if backend=ManageSieve) is bad?" You can test whether ManageSieve is working properly using this procedure: http://wiki.dovecot.org/ManageSieve/Troubleshooting > Where else can I look to resolve this? should managesieve show up as its own /etc/init.d service? Or does dovecot simply control it? > How can I confirm that deliver is being used as my LDA? The Dovecot LDA produces characteristic log lines in your system log (http://wiki1.dovecot.org/Logging). For instance, with Sieve enabled, the following is reported upon delivery to INBOX: Nov 14 10:53:41 host dovecot: deliver(stephan): sieve: msgid=: stored mail into mailbox 'INBOX' Something similar is logged without Sieve. All such messages are prefixed with 'deliver(): '. With mail_debug=yes, more verbose messages are logged as well. Regards, Stephan. From janfrode at tanso.net Mon Nov 14 12:27:19 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 14 Nov 2011 11:27:19 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: <201111141034.03060.p.heinlein@heinlein-support.de> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> Message-ID: <20111114102719.GA2330@dibs.tanso.net> On Mon, Nov 14, 2011 at 10:34:02AM +0100, Peer Heinlein wrote: > > > I have> 11 TB hard used Mailstorage, saved als maildir in ext3 on > > > HP EVA. > > > > You have 11 TB of mails on a non cluster filesystem? > > Yes. > > I don't believe a clustered filesystem would have more performance and > would be more rock solid. > > I don't have a problem on my frontend server. Why should I have two or > more of them? I have a problem in my backend. My SAN has too much to do. > Why should a cluster filesystem be better for my SAN? Agree. A non-clustered fs should give you better performance, and probably also be more reliable, if you can live with the SPoF and full downtime during patching/upgrades/maintenance. But I would expect xfs to be a better choice than ext*. We have about the same storage size as you (12TB/115M-inodes), with the backup-process almost biting itself in the tail every day, but I can't quite imagine running it all on a single local fs with no scale-out options if we should want/need more processing power for dovecot. I'm looking forward to moving to mdbox soonish.. to reduce the number of files and speed up the backup process. -jf From tlx at leuxner.net Mon Nov 14 13:30:08 2011 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 14 Nov 2011 12:30:08 +0100 Subject: [Dovecot] Managing public folder ACL files In-Reply-To: <1302268626.24579.79.camel@hurina> References: <20110406115434.3049ba70@kae.tiger-computing.wbp> <1302268626.24579.79.camel@hurina> Message-ID: <20111114113008.GA5126@nihlus.leuxner.net> On Fri, Apr 08, 2011 at 04:17:06PM +0300, Timo Sirainen wrote: > > doveadm mailbox list -u admin at domain "sales/CustA/*" | > while read mailbox; do > doveadm acl set -u user at domain "$mailbox" user at domain +lrw > done Have the +lrw modifiers been implemented at all? I can only get it to work with syntax like: $ doveadm acl set -u user at domain "Shops/ACME" owner insert lookup post read write write-seen This is a bit painful to type and does not allow to add or remove rights... Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From achekalin at lazurit.com Mon Nov 14 16:35:07 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 14 Nov 2011 17:35:07 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> Message-ID: <4EC1271B.4060604@lazurit.com> Timo, Stan, I've just tested mdbox and find it pretty nice for me, but now I got some questions for you: 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if 2Mb. Looks like not even every message can fit into such storage container volume (nowadays we used to see messages of 20Mb and even more). Should I tune it (at least mdbox_rotate_size and mdbox_rotate_interval) or its size is on purpose? As for now I store each day's messages in separate IMAP folders (mailboxes), which gives me 2000-6000 messages and 2-5 Gb (on disk) per folder. 2. I can use no compression, gz and bz2 - which one will be better for storing archive messages? I've just tested mdbox by copying 5800+ msgs from maildir to compressed mdbox, and it took exactly the same size (2.8 G) in 100+ small m.* files. No good as far. 3. What if I use maildir as I do now but turn on compression, will this speed things up? I'd like to use mdbox as storage but for now it is very new for me and I simple afraid what should I do if I'll need to manually fix the storage (maildir is really good for that, surely). After all, I simple need to speed up the search and restore process in archive. Yours, Alexander From stan at hardwarefreak.com Mon Nov 14 20:23:14 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 14 Nov 2011 12:23:14 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC1271B.4060604@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> Message-ID: <4EC15C92.4000707@hardwarefreak.com> On 11/14/2011 8:35 AM, Alexander Chekalin wrote: > Timo, Stan, > > I've just tested mdbox and find it pretty nice for me, but now I got > some questions for you: > > 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if > 2Mb. Looks like not even every message can fit into such storage > container volume (nowadays we used to see messages of 20Mb and even > more). Should I tune it (at least mdbox_rotate_size and > mdbox_rotate_interval) or its size is on purpose? As for now I store > each day's messages in separate IMAP folders (mailboxes), which gives me > 2000-6000 messages and 2-5 Gb (on disk) per folder. mdbox_rotate_size of 2MB is too small for your needs. Test 32MB and 64MB. > 2. I can use no compression, gz and bz2 - which one will be better for > storing archive messages? I've just tested mdbox by copying 5800+ msgs > from maildir to compressed mdbox, and it took exactly the same size (2.8 > G) in 100+ small m.* files. No good as far. bzip2 may give you a little better compression but at the cost of much lower de/compression speed and higher CPU and memory consumption. gzip will be faster all around, between 4x-8x, with lower mem usage, but with less compression resulting in slightly larger file sizes than bzip2. > 3. What if I use maildir as I do now but turn on compression, will this > speed things up? No. Maildir performance is limited by the disk head actuator speed, which is between 150-300 seeks per second depending on your disk (7.2k vs 15k RPM). Compressing the files doesn't change the seek physics of the disk drives. You're still reading tens of thousands of files when doing your searches thus bouncing the heads tens of thousands of times. mbox uses a single file, so head speed isn't a factor, as it may only move a few times when reading an entire mailbox file. Thus, bandwidth becomes the potential bottleneck. Using compression with large mbox files can substantially increase search performance as effective bandwidth is increased by ~4x using gzip and 6x using bzip2. This assumes you have plenty of excess CPU power. mdbox should see similar compression speedups if you use file sizes much larger than the 2MB default. Doing so should keep your IOPS well below the drive's head saturation point as you're reading only a fraction of the file count compared to maildir. > I'd like to use mdbox as storage but for now it is very new for me and I > simple afraid what should I do if I'll need to manually fix the storage > (maildir is really good for that, surely). Doveadm handles such tasks pretty well. Just make sure you keep good backups of your mdbox files. > After all, I simple need to speed up the search and restore process in > archive. The only way to accomplish this with maildir is with much bigger, faster, more expensive storage hardware. And the gain will still be much less than simply switching to a larger file format such as mbox or mdbox. As with many things some computer technologies come full circle over time. One of the reasons the creators of the UNIX mbox mail file format decided upon a single file many decades ago was the horribly limited seek performance of the slow SCSI disks of that period. Doing something like the maildir format was simply impossible at that time. In the early days of the public internet, disk became faster than the average load and maildir was born to fix the locking and corruption shortcomings of mbox. Today many sites are hitting the seek problem of a few decades ago because boxes are oversubscribed with users, emails now frequently contain attachments, everyone is storing more email, and the total volume of email is a few orders of magnitude greater. IIRC, this is one of the reasons Timo created mdbox--to decrease the massive IOPS load, and thus slow performance, of large maildir stores. -- Stan From moseleymark at gmail.com Mon Nov 14 20:23:37 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 14 Nov 2011 10:23:37 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: Message-ID: On Mon, Sep 26, 2011 at 10:11 AM, Mark Moseley wrote: > On Fri, Sep 23, 2011 at 3:35 PM, Mark Moseley wrote: >> I've been goofing with this all day with 2.0.15 and I'm starting to >> realize that either a) I'm not that smart, b) it's been so long since >> I messed with namespaces that I'm going about it completely wrong, or >> c) it's just not possible. I haven't posted 'doveconf -n' and other >> details, because mainly I'm just looking for 'yes, this is possible' >> or 'no, you're smoking crack' before posting further details. At this >> point, it's all maildir and moving to mdbox, while highly desirable in >> the future, is not possible in the near- to medium-term. >> >> I'm trying to glue a namespace underneath INBOX: >> >> namespace INBOX { >> ? ? ? ?type = private >> ? ? ? ?separator = . >> ? ? ? ?prefix = INBOX. ? ?# Yes, this used to be on Courier >> ? ? ? ?inbox = yes >> ? ? ? ?list = yes >> ? ? ? ?hidden = no >> ? ? ? ?subscriptions = yes >> ? ? ? ?location = maildir:~/Maildir >> } >> namespace archive { >> ? ? ? ?type = private >> ? ? ? ?separator = . >> ? ? ? ?prefix = INBOX.Archives. >> ? ? ? ?inbox = no >> ? ? ? ?list = children >> ? ? ? ?subscriptions = yes >> ? ? ? ?location = maildir:~/Maildir-Archive >> } >> >> >> I've tried putting namespace archive's 'prefix' as just "Archives", >> but Tbird doesn't seem to see this namespace, regardless of how much I >> futz with the imap settings in tbird. >> >> With the above setup, it actually seems to work correctly (provided >> ~/Maildir-Archive exists), though I'm sure a big gotcha is waiting in >> the wings. I can move messages around, create subfolders, subscribe to >> folders in ~/Maildir-Archive). The only thing I can't seem to get >> working is quotas. With my password_query like: >> >> password_query = ... >> CONCAT( '*:bytes=', "1M" ) AS 'userdb_quota_rule', \ >> CONCAT( '*:messages=10' ) AS 'userdb_quota_rule2', \ >> CONCAT( 'INBOX.Archives:bytes=+4900M' ) AS 'userdb_quota_rule3', \ >> CONCAT( 'INBOX.Archives:messages=+3900' ) AS 'userdb_quota_rule4' >> ... >> >> only the default quota seems to be in place for any subfolder of >> INBOX.Archives and for INBOX.Archives itself, i.e. *:bytes still >> applies to INBOX.Archives. The debug log show that: >> >> Debug: Quota root: name=User quota backend=maildir args= >> Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=0 >> Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=10 >> Debug: Quota rule: root=User quota mailbox=INBOX.Archives >> bytes=+5138022400 messages=0 >> Debug: Quota rule: root=User quota mailbox=INBOX.Archives >> bytes=+5138022400 messages=+3900 >> >> These are wildly stupid quotas but they're just there to test. With >> INBOX already at capacity (byte-wise; only set to a meg), copying >> large messages inside INBOX.Archives fails (only copying a 800k >> message but the quota should be 5gig now). >> >> Again, before I post configs, I'm just curious if what I'm trying to >> do isn't remotely possible, or that I'm approaching this entirely >> wrongly. Thanks! >> > > Thanks to a fortuitously unrelated thread ("how to disable quota for > second namespace"), I got the quota part figured out and that seems to > be working: Add a second entry to plugin {}, e.g. "quota2 = > maildir:Archive quota:ns=INBOX.Archives." and add rules for > userdb_quota2_rule, userdb_quota2_rule2, etc. > > My real question now is: Are there any fatal gotchas in this that I'm > just not thinking of? > Haven't had a chance to try this large-scale yet. Anybody have any thoughts on it? From stan at hardwarefreak.com Mon Nov 14 20:42:30 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 14 Nov 2011 12:42:30 -0600 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111114102719.GA2330@dibs.tanso.net> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> Message-ID: <4EC16116.1000902@hardwarefreak.com> On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > Agree. A non-clustered fs should give you better performance, and > probably also be more reliable, if you can live with the SPoF and > full downtime during patching/upgrades/maintenance. But I would expect > xfs to be a better choice than ext*. Depends on the workload characteristics and how well the XFS filesystem is tuned to the storage hardware. If setup properly, using many allocation groups with fast spindles, a decent amount of BBWC, and a high concurrency maildir workload (dozens to hundreds of delivery and IMAP operations), XFS will runs circles around EXTx as it can create/write/read to every AG in parallel. Much of EXT4's operation is still serialized. This is why XFS outruns all other filesystems in the highly parallel mail workload benchmarks I posted previously, EXTx by a factor of 2-3. For smaller hosts that don't see parallelism, for example SOHO servers, XFS will likely be slower than EXTx as the workload will be serialized. -- Stan From achekalin at lazurit.com Mon Nov 14 23:16:53 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 15 Nov 2011 00:16:53 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC15C92.4000707@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> Message-ID: <43B341A1-9245-4863-932A-887E21970319@lazurit.com> Locking issues on mbox is the reason for my long-lasting love affair with maildir, and it's lasts long years. Ok, the life's lessons are like this, learn something and move on with it ;) even if it's "new old thing". Thank you for pointing that! What I was doubt about is default rotate size of 2M, since I used to see pretty reasonable default settings in all Dovecot config. 32 or 64 are much close to the ones I'd personally prefer. I also about to choose now is the OS and FS for the archive. I seriously think about ZFS with compression (in fact it will be stripes over couple of mirrors = software equivalent of RAID 10 on SATA drives, with compression on FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox itself. I see pros and contras for both, so that's the question to answer! Yours, Alexander > On 11/14/2011 8:35 AM, Alexander Chekalin wrote: >> Timo, Stan, >> >> I've just tested mdbox and find it pretty nice for me, but now I got >> some questions for you: >> >> 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if >> 2Mb. Looks like not even every message can fit into such storage >> container volume (nowadays we used to see messages of 20Mb and even >> more). Should I tune it (at least mdbox_rotate_size and >> mdbox_rotate_interval) or its size is on purpose? As for now I store >> each day's messages in separate IMAP folders (mailboxes), which gives me >> 2000-6000 messages and 2-5 Gb (on disk) per folder. > > mdbox_rotate_size of 2MB is too small for your needs. Test 32MB and 64MB. > >> 2. I can use no compression, gz and bz2 - which one will be better for >> storing archive messages? I've just tested mdbox by copying 5800+ msgs >> from maildir to compressed mdbox, and it took exactly the same size (2.8 >> G) in 100+ small m.* files. No good as far. > > bzip2 may give you a little better compression but at the cost of much > lower de/compression speed and higher CPU and memory consumption. gzip > will be faster all around, between 4x-8x, with lower mem usage, but with > less compression resulting in slightly larger file sizes than bzip2. > >> 3. What if I use maildir as I do now but turn on compression, will this >> speed things up? > > No. Maildir performance is limited by the disk head actuator speed, > which is between 150-300 seeks per second depending on your disk (7.2k > vs 15k RPM). Compressing the files doesn't change the seek physics of > the disk drives. You're still reading tens of thousands of files when > doing your searches thus bouncing the heads tens of thousands of times. > > mbox uses a single file, so head speed isn't a factor, as it may only > move a few times when reading an entire mailbox file. Thus, bandwidth > becomes the potential bottleneck. Using compression with large mbox > files can substantially increase search performance as effective > bandwidth is increased by ~4x using gzip and 6x using bzip2. This > assumes you have plenty of excess CPU power. mdbox should see similar > compression speedups if you use file sizes much larger than the 2MB > default. Doing so should keep your IOPS well below the drive's head > saturation point as you're reading only a fraction of the file count > compared to maildir. > >> I'd like to use mdbox as storage but for now it is very new for me and I >> simple afraid what should I do if I'll need to manually fix the storage >> (maildir is really good for that, surely). > > Doveadm handles such tasks pretty well. Just make sure you keep good > backups of your mdbox files. > >> After all, I simple need to speed up the search and restore process in >> archive. > > The only way to accomplish this with maildir is with much bigger, > faster, more expensive storage hardware. And the gain will still be > much less than simply switching to a larger file format such as mbox or > mdbox. > > As with many things some computer technologies come full circle over > time. One of the reasons the creators of the UNIX mbox mail file format > decided upon a single file many decades ago was the horribly limited > seek performance of the slow SCSI disks of that period. Doing something > like the maildir format was simply impossible at that time. In the > early days of the public internet, disk became faster than the average > load and maildir was born to fix the locking and corruption shortcomings > of mbox. > > Today many sites are hitting the seek problem of a few decades ago > because boxes are oversubscribed with users, emails now frequently > contain attachments, everyone is storing more email, and the total > volume of email is a few orders of magnitude greater. > > IIRC, this is one of the reasons Timo created mdbox--to decrease the > massive IOPS load, and thus slow performance, of large maildir stores. > > -- > Stan From steveh at itbasecamp.com.au Tue Nov 15 02:08:33 2011 From: steveh at itbasecamp.com.au (Steve H) Date: Tue, 15 Nov 2011 00:08:33 +0000 Subject: [Dovecot] XUIDL Migration problems Message-ID: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> Hello All, I am new to Dovecot and the Dovecot mailing list. I am trying to migrate an existing Exim + qpopper - mbox (pop3 only) installation over to Exim + Dovecot using mdir (pop3 and imap) And am running into problems with preserving the XUIDL headers. I am running Dovecot 2.0.14 I have used the mb2md.pl script : http://dovecot.org/tools/mb2md.pl from this page http://wiki2.dovecot.org/Migration/MailFormat to migrate and import the mboxes And have the following pop3 uidl settings in my configuration . pop3_reuse_xuidl = yes pop3_save_uidl = yes protocols = imap pop3 service imap-login { inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } protocol pop3 { pop3_reuse_xuidl = yes pop3_save_uidl = yes pop3_uidl_format = %08Xu%08Xv } The problem specifically is that when I test the migration and import a mailbox, I am able to connect via IMAP using Thunderbird and Outlook 2007 and the Read / Unread status is preserved. However when I connect via POP3 the Read/Unread status is not preserved. I am probably missing something simple, but have been unable to find anything that seems relevant on the Dovecot wiki or searching the mailing lists. Any suggestions would be appreciated. Regards, Steven Holloway IT Technical Consultant I.T. Basecamp P/L [http://www.itbasecamp.com.au/signature.gif] Address: Office 1 Level 1/50 Owen St Huskisson NSW 2540 Phone: +61 2 44418555 ext 308 Fax: +61 2 44418551 Web: http://www.itbasecamp.com.au Subscribe to our newsletter! Visit: IT Basecamp to manage your subscription. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 2602 bytes Desc: image001.gif URL: From tss at iki.fi Tue Nov 15 02:19:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:19:17 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC1271B.4060604@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> Message-ID: <23D65BA4-912A-43EE-8480-8586DEAD76FA@iki.fi> On 14.11.2011, at 16.35, Alexander Chekalin wrote: > 1. mdbox uses 'a lot' files (m.1, m.2 ... etc), and the default size if 2Mb. Looks like not even every message can fit into such storage container volume (nowadays we used to see messages of 20Mb and even more). The messages are never split into multiple files. So if you have a 20 MB message, it gets stored into its own m.* file. > Should I tune it (at least mdbox_rotate_size and mdbox_rotate_interval) or its size is on purpose? As for now I store each day's messages in separate IMAP folders (mailboxes), which gives me 2000-6000 messages and 2-5 Gb (on disk) per folder. The main problem with larger mdbox files is that if you expunge messages, there's more data to write when packing the data into a new file. I don't really know the "best" value for mdbox_rotate_size setting. But even a 2 MB mdbox file can contain thousands of small mails, so it's not too bad.. From tss at iki.fi Tue Nov 15 02:21:09 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:21:09 +0200 Subject: [Dovecot] XUIDL Migration problems In-Reply-To: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> References: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> Message-ID: <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> On 15.11.2011, at 2.08, Steve H wrote: > I am trying to migrate an existing Exim + qpopper - mbox (pop3 only) installation over to Exim + Dovecot using mdir (pop3 and imap) > And am running into problems with preserving the XUIDL headers. Are you sure the UIDLs are the problem? Did you connect to old and new POP3 server and compare their UIDL lists? > However when I connect via POP3 the Read/Unread status is not preserved. Another possibility would be simply that if you changed the hostname in the client config, it might have thought that it's a completely different server. From tss at iki.fi Tue Nov 15 02:38:15 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 02:38:15 +0200 Subject: [Dovecot] patch for UID 0 problem In-Reply-To: <4EC0E498.7000900@team.de.inter.net> References: <4EC0E498.7000900@team.de.inter.net> Message-ID: On 14.11.2011, at 11.51, Kai Arif wrote: > attached you'll find a patch for cmd-thread.c which addresses a problem with certain clients in > conjunction with the use of the UID THREAD REFS command: > > TAG5 UID THREAD REFS us-ascii SINCE 14-May-2011 > * THREAD > (0)(246)(247)(248)(249)(250)(252)(253)(254)(255)(256)(257)(258)(259)(260)(261)(262)(263)(264)(265)(266)(267)(268) > TAG5 OK Thread completed. > > The first UID delivered (0) is invalid, some clients - @mail for instance - then try to fetch the > invalid UID 0 subsequently and run into problems. Yes, it is. It should never happen. > The patch actually disables the return of (0) and seems to work for us. Please review and - if OK - > incorporate it in the next version. Could you instead send me such a mailbox where you can reproduce this problem? Probably sending dovecot.index, dovecot.index.log and dovecot.index.thread files would be enough. None of those contain any sensitive information. From steveh at itbasecamp.com.au Tue Nov 15 03:14:26 2011 From: steveh at itbasecamp.com.au (Steve H) Date: Tue, 15 Nov 2011 01:14:26 +0000 Subject: [Dovecot] XUIDL Migration problems In-Reply-To: <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> References: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8AA2@ITBEXCH.office.itb.net.au> <7F6AFF61-4D14-4D84-8B53-F8EC9CB0E50D@iki.fi> Message-ID: <7FF85722635E4A4FBAF06D40D6BC128F5A5F8DB2@ITBEXCH.office.itb.net.au> Hi Timo, Thanks for the quick reply. > Are you sure the UIDLs are the problem? Did you connect to old and new POP3 server and compare their UIDL lists? No not 100% sure that the UIDLs are the issue; but running into dead ends with what I have tried. I will compare the UIDL lists from the old / new server and report back. > However when I connect via POP3 the Read/Unread status is not preserved. > Another possibility would be simply that if you changed the hostname in the client config, it might have thought that it's a completely different server. The new server is a completely different host and currently had a different IP, but it will end up with the same name and IP as the existing host when we cut over. At this time I have manually adjusted my lost hosts file to point towards the new host for testing. Would this be a problem even though the IMAP appears to be working were as the POP is not.?? I'll report back on the UIDLs Regards, Steven From mysqlstudent at gmail.com Tue Nov 15 04:30:35 2011 From: mysqlstudent at gmail.com (Alex) Date: Mon, 14 Nov 2011 21:30:35 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail Message-ID: Hi, I have a fedora15 system with dovecot-v2.0.15, dovecot-pigeonhole and and amavisd-new and would like to implement it on my system using postfix. I've read the basic wiki documentation, but I really can't figure out the proper starting point in my case. I've enabled "protocol lda" in 15-lda.conf and added "sieve" to the list of mail_plugins. I'm trying to adapt postfix to use deliver as its LDA for multiple virtual domains. I've added the following to master.cf: dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient} -a "${RECIPIENT}" Is that the proper format? Is it okay to use the dovecot user here, or should I create a new user? What permissions does this user need to possess? I've added the following to main.cf: dovecot_destination_recipient_limit = 1 virtual_transport = dovecot I have a number of procmail scripts and I've converted them to sieve, but they didn't convert very well. Can someone show me how to convert the following: :0 { :0 fhW | /home/alex/bin/process-mail.pl } The script skipped right over that. I'm also trying to convert the following: :0 * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) * !USER_IN_WHITELIST * !BAYES_00 * !URIBL_ * !KHOP_RCVD_UNTRUST * !RCVD_IN_DNSWL_MED * !SPF_SOFTFAIL * SPF_HELO_PASS * SPF_PASS { LOG="DNSWL: " :0: whitelist/dnswl } The script only produced the following: if true { if true { if true { if true { if true { if true { if true { if true { if true { if true { fileinto "{"; fileinto " LOG="DNSWL: ""; fileinto " :0:"; fileinto " whitelist/dnswl"; fileinto "}"; } Any ideas greatly appreciated. Thanks, Alex From JTRUTWIN at CSBSJU.EDU Tue Nov 15 05:34:55 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Tue, 15 Nov 2011 03:34:55 +0000 Subject: [Dovecot] getenv() in plugin not working Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> Hi - new to the list, can't find much on this using google. I'm trying to setup the dovecot DRAC plugin as described here: http://wiki.dovecot.org/HowTo/PopBSMTPAndDovecot#DRAC DRAC is installed and running using this startup command: /usr/local/sbin/rpc.dracd -i -e 5 /etc/postfix/dracd.db I downloaded the drac.c file linked on the above URL: http://www.dovecot.org/patches/1.1/drac.c and built it using this command: gcc -Wall -W -shared -fPIC -DHAVE_CONFIG_H drac.c -I/usr/include/dovecot -L/usr/lib64/dovecot -o drac.so -ldrac Then I copied drac.so to /usr/lib64/dovecot (this is an x86_64 SL 6.1 system - dovecot 2.0.9) and modified my /etc/dovecot/conf.d/20-imap.conf file: mail_plugin_dir = /usr/lib64/dovecot mail_plugins = drac Restart dovecot and try to login and getting these messages for ANY login attempts: dovecot: imap-login: Login: user=, method=PLAIN, rip=, lip=, mpid=4018, TLS server dovecot: imap(: Error: DRAC: IP environment not given I traced the "IP environment not given" to the drac.c plugin file, and the code is pretty simple: void drac_init(void) { const char *ip_str; struct ip_addr ip; ip_str = getenv("IP"); if (ip_str == NULL) i_error("DRAC: IP environment not given"); Is there anything special that needs to be done for getenv("IP") to report the login IP? I tried to put getenv("RIP") for remote ip and even getenv("USER") as well but always get NULL. Is this API only for dovecot 1.x? If so is there a 2.0 equivalent? The rest of the plugin is very basic... I think this is the last step in getting this setup running. I'd have to have this one last piece force me to run that icky perl program instead. Thank you, Let me know if anything else is needed. Josh From JTRUTWIN at CSBSJU.EDU Tue Nov 15 05:58:55 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Tue, 15 Nov 2011 03:58:55 +0000 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> > I'm trying to setup the dovecot DRAC plugin as described here: > Is there anything special that needs to be done for getenv("IP") to report the > login IP? I tried to put getenv("RIP") for remote ip and even getenv("USER") > as well but always get NULL. Is this API only for dovecot 1.x? If so is there > a 2.0 equivalent? The rest of the plugin is very basic... Appears this is the problem - I found this for dovecot v2 and it seems to be working fine: http://sourceforge.jp/projects/dovecot2-drac/ Might wanna make a note on the wiki page that the DRAC setup is only for dovecot 1.x? Thanks, Josh From stephan at rename-it.nl Tue Nov 15 09:41:37 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 15 Nov 2011 08:41:37 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: Message-ID: <4EC217B1.2040809@rename-it.nl> On 11/15/2011 3:30 AM, Alex wrote: > I have a number of procmail scripts and I've converted them to sieve, > but they didn't convert very well. Can someone show me how to convert > the following: > > :0 > { > :0 fhW > | /home/alex/bin/process-mail.pl > } > > The script skipped right over that. The Sieve language doesn't support invoking external programs. You can execute this program before Sieve, for instance by wrapping the LDA invocation. Also, there is a non-standard (an still pretty much experimental) language extension for this, as provided by the pipe plugin: http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe > I'm also trying to convert the following: > > :0 > * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) > * !USER_IN_WHITELIST > * !BAYES_00 > * !URIBL_ > * !KHOP_RCVD_UNTRUST > * !RCVD_IN_DNSWL_MED > * !SPF_SOFTFAIL > * SPF_HELO_PASS > * SPF_PASS > { > LOG="DNSWL: " > :0: > whitelist/dnswl > } > > The script only produced the following: [...] > Any ideas greatly appreciated. I'm not sure what these rules are supposed to mean (most conditions are obscured in variables), but by the looks of things these should also be executed outside Sieve as well. If all those conditions map to simple header and envelope tests, you can provide a Sieve alternative. However, I think USER_IN_WHITELIST queries some external list and that too is not (currently) possible from Pigeonhole Sieve. Also, it is not possible to change the log prefix from Sieve (as suggested by LOG=). Regards, Stephan From CMarcus at Media-Brokers.com Tue Nov 15 13:22:53 2011 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 15 Nov 2011 06:22:53 -0500 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> Message-ID: <4EC24B8D.9010401@Media-Brokers.com> On 2011-11-14 10:58 PM, Trutwin, Joshua wrote: > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? Why? You were on the wiki for v1, so obviously it is (or may be) only meant for v1. -- Best regards, Charles From bind at enas.net Tue Nov 15 17:02:22 2011 From: bind at enas.net (Urban Loesch) Date: Tue, 15 Nov 2011 16:02:22 +0100 Subject: [Dovecot] Question about pop3_reuse_xuidl Message-ID: <4EC27EFE.1060504@enas.net> Hi, we are in migration progress from Communigate Pro 5.0.x to Dovecot 2.0.15 with mdbox. We still migrated about 25.000 IMAP accounts from CGP to Dovecot. Also there were still added about 2000 new POP3 Accounts on Dovecot. Now we must migrate about 10.000 POP3 accounts from CGP to Dovecot. At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, or does Dovecot only change the uidl value for new received and still not downloaded mails? Many thanks and regards Urban Loesch From mysqlstudent at gmail.com Tue Nov 15 17:38:44 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 10:38:44 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <4EC217B1.2040809@rename-it.nl> References: <4EC217B1.2040809@rename-it.nl> Message-ID: Hi, >> I have a number of procmail scripts and I've converted them to sieve, >> but they didn't convert very well. Can someone show me how to convert >> the following: >> >> :0 >> { >> ? ? :0 fhW >> ? ? | /home/alex/bin/process-mail.pl >> } >> >> The script skipped right over that. > > The Sieve language doesn't support invoking external programs. You can > execute this program before Sieve, for instance by wrapping the LDA > invocation. Can you describe how I might do this? > Also, there is a non-standard (an still pretty much > experimental) language extension for this, as provided by the pipe plugin: Is there a way to just continue to use procmail? >> I'm also trying to convert the following: >> >> :0 >> * (RCVD_IN_DNSWL_HI|RCVD_IN_DNSWL_LOW|RCVD_IN_HOSTKARMA_W) >> * !USER_IN_WHITELIST >> * !BAYES_00 >> * !URIBL_ >> * !KHOP_RCVD_UNTRUST >> * !RCVD_IN_DNSWL_MED >> * !SPF_SOFTFAIL >> * SPF_HELO_PASS >> * SPF_PASS >> { >> ? ? ? ? LOG="DNSWL: " >> ? ? ? ? :0: >> ? ? ? ? whitelist/dnswl >> } >> >> The script only produced the following: > > [...] >> >> Any ideas greatly appreciated. > > I'm not sure what these rules are supposed to mean (most conditions are > obscured in variables), but by the looks of things these should also be > executed outside Sieve as well. If all those conditions map to simple header > and envelope tests, you can provide a Sieve alternative. However, I think > USER_IN_WHITELIST queries some external list and that too is not (currently) > possible from Pigeonhole Sieve. Also, it is not possible to change the log > prefix from Sieve (as suggested by LOG=). No, these are all simple header matches. There's no external program being executed here. The LOG= is also a procmail built-in, but I can do without that. Thanks again, Alex From stephan at rename-it.nl Tue Nov 15 18:49:00 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 15 Nov 2011 17:49:00 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> Message-ID: <4EC297FC.8050405@rename-it.nl> Op 15-11-2011 16:38, Alex schreef: > Hi, > >> The Sieve language doesn't support invoking external programs. You can >> execute this program before Sieve, for instance by wrapping the LDA >> invocation. > Can you describe how I might do this? You can write a shell script that is executed from your MTA instead of the Dovecot LDA. Perform the process-mail task from within that script and then call the Dovecot LDA from the shell script (and pipe the message to it). >> Also, there is a non-standard (an still pretty much >> experimental) language extension for this, as provided by the pipe plugin: > Is there a way to just continue to use procmail? Yes, you can pipe the message to the Dovecot LDA from procmail. This would be similar to the solution described above, only using Procmail instead of a shell script. Regards, Stephan. From mysqlstudent at gmail.com Tue Nov 15 20:09:39 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 13:09:39 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <4EC297FC.8050405@rename-it.nl> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> Message-ID: Hi, >>> The Sieve language doesn't support invoking external programs. You can >>> execute this program before Sieve, for instance by wrapping the LDA >>> invocation. >> >> Can you describe how I might do this? > > You can write a shell script that is executed from your MTA instead of the > Dovecot LDA. Perform the process-mail task from within that script and then > call the Dovecot LDA from the shell script (and pipe the message to it). So you mean from master.cf, in the same way as deliver is being executed? dovecot unix - n n - - pipe flags=DRhu user=dovecot:dovecot argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient} -a "${RECIPIENT}" Replacing deliver here with a shell script? Then execute deliver from within the shell script, or just do the delivery from within a shell script altogether? >> Is there a way to just continue to use procmail? > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > be similar to the solution described above, only using Procmail instead of a > shell script. Are you referring to doing something like this: http://wiki2.dovecot.org/procmail I just don't understand how to invoke procmail from deliver. My main motivation for trying to continue to use procmail is that the sieve language seems far more complicated and involved than even a procmail recipe, and isn't as fully developed. I don't want to make this even more complicated, however. Perhaps you can explain how to filter a basic message based on something in the body or a header and sort it into a file? Thanks so much for your help. Best, Alex From stan at hardwarefreak.com Tue Nov 15 20:26:38 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 15 Nov 2011 12:26:38 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <43B341A1-9245-4863-932A-887E21970319@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> Message-ID: <4EC2AEDE.2080306@hardwarefreak.com> On 11/14/2011 3:16 PM, Alexander Chekalin wrote: > Locking issues on mbox is the reason for my long-lasting love affair with maildir, Same reason most others fell in love with it. Many now want to divorce maildir, as the cost of the storage to maintain acceptable performance is now too high. > and it's lasts long years. Ok, the life's lessons are like this, learn something and move on with it ;) even if it's "new old thing". Thank you for pointing that! Many old UNIX gurus still use mbox, not maildir, and never will. If you ask them why they'll likely say "you don't use a screwdriver to drive a nail do you?" > What I was doubt about is default rotate size of 2M, since I used to see pretty reasonable default settings in all Dovecot config. 32 or 64 are much close to the ones I'd personally prefer. Given the fact that we're talking about an archive server, you'd be better off using a very large mdbox file size, say 1GB. You're never deleting individual messages from this archive correct? No expunges? This is why I recommended mbox in the first place. If your only writes to these mailbox files are appends of new messages, mbox is the best format by far. It's faster at appending than any other format, and it's faster for searching than any other. > I also about to choose now is the OS and FS for the archive. I seriously think about ZFS with compression (in fact it will be stripes over couple of mirrors = software equivalent of RAID 10 on SATA drives, with compression on FS level) on FreeBSD, or XFS over LVM on Debian with compression in mdbox itself. I see pros and contras for both, so that's the question to answer! It's an archive. You're not going to use maildir so you don't need random IOPS performance. Thus RAID5/6 are a much better fit for an archive as you get better read performance, with more than adequate write performance, and you use less disks. And as this is an archive, you don't need real time automatic/transparent compression. Thus I recommend something like: 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled 2.6.39 or later kernel 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 Specify a strip size of 256KB for the array Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading ahead 1024 sectors at a time instead of the default of 256. This will speed up your searches quite a bit. 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults 4. mbox w/zlib plugin. Compress daily files each night with a script 5. You don't need LVM with a good RAID card (or with mdraid). This controller can expand the RAID5 up to 8 drives (up to 32 drives max using SAS expanders) -- Stan From tss at iki.fi Tue Nov 15 21:02:43 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:02:43 +0200 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC2AEDE.2080306@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> Message-ID: <1321383763.21919.484.camel@hurina> On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. Just as long as you're not simultaneously trying to read and write the mbox file (or just write in 2+ sessions). Then there's a lot waiting on locks. (mdbox has no read locks, and its write locks are very short lived.) From micah at riseup.net Tue Nov 15 21:24:18 2011 From: micah at riseup.net (Micah Anderson) Date: Tue, 15 Nov 2011 14:24:18 -0500 Subject: [Dovecot] dsync problems Message-ID: <871ut9i3z1.fsf@algae.riseup.net> When a user renames their username, I am using dsync to copy their mail over to the new username's mail location[0]. Some of the dsyncs are failing with errors that I dont know how to work with, for example: dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=42fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=43fcd40303c8a64e432300007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=44fcd40303c8a64e432300007ef44c7a ERROR: dsync failed, exit code: 256 The errors cause a non-zero exit code from dsync, which causes my rename script to bail out. What are these errors, and how can I fix them? Thanks, micah 0. Why use dsync instead of a simple mv operation? This seems to be necessary for two corner cases: 1. dovecot creates the new mailbox automatically when the user logs in or receives a mail, so if the user changes their mail and logs in or receives an email before the move has been done, then the mailbox is created and then a move command will fail. 2. If there has been new mail created under the new name, we can't just simply remove the stuff that is automatically created there and replace it with the old things because we could potentially be removing mail that has been delivered in the mean time. I'd be really interested if people had suggestions for a better mechanism, or perhaps a way to have dovecot not create the new mail location automatically. -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Tue Nov 15 21:25:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:25:55 +0200 Subject: [Dovecot] Question about pop3_reuse_xuidl In-Reply-To: <4EC27EFE.1060504@enas.net> References: <4EC27EFE.1060504@enas.net> Message-ID: <1321385157.21919.497.camel@hurina> On Tue, 2011-11-15 at 16:02 +0100, Urban Loesch wrote: > At the beginning of our migration we didn't set the "pop3_reuse_xuidl" configuration option to yes. > Do you know what happens when we activate the "pop3_reuse_xuidl" option on our running dovecot? > > Will Dovecot change the uidl value for all existing mails which have set "X-UIDL" in its mailheader, Yes. > or does Dovecot only change the uidl value for new received and still not downloaded mails? Nope.. There is pop3_save_uidl=yes setting, which would do this for Maildir, assuming the POP3 clients had logged in while this option was enabled, but this isn't implemented for dbox. The biggest problem with implementing this for dbox would be that it has no variable length permanent per-mail storage (Maildir's dovecot-uidlist equivalent). One way to solve this would be by patching dsync a little bit, and then dsyncing existing mdbox to mdbox-new, which adds existing POP3 UIDLs as dbox metadata. Those will override X-UIDL: headers. The patching would be basically by making dsync/dsync-local-worker.c local_worker_msg_get_next() always set data.pop3_uidl variable to the correct UIDL (which would depend on your pop3_uidl_format setting - pop3/pop3-commands.c:pop3_get_uid() shows how). From tss at iki.fi Tue Nov 15 21:30:37 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:30:37 +0200 Subject: [Dovecot] getenv() in plugin not working In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E41A045C6@MAIL-MBX2.ad.csbsju.edu> <710C58696EA3BC42B425E4DBB39C1D5E41A046D8@MAIL-MBX2.ad.csbsju.edu> Message-ID: <1321385437.21919.498.camel@hurina> On Tue, 2011-11-15 at 03:58 +0000, Trutwin, Joshua wrote: > Appears this is the problem - I found this for dovecot v2 and it seems to be > working fine: > > http://sourceforge.jp/projects/dovecot2-drac/ > > Might wanna make a note on the wiki page that the DRAC setup is only for > dovecot 1.x? I changed the wiki2 link to this URL. From mgoldey at goldey.net Tue Nov 15 21:31:50 2011 From: mgoldey at goldey.net (Mark Goldey) Date: Tue, 15 Nov 2011 14:31:50 -0500 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 Message-ID: <4EC2BE26.3030807@goldey.net> Folks: I've been running Dovecot 2.0.7 on a Qmail / Vpopmail installation built on Debian Squeeze, with no problems for about a year. Dovecot is used for IMAP/S auth only. Dovecot is built from source. I just upgraded to 2.0.15 but it segfaults whenever an IMAP auth is attempted, regardless of the user who tries to login, whether by webmail (Roundmail) Android phone (K-9 Mail) or iPad (Roundmail) I downgraded to 2.0.13 but the segfault persisted. Re-installed 2.0.7 and that cleared it right up. Here's the pertinent details from 2.0.15. Anything else I can provide? root at XXX:/# uname -a Linux XXX 2.6.32-5-686 #1 SMP Wed Jan 12 04:01:41 UTC 2011 i686 GNU/Linux root at XXX:/# dovecot -n # 2.0.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.3 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no first_valid_uid = XXXX last_valid_uid = XXXX log_path = /var/log/dovecot.log mail_debug = yes passdb { driver = vpopmail } protocols = imap service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 128 vsz_limit = 64 M } ssl_cert = , method=PLAIN, rip=174.79.xxx.xxx, lip=192.xxx.xxx.xxx, mpid=691, TLS Nov 15 14:08:43 imap-login: Warning: SSL alert: where=0x4008, ret=256: warning close notify [174.79.xxx.xxx] Nov 15 14:08:43 auth-worker: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth I ought to mention that /usr/local/lib/dovecot/auth is empty. Here's the backtrace: root at XXX:/# gdb /usr/local/libexec/dovecot/auth /usr/local/var/run/dovecot/core GNU gdb (GDB) 7.0.1-debian [snip] Reading symbols from /usr/local/libexec/dovecot/auth...done. warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/i686/cmov/libcrypt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/libcrypt-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libcrypt.so.1 Reading symbols from /home/vpopmail/lib/libvpopmail.so...(no debugging symbols found)...done. Loaded symbols for /home/vpopmail/lib/libvpopmail.so Reading symbols from /lib/i686/cmov/libdl.so.2...Reading symbols from /usr/lib/debug/lib/i686/cmov/libdl-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libdl.so.2 Reading symbols from /lib/i686/cmov/librt.so.1...Reading symbols from /usr/lib/debug/lib/i686/cmov/librt-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/librt.so.1 Reading symbols from /lib/i686/cmov/libc.so.6...Reading symbols from /usr/lib/debug/lib/i686/cmov/libc-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libc.so.6 Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/i686/cmov/libpthread.so.0...Reading symbols from /usr/lib/debug/lib/i686/cmov/libpthread-2.11.2.so...done. (no debugging symbols found)...done. Loaded symbols for /lib/i686/cmov/libpthread.so.0 Reading symbols from /home/vpopmail/etc/vpopmail.authmodule...(no debugging symbols found)...done. Loaded symbols for /home/vpopmail/etc/vpopmail.authmodule Core was generated by `dovecot/auth -w'. Program terminated with signal 11, Segmentation fault. #0 0x00000000 in ?? () (gdb) bt full #0 0x00000000 in ?? () No symbol table info available. #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at auth-worker-client.c:405 auth_request = 0x82b56e8 userdb_id = 1 #2 auth_worker_handle_line (client=0x82b34d0) at auth-worker-client.c:534 id = 2 #3 auth_worker_input (client=0x82b34d0) at auth-worker-client.c:614 _data_stack_cur_id = 3 line = ret = 232 #4 0xb785f021 in io_loop_call_io (io=0x82b3688) at ioloop.c:384 ioloop = 0x82a6398 t_id = 2 #5 0xb78600d3 in io_loop_handler_run (ioloop=0x82a6398) at ioloop-epoll.c:213 ctx = 0x82b1f38 event = 0x82b1f78 list = 0x82b36b0 io = 0x829e2b0 tv = {tv_sec = 2147483, tv_usec = 0} msecs = 1 ret = 1 i = 0 j = 0 call = 192 #6 0xb785efa0 in io_loop_run (ioloop=0x82a6398) at ioloop.c:405 No locals. #7 0xb784abaa in master_service_run (service=0x82a62e8, callback=0x805c470 ) at master-service.c:481 No locals. #8 0x0805c828 in main (argc=2, argv=0x82a61c0) at main.c:298 c = (gdb) --mhg From tss at iki.fi Tue Nov 15 21:35:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:35:03 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> Message-ID: <1321385703.21919.501.camel@hurina> On Tue, 2011-11-15 at 13:09 -0500, Alex wrote: > > > Yes, you can pipe the message to the Dovecot LDA from procmail. This would > > be similar to the solution described above, only using Procmail instead of a > > shell script. > > Are you referring to doing something like this: > > http://wiki2.dovecot.org/procmail > > I just don't understand how to invoke procmail from deliver. The idea was to invoke deliver from procmail. I still do this myself since I haven't bothered changing my procmail scripts to Sieve: DELIVER=/usr/local/libexec/dovecot/deliver :0 * ^List-Id.*dovecot.dovecot.org | $DELIVER -m dovecot :0 | $DELIVER etc. From tss at iki.fi Tue Nov 15 21:39:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:39:50 +0200 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: Message-ID: <1321385991.21919.503.camel@hurina> On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote: > > Thanks to a fortuitously unrelated thread ("how to disable quota for > > second namespace"), I got the quota part figured out and that seems to > > be working: Add a second entry to plugin {}, e.g. "quota2 = > > maildir:Archive quota:ns=INBOX.Archives." and add rules for > > userdb_quota2_rule, userdb_quota2_rule2, etc. > > > > My real question now is: Are there any fatal gotchas in this that I'm > > just not thinking of? > > > > Haven't had a chance to try this large-scale yet. Anybody have any > thoughts on it? The gotcha is that you have two completely independent quotas with independent usage/limits for the INBOX and Archive namespaces. If that is what you want, it should all be fine. From tss at iki.fi Tue Nov 15 21:44:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:44:20 +0200 Subject: [Dovecot] Chroot In-Reply-To: References: Message-ID: <1321386260.21919.505.camel@hurina> On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote: > all maildirs are located in: /var/mail/jdoe at example.com/Maildir So this is /var/mail/%u/Maildir template. > namespace { > type = shared > separator = / > prefix = shared/%%u/ > location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u You can replace this with: location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u Although if this is used outside chroot (e.g. lmtp, doveadm) it won't work correctly.. > Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No > such file or directory This is done because you used %%h, to look up home directory from userdb. If you don't use %%h, it's not done. From tss at iki.fi Tue Nov 15 21:49:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 21:49:20 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: Message-ID: <1321386560.21919.509.camel@hurina> On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > I configured dovecot to use submission smtp host becouse of chroot. > > submission_host = 127.0.0.1 > > Unfortunatelly: > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > synchronization error (next input sent too soon: pipelining was not > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > input="MAIL FROM:\\r\\n" This basically says that Dovecot's SMTP client is being noncompliant, because Exim doesn't advertise PIPELINING extension. But I'd rather not add more code to fix this, since PIPELINING gives you a little bit better performance anyway and you can most likely fix this by modifying Exim's configs in some way (I can't believe Exim wouldn't support PIPENING..). From tss at iki.fi Tue Nov 15 22:16:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:16:14 +0200 Subject: [Dovecot] Migration From Courier Imap. In-Reply-To: <653a51f07131e61c52bf983359f4710c@192.168.0.251> References: <653a51f07131e61c52bf983359f4710c@192.168.0.251> Message-ID: <1321388180.21919.515.camel@hurina> On Sat, 2011-11-12 at 12:58 +0530, Rajeev Sharma wrote: > I am using vpopmail,qmail along with courier-imap 4.0 i want to > migrate from courier to dovecot but i have 100 shared folder > under /home/vpopmail/domains/domain.com/Malidir and user shared them > on the basis of rights assign. > > I am very much clear how to migrate from courier to dovecot but i am > unable how to migrate shared folder and assign rights. I don't think there is any existing documentation about this. I've also not used Courier shared folders myself much. I think the ACL files' contents are compatible, although Dovecot names them differently. IIRC Courier used some kind of symlinked maildir files for shared folders to provide per-user flags? Migrating those to Dovecot would be a bit tricky, especially because Dovecot currently supports only per-user \Seen flags (maybe Courier does too? I'm not sure) and Dovecot stores the per-user \Seen flags only in per-user index files. Anyway, I don't really have time to look into details of this, so you'll just need to figure out the specifics yourself (or alternatively pay me to do that via my company). From tss at iki.fi Tue Nov 15 22:24:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:24:57 +0200 Subject: [Dovecot] Invalid "doveadm search" queries not rejected In-Reply-To: <1320997281.2261.116.camel@iota.localdomain> References: <1320997281.2261.116.camel@iota.localdomain> Message-ID: <1321388697.21919.518.camel@hurina> On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > 6). > > According to the doveadm-search-query(7) manual page, a "mailbox" > specification can only come at the beginning of a search query. It doesn't exactly say that, but yeah, it doesn't also include MAILBOX and MAILBOX-GUID as search keys. Added now to future versions. > However, > if I put a mailbox specification in the middle of a query, not only do I > not get an error message, I get surprising results: > > $ doveadm search mailbox A |wc -l > 6979 > $ doveadm search mailbox B |wc -l > 206 > $ doveadm search mailbox A OR mailbox B |wc -l > 42492 I'm pretty sure this was fixed in v2.0.14. > If it became possible to do boolean logic on mailbox queries, that'd be > cool. It is. From tss at iki.fi Tue Nov 15 22:27:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:27:20 +0200 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <874nybztuz.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> Message-ID: <1321388840.21919.520.camel@hurina> On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > I really like the feature where you can define quota rules with percents > which trigger off of the default values[0] (so you can set the Trash to > allow for 10% more of the user's quota for example). > > What I would really love in dovecot would be for the ability to > configure a quota rule for dovecot-lda. I would like to configure things > so we don't bounce emails for users until they are well over quota, the > IMAP quota plugin is a really great way to notify people that they are > over quota because it fails to write to other folders.... that should be > enough to get people's attention that they need to deal with things, but > bouncing is harsh. You can give LDA-specific quota settings. Like maybe: protocol lda { plugin { quota_rule2 = INBOX:storage=200% } } or even not enforcing quota at all: protocol lda { plugin { quota = maildir:User quota:noenforcing } } From tss at iki.fi Tue Nov 15 22:28:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:28:57 +0200 Subject: [Dovecot] Dovecot 2.0.15 quota configuration with mbox In-Reply-To: <4EBBB22A.90501@bsc.es> References: <4EBA4DD4.4080505@bsc.es> <1320861378.21919.376.camel@hurina> <4EBBB22A.90501@bsc.es> Message-ID: <1321388937.21919.521.camel@hurina> On Thu, 2011-11-10 at 12:14 +0100, David Ocana wrote: > >> namespace { > >> separator = / > >> prefix = INBOX/ > >> location = mbox:/var/empty:INBOX=/mail/%d/%n:INDEX=/var/dovecot/%d/%n > >> inbox = yes > >> hidden = yes > >> } > >> > >> plugin { > >> quota = dirsize:User quota > > > > quota = dirsize:User quota:ns=INBOX/ > > Actually I forgot to mention that I also tried that, but I got the > following error: > > Error: Initialization failed: Failed to initialize quota: Quota root > User quota: Unknown parameter for backend dirsize: ns=INBOX/ Oh. Yeah, dirsize backend doesn't have all the features that other backends have, because it's so little used. You could switch to dict file quota.. From tss at iki.fi Tue Nov 15 22:30:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:30:16 +0200 Subject: [Dovecot] Limit the number of pop/imap requests In-Reply-To: References: Message-ID: <1321389016.21919.522.camel@hurina> On Thu, 2011-11-10 at 10:40 +0100, wwl wrote: > Hello Group, > > is it possible to Limit the number of pop/imap requests to Dovecot? > > I read the Wiki and found > > max_mail_processes = 512 > > is this the way to limit this ? This limits the number of simultaneous IMAP and POP3 connections. Is that what you mean with "requests"? If so, yeah, you can reduce this value. I think the counts are tracked separately for IMAP and POP3 (so you can have 512 IMAP and 512 POP3 connections). From tss at iki.fi Tue Nov 15 22:32:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:32:08 +0200 Subject: [Dovecot] indexes permissions problem In-Reply-To: <4EBB8BA7.9070603@cruzio.com> References: <4EBB8BA7.9070603@cruzio.com> Message-ID: <1321389128.21919.523.camel@hurina> On Thu, 2011-11-10 at 00:30 -0800, Mark Hanford wrote: > I've got a centos 6 server running Dovecot 2.0.beta6 (3156315704ef). > For legacy reasons (I'm moving mail from a Dovecot 1.1.1 and FreeBSD box > with user home directories NFS mounted), my index files are setup to be > in /u/indexes/ > > On the Dovecot 1.1.1 installation, the perms on the indexes directory is > 777 with root:mail ownership. > > The same thing on the Dovecot 2 / Centos server results in a 'permission > denied' error when Dovecot tries to create files. What exactly is the error message? Copy&paste it entirely. > So, I guess my main question is, what perms and ownership should > /u/indexes be set to? I've tried several different things before this > cry for help... Those permissions look ok to me. It's possible that this was a bug in 2.0.beta6 and upgrading might help (but might not). From tss at iki.fi Tue Nov 15 22:34:21 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:34:21 +0200 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: References: Message-ID: <1321389261.21919.524.camel@hurina> On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > I am familiar with this article > http://wiki.dovecot.org/LDA#multipleuids > > I want to disable setuid-root, but I don't know the default group and > permissions. > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > but my delivery is still failing > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=, > orig_to=, relay=dovecot, delay=0.05, delays=0.04/0/0/0.01, > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: fatal: > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied ) dovecot-lda's group should be the same as you specify to the pipe in Postfix's master.cf. From tss at iki.fi Tue Nov 15 22:35:36 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:35:36 +0200 Subject: [Dovecot] Dovecot SASL on another machine In-Reply-To: <20111105194350.GA3782@state-of-mind.de> References: <8168839159bdf20b93fe8eca132b8ff9@hachmer.de> <1320508907.21919.239.camel@hurina> <20111105194350.GA3782@state-of-mind.de> Message-ID: <1321389336.21919.525.camel@hurina> On Sat, 2011-11-05 at 20:43 +0100, Patrick Ben Koetter wrote: > * Timo Sirainen : > > On Sat, 2011-11-05 at 15:13 +0100, Tobias Hachmer wrote: > > > > > is there a way to use Dovecot SASL Authentication for submission on a > > > different machine dovecot runs? > > > > With v2.0 you should be able to add inet_listener for auth service, > > which acts as an auth-client socket. > > > > > Dovecot machine <-> postfix machine(for submission) > > > > Looking at Postfix v2.8.0 code, looks like Postfix also supports it. > > Document it and Wietse will support it on the mailing list. If its not > documented support will be declined. You mean document in Dovecot wiki or for Postfix? > Should I? Please do :) From tss at iki.fi Tue Nov 15 22:39:13 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:39:13 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC2BE26.3030807@goldey.net> References: <4EC2BE26.3030807@goldey.net> Message-ID: <1321389553.21919.527.camel@hurina> On Tue, 2011-11-15 at 14:31 -0500, Mark Goldey wrote: Could you do a bit more with gdb and this core: > #1 0x08059890 in auth_worker_handle_user (client=0x82b34d0) at > auth-worker-client.c:405 > auth_request = 0x82b56e8 > userdb_id = 1 fr 1 p *auth_request->userdb->userdb p *auth_request->userdb->userdb->iface From tss at iki.fi Tue Nov 15 22:43:24 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 15 Nov 2011 22:43:24 +0200 Subject: [Dovecot] dsync problems In-Reply-To: <871ut9i3z1.fsf@algae.riseup.net> References: <871ut9i3z1.fsf@algae.riseup.net> Message-ID: <1321389804.21919.530.camel@hurina> On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > When a user renames their username, I am using dsync to copy their mail > over to the new username's mail location[0]. > > Some of the dsyncs are failing with errors that I dont know how to work > with, for example: > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a These shouldn't really happen. Something's going internally wrong with dsync. Can you reproduce this reliably somehow? > The errors cause a non-zero exit code from dsync, which causes my rename > script to bail out. What are these errors, and how can I fix them? Does a second dsync on error succeed? :) > 0. Why use dsync instead of a simple mv operation? This seems to be > necessary for two corner cases: > > 1. dovecot creates the new mailbox automatically when the user logs in > or receives a mail, so if the user changes their mail and logs in or > receives an email before the move has been done, then the mailbox is > created and then a move command will fail. > > 2. If there has been new mail created under the new name, we can't just > simply remove the stuff that is automatically created there and replace > it with the old things because we could potentially be removing mail > that has been delivered in the mean time. You could temporarily change the permissions for the home directory so that no new mailboxes/mails could be created during the move (e.g. 0700 root). From mrvjtod at gmail.com Tue Nov 15 23:13:13 2011 From: mrvjtod at gmail.com (Chris Young) Date: Tue, 15 Nov 2011 16:13:13 -0500 Subject: [Dovecot] default Dovecot-LDA CHGRP/CHMOD In-Reply-To: <1321389261.21919.524.camel@hurina> References: <1321389261.21919.524.camel@hurina> Message-ID: Thanks for the response. I was able to get it working with chmod 755 On Tue, Nov 15, 2011 at 3:34 PM, Timo Sirainen wrote: > On Wed, 2011-11-09 at 14:04 -0500, Chris Young wrote: > > I am familiar with this article > > http://wiki.dovecot.org/LDA#multipleuids > > > > I want to disable setuid-root, but I don't know the default group and > > permissions. > > > > sudo chgrp root /usr/libexec/dovecot/dovecot-lda > > sudo chmod 00750 /usr/libexec/dovecot/dovecot-lda > > > > but my delivery is still failing > > > > Nov 9 18:22:39 vl42 postfix/pipe[23039]: DEC54700B1: to=< > user at domain.com>, > > orig_to=, relay=dovecot, delay=0.05, > delays=0.04/0/0/0.01, > > dsn=4.3.0, status=deferred (temporary failure. Command output: pipe: > fatal: > > pipe_command: execvp /usr/libexec/dovecot/dovecot-lda: Permission denied > ) > > dovecot-lda's group should be the same as you specify to the pipe in > Postfix's master.cf. > > > From mysqlstudent at gmail.com Wed Nov 16 00:14:50 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 17:14:50 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321385703.21919.501.camel@hurina> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> I just don't understand how to invoke procmail from deliver. > > The idea was to invoke deliver from procmail. I still do this myself > since I haven't bothered changing my procmail scripts to Sieve: > > DELIVER=/usr/local/libexec/dovecot/deliver > > :0 > * ^List-Id.*dovecot.dovecot.org > | $DELIVER -m dovecot > > :0 > | $DELIVER How do I tell dovecot to use procmail so that it may invoke deliver? Thanks again, Alex From tss at iki.fi Wed Nov 16 00:27:12 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 00:27:12 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 0.14, Alex wrote: >>> I just don't understand how to invoke procmail from deliver. >> >> The idea was to invoke deliver from procmail. I still do this myself >> since I haven't bothered changing my procmail scripts to Sieve: >> >> DELIVER=/usr/local/libexec/dovecot/deliver >> >> :0 >> * ^List-Id.*dovecot.dovecot.org >> | $DELIVER -m dovecot >> >> :0 >> | $DELIVER > > How do I tell dovecot to use procmail so that it may invoke deliver? You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc. From nick.z.edwards at gmail.com Wed Nov 16 00:48:19 2011 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 16 Nov 2011 08:48:19 +1000 Subject: [Dovecot] Excessive imap-logins, never die Message-ID: Hiya, Today I note there are 38 imap-logins that seem to be stale on our staff imap server, they do not seem to die off, new imap-logins are created with new logins and exit accordingly. Now, these are from our monitoring software that runs on another server on our LAN. Our corporate link was taken down for 2 hours for urgent relocation of fibre (demolishing old adjoining building, and just in case, they needed to move it which we were glad about), so, this begs the question, why would imap-logins be there I guess in some sort of wait state, even with a still working internal DNS (internal view as well) the only difference is no actual internet connection in this time. A week ago when they first started to working and we lost internet for 45 minutes I also saw a few extra stale imap-logins, but after todays events, I decided this is a bigger issue that needs to be looked into, as dovecot required a restart to clear it all (I wasnt going to sit there killing individual PIDs all evening). Dovecot version 1.2.17, built from source From tss at iki.fi Wed Nov 16 00:50:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 00:50:25 +0200 Subject: [Dovecot] Excessive imap-logins, never die In-Reply-To: References: Message-ID: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> On 16.11.2011, at 0.48, Nick Edwards wrote: > Today I note there are 38 imap-logins that seem to be stale on our staff > imap server, > they do not seem to die off, new imap-logins are created with new logins > and exit accordingly. .. > Dovecot version 1.2.17, built from source All this code has been rewritten in v2.0 and there's not much point in spending time on figuring out why things aren't working perfectly in v1.x. But you could most likely avoid this problem by switching to "high-performance mode": http://wiki.dovecot.org/LoginProcess From nick.z.edwards at gmail.com Wed Nov 16 01:28:07 2011 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Wed, 16 Nov 2011 09:28:07 +1000 Subject: [Dovecot] Excessive imap-logins, never die In-Reply-To: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> References: <6B095435-9362-407F-9485-8EB7A4674BE6@iki.fi> Message-ID: On Wed, Nov 16, 2011 at 8:50 AM, Timo Sirainen wrote: > On 16.11.2011, at 0.48, Nick Edwards wrote: > > > Today I note there are 38 imap-logins that seem to be stale on our staff > > imap server, > > they do not seem to die off, new imap-logins are created with new logins > > and exit accordingly. > .. > > Dovecot version 1.2.17, built from source > > All this code has been rewritten in v2.0 and there's not much point in > spending time on figuring out why things aren't working perfectly in v1.x. > But you could most likely avoid this problem by switching to > "high-performance mode": http://wiki.dovecot.org/LoginProcess > > Okay, we do use high security mode specifically, and we require SSL on that server, I might just leave it as is, if there is no timeout value for these which I couldn't see when googling earlier today. I want to look at v2, but need to wait until all staff on are Christmas holidays, so if upgrade goes pair shaped, I can restore and not get bombarded with phone calls from management who can't get their mail. From mysqlstudent at gmail.com Wed Nov 16 02:00:13 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 19:00:13 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >>>> I just don't understand how to invoke procmail from deliver. >>> >>> The idea was to invoke deliver from procmail. I still do this myself >>> since I haven't bothered changing my procmail scripts to Sieve: >>> >>> DELIVER=/usr/local/libexec/dovecot/deliver >>> >>> :0 >>> * ^List-Id.*dovecot.dovecot.org >>> | $DELIVER -m dovecot >>> >>> :0 >>> | $DELIVER >> >> How do I tell dovecot to use procmail so that it may invoke deliver? > > You don't. You tell your MTA to invoke procmail, which is what I think you're already doing? So the only change would be to modify your ~/.procmailrc. Okay, I'm making progress. I disabled the virtual_domain stuff in main.cf for postfix and just enabled mail_command=procmail. I think there's something wrong with the deliver command. It's now delivering, but my deliver command must be wrong. This is my ~/.procmailrc: # Log an abstract of all messages LOGABSTRACT=all PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/lib:/usr/local/bin LOGFILE=$HOME/.maillog MAILDIR=$HOME/mail PROCHOME=$HOME/.procmail SENDMAIL=/usr/sbin/sendmail VERBOSE=yes DELIVER=/usr/libexec/dovecot/deliver :0 fhW | $DELIVER -m xspamtest :0 fhw | $DELIVER When a mail is received, it prints this to .maillog: procmail: [23382] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: [23383] Tue Nov 15 18:56:04 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Assigning "LASTFOLDER=/var/spool/mail/alexuser" procmail: Opening "/var/spool/mail/alexuser" procmail: Acquiring kernel-lock procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Locking "/var/spool/mail/alexuser.lock" procmail: Unlocking "/var/spool/mail/alexuser.lock" >From prvs=293a46ea3=mm at mydomain.com Tue Nov 15 18:56:04 2011 Subject: Grand Manor, Corvallis, OR, USA, AUTOMATED Extrane Folder: /var/spool/mail/bcc-user 17025 It delivers to the mail spool, but executing deliver as its printed in the procmailrc: # # /usr/libexec/dovecot/deliver -m spamtest lda(root): Fatal: destination user parameter (-d user) not given Am I doing something wrong? Thanks, Alex From tss at iki.fi Wed Nov 16 02:03:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 02:03:16 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 2.00, Alex wrote: > It's now delivering, but my deliver command must be wrong. This is my > ~/.procmailrc: .. > # # /usr/libexec/dovecot/deliver -m spamtest > lda(root): Fatal: destination user parameter (-d user) not given > > Am I doing something wrong? Is it trying to deliver mails to root user? That's not supported. From moseleymark at gmail.com Wed Nov 16 02:04:46 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 15 Nov 2011 16:04:46 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: <1321385991.21919.503.camel@hurina> References: <1321385991.21919.503.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 11:39 AM, Timo Sirainen wrote: > On Mon, 2011-11-14 at 10:23 -0800, Mark Moseley wrote: > >> > Thanks to a fortuitously unrelated thread ("how to disable quota for >> > second namespace"), I got the quota part figured out and that seems to >> > be working: Add a second entry to plugin {}, e.g. "quota2 = >> > maildir:Archive quota:ns=INBOX.Archives." and add rules for >> > userdb_quota2_rule, userdb_quota2_rule2, etc. >> > >> > My real question now is: Are there any fatal gotchas in this that I'm >> > just not thinking of? >> > >> >> Haven't had a chance to try this large-scale yet. Anybody have any >> thoughts on it? > > The gotcha is that you have two completely independent quotas with > independent usage/limits for the INBOX and Archive namespaces. If that > is what you want, it should all be fine. Nope, that's totally fine. The idea is to put Archive on cheaper (slower) storage and then grant more generous quotas there to make it worth their while to use, without slowing down their Inbox. Another application would be to put their Spam in another namespace (for people who choose to have it put in a separate folder) with a lower quota, again to offload it onto cheaper storage, since hardly anyone actually looks at it. Or is this something that I could be doing more transparently in 2.1 with imapc? From mysqlstudent at gmail.com Wed Nov 16 02:08:34 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 19:08:34 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> It's now delivering, but my deliver command must be wrong. This is my >> ~/.procmailrc: > .. >> # # /usr/libexec/dovecot/deliver -m spamtest >> lda(root): Fatal: destination user parameter (-d user) not given >> >> Am I doing something wrong? > > Is it trying to deliver mails to root user? That's not supported. No, I was just running as root from the command-line for an example. The maillog shows error 75: procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded procmail: Executing "/usr/libexec/dovecot/deliver" procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" procmail: Rescue of unfiltered data succeeded This was for a test user named "alexuser" I created. Thanks again, Alex From tss at iki.fi Wed Nov 16 02:16:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 02:16:33 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: On 16.11.2011, at 2.08, Alex wrote: > No, I was just running as root from the command-line for an example. > The maillog shows error 75: > > procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" > procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" > procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > procmail: Executing "/usr/libexec/dovecot/deliver" > procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" > procmail: Rescue of unfiltered data succeeded > > This was for a test user named "alexuser" I created. deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging From micah at riseup.net Wed Nov 16 03:04:42 2011 From: micah at riseup.net (Micah Anderson) Date: Tue, 15 Nov 2011 20:04:42 -0500 Subject: [Dovecot] Deleting a mailbox and all its children Message-ID: <87pqgsho7p.fsf@algae.riseup.net> I'm looking at scripting a mechanism to delete a mailbox, which is easy with 'doveadm mailbox delete' -- however it gets complicated when there are children of arbitrary depth, for example, I may have this: restored.daily1.INBOX restored.daily1.Sent restored.daily1.Mystuff restored.daily1.Mystuff.foo I would like to delete the 'restored' mailbox and all its children (and unsubscribe the folders), but the 'doveadm mailbox delete' command wont delete any children. I'm surprised that there isn't a -r (recursive) option to 'doveadm mailbox delete', but perhaps I am missing the easier way to handle this? thanks for any ideas or suggestions! micah -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Wed Nov 16 03:15:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 03:15:02 +0200 Subject: [Dovecot] Deleting a mailbox and all its children In-Reply-To: <87pqgsho7p.fsf@algae.riseup.net> References: <87pqgsho7p.fsf@algae.riseup.net> Message-ID: <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> On 16.11.2011, at 3.04, Micah Anderson wrote: > I'm looking at scripting a mechanism to delete a mailbox, which is easy > with 'doveadm mailbox delete' -- however it gets complicated when there > are children of arbitrary depth, for example, I may have this: > > restored.daily1.INBOX > restored.daily1.Sent > restored.daily1.Mystuff > restored.daily1.Mystuff.foo > > I would like to delete the 'restored' mailbox and all its children (and > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > delete any children. > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > mailbox delete', but perhaps I am missing the easier way to handle this? Well, how about something like: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm mailbox delete restored That's what it would have to do internally anyway.. From mysqlstudent at gmail.com Wed Nov 16 03:50:56 2011 From: mysqlstudent at gmail.com (Alex) Date: Tue, 15 Nov 2011 20:50:56 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: Hi, >> procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" >> procmail: Executing "/usr/libexec/dovecot/deliver,-m,xspamtest" >> procmail: Non-zero exitcode (75) from "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> procmail: Executing "/usr/libexec/dovecot/deliver" >> procmail: Program failure (75) of "/usr/libexec/dovecot/deliver" >> procmail: Rescue of unfiltered data succeeded >> >> This was for a test user named "alexuser" I created. > > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging Yes, looks like permissions. I have the following now in syslog: Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): Operation not permitted Where is it trying to write that it can't? Does alexuser need to be a member of group mail? Thanks, Alex From micah at riseup.net Wed Nov 16 04:23:47 2011 From: micah at riseup.net (micah anderson) Date: Tue, 15 Nov 2011 21:23:47 -0500 Subject: [Dovecot] Deleting a mailbox and all its children In-Reply-To: <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> References: <87pqgsho7p.fsf@algae.riseup.net> <2FF8B8F6-3A20-46D6-B7C8-B708BC7BF3BD@iki.fi> Message-ID: <87lirghkjw.fsf@algae.riseup.net> On Wed, 16 Nov 2011 03:15:02 +0200, Timo Sirainen wrote: > On 16.11.2011, at 3.04, Micah Anderson wrote: > > > I'm looking at scripting a mechanism to delete a mailbox, which is easy > > with 'doveadm mailbox delete' -- however it gets complicated when there > > are children of arbitrary depth, for example, I may have this: > > > > restored.daily1.INBOX > > restored.daily1.Sent > > restored.daily1.Mystuff > > restored.daily1.Mystuff.foo > > > > I would like to delete the 'restored' mailbox and all its children (and > > unsubscribe the folders), but the 'doveadm mailbox delete' command wont > > delete any children. > > > > I'm surprised that there isn't a -r (recursive) option to 'doveadm > > mailbox delete', but perhaps I am missing the easier way to handle this? > > Well, how about something like: > > doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete > doveadm mailbox delete restored That wont work because 'restored.daily1.Mystuff.foo' wont get removed before 'restored.daily1.Mystuff' is attempted, resulting in the following: doveadm mailbox list 'restored.*' | xargs -1 doveadm mailbox delete doveadm(micah): Error: Can't delete mailbox restored.daily1.Mystuff: Mailbox has children, delete them first The problem is the subfolders need to be removed first and then walk up the tree. perhaps I can do: doveadm mailbox list 'restored.*' | sort -r | xargs -n1 doveadm mailbox delete doveadm mailbox delete restored although I'm not sure if the reverse sort is going to work in all cases. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From achekalin at lazurit.com Wed Nov 16 08:16:04 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 16 Nov 2011 09:16:04 +0300 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC2AEDE.2080306@hardwarefreak.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> Message-ID: <4EC35524.30803@lazurit.com> Hello, Stan, > This is why I recommended mbox in the first place. If your only writes > to these mailbox files are appends of new messages, mbox is the best > format by far. It's faster at appending than any other format, and it's > faster for searching than any other. I now seriously consider to use mdbox due to its nice self-regulation. After all it I believe mdbox should do file compression on its own, no cron scripts required. > It's an archive. You're not going to use maildir so you don't need > random IOPS performance. Thus RAID5/6 are a much better fit for an > archive as you get better read performance, with more than adequate > write performance, and you use less disks. And as this is an archive, > you don't need real time automatic/transparent compression. Thus I > recommend something like: > > 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled > 2.6.39 or later kernel > 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors > e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 > Specify a strip size of 256KB for the array > Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading > ahead 1024 sectors at a time instead of the default of 256. This > will speed up your searches quite a bit. > 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults > 4. mbox w/zlib plugin. Compress daily files each night with a script > 5. You don't need LVM with a good RAID card (or with mdraid). This > controller can expand the RAID5 up to 8 drives (up to 32 drives max > using SAS expanders) We are considering to get HP DL180G6 server for 8 or 14 drives bays (base model price is somewhat equal, but additional drives adds up cost) with HP Smart Array P410 RAID controller (some servers are equipped with this controller by default) with 256 Mb battery-backed cache, but I'll check your suggestions! What memory size should I plan in the server? You're talking about AMD64 OS image, and 64-bit software are like to consume more memory that 32-bit, so looks like your talking about pretty huge RAM, and I don't believe it's necessary, or maybe I'm wrong? Problem is I have no experience with XFS and not sure I can tune it in the best way, so I'll go with mkfs.xfs defaults, I think. Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins and 2.1 handle that much better, but I don't like the idea of use unstable in production. Thank you for taking your time on my case, yours, Alexander From peter at newton.cx Wed Nov 16 11:00:50 2011 From: peter at newton.cx (Peter Williams) Date: Wed, 16 Nov 2011 01:00:50 -0800 Subject: [Dovecot] Invalid "doveadm search" queries not rejected In-Reply-To: <1321388697.21919.518.camel@hurina> References: <1320997281.2261.116.camel@iota.localdomain> <1321388697.21919.518.camel@hurina> Message-ID: <1321434050.2493.5.camel@iota.localdomain> On Tue, 2011-11-15 at 22:24 +0200, Timo Sirainen wrote: > On Thu, 2011-11-10 at 23:41 -0800, Peter Williams wrote: > > > I think I've encountered a bug in Dovecot 2.0.9 (as provided on CentOS > > 6). > > > > According to the doveadm-search-query(7) manual page, a "mailbox" > > specification can only come at the beginning of a search query. > > It doesn't exactly say that, but yeah, it doesn't also include MAILBOX > and MAILBOX-GUID as search keys. Added now to future versions. Thanks for explaining the situation. As a small improvement I would argue that the manpage was, and still is, a little bit misleading -- here's the bit of text that I was focusing on: Use either the keyword _mailbox_ and the name of the mailbox or the keyword _mailbox-guid_ and the mailbox's globally unique identifier ***at the beginning of the search_query.*** Peter -- Peter Williams / peter at newton.cx From odhiambo at gmail.com Wed Nov 16 11:26:54 2011 From: odhiambo at gmail.com (Odhiambo Washington) Date: Wed, 16 Nov 2011 12:26:54 +0300 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <1320856634.21919.371.camel@hurina> References: <1320791738.21919.350.camel@hurina> <1320856634.21919.371.camel@hurina> Message-ID: On Wed, Nov 9, 2011 at 19:37, Timo Sirainen wrote: > On Wed, 2011-11-09 at 18:34 +0300, Odhiambo Washington wrote: > > I have today tried compiling this -beta, but I saw so many warning about > > linking libraries, which I ignored, > > Probably plugin warnings which you can safely ignore. > > > Making all in wiki > > make: don't know how to make 2b. Stop > > Looks like there's some weird file went into wiki docs, which some makes > don't like. You could just find the line containing "2b" from > doc/wiki/Makefile and delete it. > > That did it. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. Please consider the environment before printing this email. -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 652 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 16:00:38 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 15:00:38 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321386560.21919.509.camel@hurina> References: <1321386560.21919.509.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 8:49 PM, Timo Sirainen wrote: > On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > >> I configured dovecot to use submission smtp host becouse of chroot. >> >> submission_host = 127.0.0.1 >> >> Unfortunatelly: >> >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol >> synchronization error (next input sent too soon: pipelining was not >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next >> input="MAIL FROM:\\r\\n" > > This basically says that Dovecot's SMTP client is being noncompliant, > because Exim doesn't advertise PIPELINING extension. But I'd rather not > add more code to fix this, since PIPELINING gives you a little bit > better performance anyway and you can most likely fix this by modifying > Exim's configs in some way (I can't believe Exim wouldn't support > PIPENING..). > > Thanks for your reply. Indeed exim supports PIPELINING by default: # telnet mail.apisoft.pl 25 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. 220 zeus.apisoft.pl ESMTP Exim 4.76 Wed, 16 Nov 2011 14:55:25 +0100 EHLO test 250-zeus.apisoft.pl Hello localhost [127.0.0.1] 250-SIZE 52428800 250-PIPELINING 250-AUTH LOGIN PLAIN CRAM-MD5 DIGEST-MD5 250-STARTTLS 250 HELP and PIPELINING seems to work fine. But maybe there is a problem with smtp dialog between dovecot and exim. From jester at arcor.de Wed Nov 16 16:02:51 2011 From: jester at arcor.de (Oliver Eales) Date: Wed, 16 Nov 2011 15:02:51 +0100 Subject: [Dovecot] Usage of director without LMTP Message-ID: <4EC3C28B.7040700@arcor.de> Hello, we want to migrate our dovecot 1.2 installation to verison 2.0 and want to use the director feature of version 2. Currently we have a two tier archtiecture, with dovecot proxys and a mailstore togehter with indexes on NFS. (With the common problems of this setup...) For the final setup we want to utilize the director for all protocols (IMAP,POP,LMTP,SIEVE). During the transition phase we will not be able to use LMTP for mail delivery, but we are still stuck with deliver as LDA. So for mail delivery we will not have the fixed user/server assignment, but rather a random assignment. What NFS- and locking settings should be used on the backend servers in this case. And should i use different settings for the host the deliver is executed than the IMAP/POP3 hosts which are accessed through a director proxy. Thanks in advance, Oliver Eales From mh+dovecot at zugschlus.de Wed Nov 16 16:11:25 2011 From: mh+dovecot at zugschlus.de (Marc Haber) Date: Wed, 16 Nov 2011 15:11:25 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321386560.21919.509.camel@hurina> References: <1321386560.21919.509.camel@hurina> Message-ID: <20111116141125.GC4966@torres.zugschlus.de> On Tue, Nov 15, 2011 at 09:49:20PM +0200, Timo Sirainen wrote: > On Sat, 2011-11-12 at 05:32 +0100, Bambero wrote: > > I configured dovecot to use submission smtp host becouse of chroot. > > > > submission_host = 127.0.0.1 > > > > Unfortunatelly: > > > > Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > > synchronization error (next input sent too soon: pipelining was not > > advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > > input="MAIL FROM:\\r\\n" > > This basically says that Dovecot's SMTP client is being noncompliant, > because Exim doesn't advertise PIPELINING extension. But I'd rather not > add more code to fix this, since PIPELINING gives you a little bit > better performance anyway and you can most likely fix this by modifying > Exim's configs in some way (I can't believe Exim wouldn't support > PIPENING..). I have always interpreted the standard in the way that a client MUST NOT assume that the server supports pipelining before it has advertised PIPELINING. Since PIPELINING is only advertised after the client has identified itself as being ESMTP compliant by saying EHLO instead of HELO, I believe that the client MUST wait with his EHLO until the server has shown its banner. Forcing synchronization is a very effective means of spam protection since most spam bots just blast away with EHLO, MAIL FROM without bothering to wait for the server's banner. Greetings Marc -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 31958061 Nordisch by Nature | How to make an American Quilt | Fax: *49 621 31958062 From llonchj at gmail.com Wed Nov 16 16:13:44 2011 From: llonchj at gmail.com (Jordi Llonch) Date: Thu, 17 Nov 2011 01:13:44 +1100 Subject: [Dovecot] Creating a password_scheme Message-ID: Hello, I am trying to develop a password_scheme for the Django passwords. After building and installing the module in $module_dir/auth/password/djangopass.so, also tried libdjangopass.so and libdjangopass_plugin.so, Dovecot does not recognize the scheme: doveadm pw -l CRYPT MD5 MD5-CRYPT SHA SHA1 SHA256 SHA512 SMD5 SSHA SSHA256 SSHA512 PLAIN CLEARTEXT CRAM-MD5 HMAC-MD5 DIGEST-MD5 PLAIN-MD4 PLAIN-MD5 LDAP-MD5 LANMAN NTLM OTP SKEY RPA SHA256-CRYPT SHA512-CRYPT Please can you help me? Thanks -- Jordi Llonch # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.40.3-0.fc15.x86_64 x86_64 Fedora release 15 (Lovelock) ext4 ----------password-scheme-django.c-------------------------- #include #include #include #include #include #include #include #define uchar unsigned char static bool djangopass_verify(const char *plaintext, const char *user, const unsigned char *raw_password, size_t size) { return TRUE; } void djangopass_generate(const char *plaintext, const char *user, const unsigned char **raw_password_r, size_t *size_r) { } struct password_scheme djangopass_scheme = { "DJANGO", PW_ENCODING_BASE64, 0, djangopass_verify, djangopass_generate }; struct module; void djangopass_init(struct module *module) { password_scheme_register(&djangopass_scheme); } void djangopass_deinit(void) { password_scheme_unregister(&djangopass_scheme); } --------------------------------------------------------------------------- From david at blue-labs.org Wed Nov 16 16:14:54 2011 From: david at blue-labs.org (David Ford) Date: Wed, 16 Nov 2011 09:14:54 -0500 Subject: [Dovecot] submission_host problem In-Reply-To: <20111116141125.GC4966@torres.zugschlus.de> References: <1321386560.21919.509.camel@hurina> <20111116141125.GC4966@torres.zugschlus.de> Message-ID: <4EC3C55E.8040401@blue-labs.org> this and several other features are tools i use with tremendous success at battling spam. every MTA connection that violates protocol by making an assumption or posts invalid data for the SMTP phase, gets kicked off with a 421. -david On 11/16/2011 09:11 AM, Marc Haber wrote: > I have always interpreted the standard in the way that a client MUST > NOT assume that the server supports pipelining before it has > advertised PIPELINING. Since PIPELINING is only advertised after the > client has identified itself as being ESMTP compliant by saying EHLO > instead of HELO, I believe that the client MUST wait with his EHLO > until the server has shown its banner. Forcing synchronization is a > very effective means of spam protection since most spam bots just > blast away with EHLO, MAIL FROM without bothering to wait for the > server's banner. Greetings Marc From tss at iki.fi Wed Nov 16 16:37:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 16:37:46 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> Message-ID: <1321454266.21919.545.camel@hurina> On Wed, 2011-11-16 at 15:00 +0100, Bambero wrote: > >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol > >> synchronization error (next input sent too soon: pipelining was not > >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next > >> input="MAIL FROM:\\r\\n" > > > > This basically says that Dovecot's SMTP client is being noncompliant, > > because Exim doesn't advertise PIPELINING extension. But I'd rather not > > add more code to fix this, since PIPELINING gives you a little bit > > better performance anyway and you can most likely fix this by modifying > > Exim's configs in some way (I can't believe Exim wouldn't support > > PIPENING..). > > > > > Thanks for your reply. > Indeed exim supports PIPELINING by default: Oh. Well, see if the attached patch helps? If it does, I'll add it to next version. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 1493 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 16:54:43 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 15:54:43 +0100 Subject: [Dovecot] Chroot In-Reply-To: <1321386260.21919.505.camel@hurina> References: <1321386260.21919.505.camel@hurina> Message-ID: On Tue, Nov 15, 2011 at 8:44 PM, Timo Sirainen wrote: > On Sun, 2011-11-13 at 17:25 +0100, Bambero wrote: > >> all maildirs are located in: /var/mail/jdoe at example.com/Maildir > > So this is /var/mail/%u/Maildir template. > >> namespace { >> ? type = shared >> ? separator = / >> ? prefix = shared/%%u/ >> ? location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u > > You can replace this with: > > location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u > > Although if this is used outside chroot (e.g. lmtp, doveadm) it won't > work correctly.. > >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >> such file or directory > > This is done because you used %%h, to look up home directory from > userdb. If you don't use %%h, it's not done. > Thanks but my db backend returns home dir in chroot like /test at example.com while the realpath with no chroot is: /var/mail/test at example.com So both locations in this case are the same location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u From tss at iki.fi Wed Nov 16 16:59:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 16:59:06 +0200 Subject: [Dovecot] Chroot In-Reply-To: References: <1321386260.21919.505.camel@hurina> Message-ID: <1321455546.21919.553.camel@hurina> On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: > >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No > >> such file or directory > > > > This is done because you used %%h, to look up home directory from > > userdb. If you don't use %%h, it's not done. > > > > Thanks but my db backend returns home dir in chroot like > /test at example.com > > while the realpath with no chroot is: > /var/mail/test at example.com > > So both locations in this case are the same > location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u > location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u Yes, they are the same, but the difference is that with the second one you shouldn't fail with the userdb lookup error. Right? From tss at iki.fi Wed Nov 16 17:02:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 17:02:50 +0200 Subject: [Dovecot] Usage of director without LMTP In-Reply-To: <4EC3C28B.7040700@arcor.de> References: <4EC3C28B.7040700@arcor.de> Message-ID: <1321455770.21919.555.camel@hurina> On Wed, 2011-11-16 at 15:02 +0100, Oliver Eales wrote: > For the final setup we want to utilize the director for all protocols > (IMAP,POP,LMTP,SIEVE). > During the transition phase we will not be able to use LMTP for mail > delivery, but we are still stuck with deliver as LDA. > So for mail delivery we will not have the fixed user/server assignment, > but rather a random assignment. > > What NFS- and locking settings should be used on the backend servers in > this case. > And should i use different settings for the host the deliver is executed > than the IMAP/POP3 hosts which are accessed through a director proxy. All the LDA/IMAP/POP3 servers should use these settings to minimize problems: mail_nfs_storage = yes mail_nfs_index = yes Once you're using LMTP proxying also you can remove those. Otherwise the settings are the same. From leo at strike.wu.ac.at Wed Nov 16 17:04:20 2011 From: leo at strike.wu.ac.at (Alexander 'Leo' Bergolth) Date: Wed, 16 Nov 2011 16:04:20 +0100 Subject: [Dovecot] setting namespaces using script-login environment in 2.0 In-Reply-To: <1320441536.21919.206.camel@hurina> References: <4EAED444.70608@strike.wu.ac.at> <1320441536.21919.206.camel@hurina> Message-ID: <4EC3D0F4.7020204@strike.wu.ac.at> On 11/04/2011 10:18 PM, Timo Sirainen wrote: > On Mon, 2011-10-31 at 18:00 +0100, Alexander 'Leo' Bergolth wrote: >> NAMESPACE_2='maildir:/var/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep' >> NAMESPACE_2_PREFIX=Shared/spamrep/ > > The settings are now named. > http://wiki2.dovecot.org/MailLocation#Custom_namespace_location shows > the basics. So for example if you have: > > namespace foo { > .. > } > > You can change it via: > > NAMESPACE/FOO/PREFIX=Shared/spamrep/ > NAMESPACE/FOO/SEP=/ > ..etc.. > > The '/' character is a bit problematic with shells scripts though, see > the wiki link how to do it via "env" at least. > > Another problem is if you want to add more namespaces than you have > configured. IIRC it's possible to simply add: > > NAMESPACE=foo Thanks a lot, Timo! Changing to named, /-separated environment variables did the job. NAMESPACE/S-SPAMREP/LIST=yes NAMESPACE/S-SPAMREP/LOCATION=maildir:/var/spool/mail/Shared/spamrep:INDEX=~/Maildir/index/Shared/spamrep NAMESPACE/S-SPAMREP/PREFIX=Shared/spamrep/ NAMESPACE/S-SPAMREP/SEPARATOR=/ NAMESPACE/S-SPAMREP/TYPE=public NAMESPACE/S-TESTSHARED/LIST=yes NAMESPACE/S-TESTSHARED/LOCATION=maildir:/var/spool/mail/Shared/testshared:INDEX=~/Maildir/index/Shared/testshared NAMESPACE/S-TESTSHARED/PREFIX=Shared/testshared/ NAMESPACE/S-TESTSHARED/SEPARATOR=/ NAMESPACE=s-testshared s-spamrep NAMESPACE/S-TESTSHARED/TYPE=public USERDB_KEYS=SYSTEM_GROUPS_USER UID GID HOME NAMESPACE/S-SPAMREP/LIST NAMESPACE NAMESPACE/S-TESTSHARED/SEPARATOR NAMESPACE/S-TESTSHARED/TYPE NAMESPACE/S-TESTSHARED/PREFIX NAMESPACE/S-TESTSHARED/LIST NAMESPACE/S-TESTSHARED/LOCATION NAMESPACE/S-SPAMREP/SEPARATOR NAMESPACE/S-SPAMREP/TYPE NAMESPACE/S-SPAMREP/PREFIX NAMESPACE/S-SPAMREP/LOCATION > And more than one is probably either "foo bar" or "foo,bar". If you get > something to work, please add example to > http://wiki2.dovecot.org/PostLoginScripting :) I'll add my script to the Wiki. Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria From tss at iki.fi Wed Nov 16 17:04:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 17:04:42 +0200 Subject: [Dovecot] Creating a password_scheme In-Reply-To: References: Message-ID: <1321455883.21919.556.camel@hurina> On Thu, 2011-11-17 at 01:13 +1100, Jordi Llonch wrote: > I am trying to develop a password_scheme for the Django passwords. > > After building and installing the module in > $module_dir/auth/password/djangopass.so, also tried libdjangopass.so and > libdjangopass_plugin.so, Dovecot does not recognize the scheme: There is no longer a separate password/ directory. Just put it to $module_dir/auth/. Where do you see the password/ directory documented? It should be changed.. From achekalin at lazurit.com Wed Nov 16 17:16:35 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Wed, 16 Nov 2011 18:16:35 +0300 Subject: [Dovecot] How to do SEARCH with doveadm Message-ID: <4EC3D3D3.7000601@lazurit.com> Hello, I'm still playing with search messages from/to specified email address. In fact I need to do that to take such messages and copy it to another mailbox (so that box will contain 'Search results' and will be available to examine by any IMAP client). I've put my mailings in mdbox storage format, and use latest 2.1 beta version of Dovecot to be sure I use the freshest approach in search (and plan to use Solr FTS as well). The setup is each, I use passdb { driver = passwd-file args = /etc/dovecot/auth/%Lu } and userdb { driver = static args = uid=dovecot gid=dovecot home=/MAIL/mail_storage/%Lu } and put user:password pair in file named after my mailbox in /etc/dovecot/auth/ (let's name it test at domain.com), and the messages are at /MAIL/mail_storage/test at domain.com/... While I use it from IMAP client it works nicely. Ok, I put somewhat 5800 messages in INBOX, and IMAP pretty fast in working with it. But as I try to use doveadm to search for messages I need I simple lost my way (doing it under root): # doveadm search ALL doveadm(root): Error: chdir(/root/) failed: Permission denied (euid=104(dovecot) egid=106(dovecot) missing +x perm: /root, dir owned by 0:0 mode=0700) doveadm(root): Error: chdir(/root) failed: Permission denied Whatever cli command I try doveadv try to find messages in /root dispute it is at /MAIL/mail_storage/%Lu. How can I make doveadv use right mail path? In fact, I don't want to use system users to work with mail, it'll be simple virtual mail user, so it is pointless to set up special system user to do that search. I think I lost myself in general concept of how Dovecot handles users and credentials, so please please please push me in right direction! Thank you, Alexander From micah at riseup.net Wed Nov 16 18:09:12 2011 From: micah at riseup.net (micah anderson) Date: Wed, 16 Nov 2011 11:09:12 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <1321388840.21919.520.camel@hurina> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> Message-ID: <87ehx8gic7.fsf@algae.riseup.net> On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen wrote: > On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > > I really like the feature where you can define quota rules with percents > > which trigger off of the default values[0] (so you can set the Trash to > > allow for 10% more of the user's quota for example). > > > > What I would really love in dovecot would be for the ability to > > configure a quota rule for dovecot-lda. I would like to configure things > > so we don't bounce emails for users until they are well over quota, the > > IMAP quota plugin is a really great way to notify people that they are > > over quota because it fails to write to other folders.... that should be > > enough to get people's attention that they need to deal with things, but > > bouncing is harsh. > > You can give LDA-specific quota settings. Like maybe: > > protocol lda { > plugin { > quota_rule2 = INBOX:storage=200% > } > } Aha! Just what I was looking for. I presume the number following 'quota_rule' must be an unique number across all quota_rules in my config? > or even not enforcing quota at all: > > protocol lda { > plugin { > quota = maildir:User quota:noenforcing > } > } I was not aware of the noenforcing option, that answered another question I had not asked yet, so thanks for that! :) micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From bambero at gmail.com Wed Nov 16 18:23:02 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:23:02 +0100 Subject: [Dovecot] Chroot In-Reply-To: <1321455546.21919.553.camel@hurina> References: <1321386260.21919.505.camel@hurina> <1321455546.21919.553.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 3:59 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: >> >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >> >> such file or directory >> > >> > This is done because you used %%h, to look up home directory from >> > userdb. If you don't use %%h, it's not done. >> > >> >> Thanks but my db backend returns home dir in chroot like >> /test at example.com >> >> while the realpath with no chroot is: >> /var/mail/test at example.com >> >> So both locations in this case are the same >> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u >> location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u > > Yes, they are the same, but the difference is that with the second one > you shouldn't fail with the userdb lookup error. Right? > Thanks a lot it almost helped. But now there is another problem: Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX ** test at apisoft.pl R=localuser T=local_delivery: Child process of local_delivery transport (running command "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain") was terminated by signal 11 (Segmentation fault) But it anly appear if user has .sieve filter with mail redirection (uses submission_host) From bambero at gmail.com Wed Nov 16 18:27:13 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:27:13 +0100 Subject: [Dovecot] Chroot In-Reply-To: References: <1321386260.21919.505.camel@hurina> <1321455546.21919.553.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 5:23 PM, Bambero wrote: > On Wed, Nov 16, 2011 at 3:59 PM, Timo Sirainen wrote: >> On Wed, 2011-11-16 at 15:54 +0100, Bambero wrote: >>> >> Error: userdb lookup: connect(/var/dovecot/auth-userdb) failed: No >>> >> such file or directory >>> > >>> > This is done because you used %%h, to look up home directory from >>> > userdb. If you don't use %%h, it's not done. >>> > >>> >>> Thanks but my db backend returns home dir in chroot like >>> /test at example.com >>> >>> while the realpath with no chroot is: >>> /var/mail/test at example.com >>> >>> So both locations in this case are the same >>> location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u >>> location = maildir:/%%u/Maildir:INDEX=~/Maildir/shared/%%u >> >> Yes, they are the same, but the difference is that with the second one >> you shouldn't fail with the userdb lookup error. Right? >> > > Thanks a lot it almost helped. But now there is another problem: > Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX > ** test at apisoft.pl R=localuser T=local_delivery: Child process of > local_delivery transport (running command > "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f > $sender_address -a $original_local_part@$original_domain") was > terminated by signal 11 (Segmentation fault) > > But it anly appear if user has .sieve filter with mail redirection > (uses submission_host) > Sorry wrong thread. I this case you were right now everything work fine. From tss at iki.fi Wed Nov 16 18:27:20 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 18:27:20 +0200 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <87ehx8gic7.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> Message-ID: <1321460847.21919.562.camel@hurina> On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote: > > protocol lda { > > plugin { > > quota_rule2 = INBOX:storage=200% > > } > > } > > Aha! Just what I was looking for. I presume the number following > 'quota_rule' must be an unique number across all quota_rules in my > config? No only unique, but all of the numbers must exist. For example quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10 isn't used, unless quota_rule2..9 also exist. From bambero at gmail.com Wed Nov 16 18:28:16 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 17:28:16 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321454266.21919.545.camel@hurina> References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 3:37 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 15:00 +0100, Bambero wrote: >> >> Nov 12 05:11:15 myhost exim[23366]: 2011-11-12 05:11:15 SMTP protocol >> >> synchronization error (next input sent too soon: pipelining was not >> >> advertised): rejected "EHLO myhost" H=localhost [127.0.0.1] next >> >> input="MAIL FROM:\\r\\n" >> > >> > This basically says that Dovecot's SMTP client is being noncompliant, >> > because Exim doesn't advertise PIPELINING extension. But I'd rather not >> > add more code to fix this, since PIPELINING gives you a little bit >> > better performance anyway and you can most likely fix this by modifying >> > Exim's configs in some way (I can't believe Exim wouldn't support >> > PIPENING..). >> > >> > >> Thanks for your reply. >> Indeed exim supports PIPELINING by default: > > Oh. Well, see if the attached patch helps? If it does, I'll add it to > next version. > > Thanks a lot it almost helped. But now there is another problem: Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX ** test at apisoft.pl R=localuser T=local_delivery: Child process of local_delivery transport (running command "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f $sender_address -a $original_local_part@$original_domain") was terminated by signal 11 (Segmentation fault) But it anly appear if user has .sieve filter with mail redirection (uses submission_host) From micah at riseup.net Wed Nov 16 18:31:01 2011 From: micah at riseup.net (micah anderson) Date: Wed, 16 Nov 2011 11:31:01 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <1321460847.21919.562.camel@hurina> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> <1321460847.21919.562.camel@hurina> Message-ID: <87ty64f2re.fsf@algae.riseup.net> On Wed, 16 Nov 2011 18:27:20 +0200, Timo Sirainen wrote: > On Wed, 2011-11-16 at 11:09 -0500, micah anderson wrote: > > > protocol lda { > > > plugin { > > > quota_rule2 = INBOX:storage=200% > > > } > > > } > > > > Aha! Just what I was looking for. I presume the number following > > 'quota_rule' must be an unique number across all quota_rules in my > > config? > > No only unique, but all of the numbers must exist. For example > quota_rule2 isn't used if quota_rule doesn't exist. Or quota_rule10 > isn't used, unless quota_rule2..9 also exist. Ah, interesting, good to know. I presume that 'quota_rule' stands in for 'quota_rule1' and one does not need to specify a quota_rule1 if one has a 'quota_rule'. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From stan at hardwarefreak.com Wed Nov 16 18:36:12 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 16 Nov 2011 10:36:12 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <4EC35501.6050601@lazurit.com> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> <4EC35501.6050601@lazurit.com> Message-ID: <4EC3E67C.4050107@hardwarefreak.com> On 11/16/2011 12:15 AM, Alexander Chekalin wrote: > Hello, Stan, > >> This is why I recommended mbox in the first place. If your only writes >> to these mailbox files are appends of new messages, mbox is the best >> format by far. It's faster at appending than any other format, and it's >> faster for searching than any other. > > I now seriously consider to use mdbox due to its nice self-regulation. > After all it I believe mdbox should do file compression on its own, no > cron scripts required. mbox and mdbox each has strengths and weaknesses. mbox will compress with a higher ratio than mdbox. You already have a nightly script that moves all mail from the day into a new file. Piping that through gzip or bzip2 is a no brainer. It'll add one line to your existing script, if that. Dovecot will decompress the file transparently when you access it via IMAP. And again since it's a single file searching it is much faster. With mbox you will have a single file for each day of emails. This seems ideal for archive purposes, one file per day. mdbox does fully transparent de/compression which is nice. The downside is that Dovecot does dbox compression on a per email basis, not a per file basis. So your compression ratio will be much less than with mbox, especially with bzip2 which works best on files over 900KB in size. Most emails are less than 8KB. Using mdbox will yield multiple files per day of emails instead of just one. Either format is much better than maildir for archiving. >> It's an archive. You're not going to use maildir so you don't need >> random IOPS performance. Thus RAID5/6 are a much better fit for an >> archive as you get better read performance, with more than adequate >> write performance, and you use less disks. And as this is an archive, >> you don't need real time automatic/transparent compression. Thus I >> recommend something like: >> >> 1. Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled >> 2.6.39 or later kernel >> 2. hardware RAID5 w/large (2TB) SATA disks, 512B native sectors >> e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011 >> Specify a strip size of 256KB for the array >> Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading >> ahead 1024 sectors at a time instead of the default of 256. This >> will speed up your searches quite a bit. >> 3. XFS filesystem on the RAID device, created with mkfs.xfs defaults >> 4. mbox w/zlib plugin. Compress daily files each night with a script >> 5. You don't need LVM with a good RAID card (or with mdraid). This >> controller can expand the RAID5 up to 8 drives (up to 32 drives max >> using SAS expanders) > > We are considering to get HP DL180G6 server for 8 or 14 drives bays The P410 tops out at 8 drives, so get the 8 drive model. Start with 4 x 2TB drives in RAID5. Add 4 more drives when you need the capacity, and when drive prices are back down to normal (see below). http://h18004.www1.hp.com/products/quickspecs/13248_na/13248_na.html > (base model price is somewhat equal, but additional drives adds up cost) Especially right now in 2011. Flooding in Thailand, where 25% of the world's drives are produced, has doubled the cost of all hard drives worldwide. Now is a horrible time to buy spinning drives. I've read it may be 12 months before prices start coming back down... > with HP Smart Array P410 RAID controller (some servers are equipped with > this controller by default) with 256 Mb battery-backed cache, but I'll > check your suggestions! The P410 should be fine for a dedicated archive server. > What memory size should I plan in the server? You're talking about AMD64 > OS image, and 64-bit software are like to consume more memory that > 32-bit, so looks like your talking about pretty huge RAM, and I don't > believe it's necessary, or maybe I'm wrong? The memory footprint of 64bit binaries is nothing to worry about. The additional amount consumed is more than offset by the performance gained with direct access to RAM above 4GB compared to the performance of PAE. Keep in mind that 90% of your memory will be eaten by Linux buffer cache. Your binaries will account for less than 5% of your RAM consumption. If I understand correctly how you will use this archive server, then 8GB should be plenty. 8GB is standard on the 8 drive DL180 G6. http://h18004.www1.hp.com/products/quickspecs/13248_na/13248_na.html > Problem is I have no experience with XFS and not sure I can tune it in > the best way, so I'll go with mkfs.xfs defaults, I think. With only 4 drives and using a P410 w/cache and RAID5, doing manual XFS tuning isn't necessary for good performance, especially for an archive application which is data heavy, not metadata heavy. Setting sunit/swidth to match the RAID5 layout may increase performance slightly due to stripe aligned writes, but not enough that I'd worry about it. Just use the mkfs.xfs defaults. If you get the BBWC for the P410, enable the controller write cache, and mount XFS with 'nobarrier'. This will increase write performance quite a bit as fsyncs will complete instantly. > Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins > and 2.1 handle that much better, but I don't like the idea of use > unstable in production. Me neither. Speaking of archive/search, did you take look at Enkive yet? http://www.enkive.org/ > Thank you for taking your time on my case, You're welcome Alexander. -- Stan P.S. You may wish to implement dnswl.org ;) From tss at iki.fi Wed Nov 16 18:38:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 18:38:07 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> Message-ID: <1321461498.21919.566.camel@hurina> On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: > Thanks a lot it almost helped. But now there is another problem: > Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX > ** test at apisoft.pl R=localuser T=local_delivery: Child process of > local_delivery transport (running command > "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain -f > $sender_address -a $original_local_part@$original_domain") was > terminated by signal 11 (Segmentation fault) > > But it anly appear if user has .sieve filter with mail redirection > (uses submission_host) It's difficult to debug crashes without gdb backtrace. But since you're running in a strange chrooted environment, it may be difficult to get a core file.. Probably the easiest way would be if you apply the attached patch, then try to deliver a mail that causes this crash (and no other mails), and attach gdb into the process during the 10 second wait: gdb -p `pidof dovecot-lda` cont bt full -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 286 bytes Desc: not available URL: From tss at iki.fi Wed Nov 16 20:02:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:02:18 +0200 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <4EBACFC0.6060703@gmx.de> References: <4EB6AFE3.3020001@gmx.de> <4EBACFC0.6060703@gmx.de> Message-ID: <1321466541.21919.576.camel@hurina> On Wed, 2011-11-09 at 20:08 +0100, e-frog wrote: > On 06.11.2011 17:03, wrote e-frog: > > 1 lsub "" "virtual/*" > > * LSUB () "/" "virtual" > > * LSUB () "/" "virtual/Lists" > > * LSUB () "/" "virtual/unread" > > 1 OK Lsub completed. > > Just tested this again with 2.1beta1 with the same result. Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/16024d03f66e From tss at iki.fi Wed Nov 16 20:15:44 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:15:44 +0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EB72177.8090801@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> Message-ID: <1321467345.21919.580.camel@hurina> On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: > Ed W schrieb: > > > See the "sticky" in my reply. You use one of several techniques to > > ensure that users always end up on the server with the indexes on. That > > way much of the IO is served from that local machine and you only access > > the SAN for the (in theory much less frequent) access to the mail files > > themselves. > > I know you can afford that (IMAP) users always end up on one particular > server but afaik this only works for incoming IMAP connections. > > My mail exchangers use dovecot-lda and I think indexes will be written > from these servers too or am I wrong with this? You can use LMTP and LMTP proxying. From tss at iki.fi Wed Nov 16 20:18:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:18:55 +0200 Subject: [Dovecot] Missing public folder In-Reply-To: <201111071110.20143.komodo@uvt.cz> References: <201111071110.20143.komodo@uvt.cz> Message-ID: <1321467535.21919.582.camel@hurina> On Mon, 2011-11-07 at 11:10 +0100, komodo wrote: > Hi > > i am using dovecot 2.0.13 and i can't see public folder in my mail client. But what is strange, it > was working after instalation, and i don't know when it dissapeared. Maybe after some upgrade ? Try talking IMAP protocol directly: telnet localhost 143 a login user pass b list "" * c select spam_learner > namespace { > location = maildir:/home/shared/Maildir Is this the "spam_learner" mailbox itself alone, or does it have children? ("spam_learner/spam"?) From tss at iki.fi Wed Nov 16 20:25:29 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:25:29 +0200 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <4EB7BF9A.9020900@webfusion.com> References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> <4EB7BF9A.9020900@webfusion.com> Message-ID: <1321467929.21919.583.camel@hurina> On Mon, 2011-11-07 at 13:23 +0200, Mark Zealey wrote: > 04-11-2011 23:56, Timo Sirainen yazm??: > From my reading of the code in 2.0.15, you pass the local & remote IP > in lmtp/commands.c:427 but the struct mail_storage_service_input only > has remote_ip and local_ip fields, not port fields (unlike the > auth_user_info struct which has both ip's and port entries as well). I > think that the mail_storage_service_input struct needs the uint > local_port, remote_port fields adding in & appropriate code changes to > pass these through? Yeah, you're right, I missed that one. Added to hg now. From tss at iki.fi Wed Nov 16 20:31:15 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:31:15 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC3FDB5.7090301@goldey.net> References: <4EC2BE26.3030807@goldey.net> <4EC3FDB5.7090301@goldey.net> Message-ID: <1321468275.21919.586.camel@hurina> Your message was discarded by mailman, probably because it was html-only. Anyway: On Wed, 2011-11-16 at 13:15 -0500, Mark Goldey wrote: > (gdb) p *auth_request->userdb->userdb->iface > $2 = {name = 0x807023e "vpopmail", preinit = 0x8065d60 > , init = 0, deinit = 0, > lookup = 0x8066010 , iterate_init = 0, iterate_next > = 0, iterate_deinit = 0} OK, so "auth_request->userdb->userdb->iface->lookup" should call vpopmail_lookup(), it's not NULL as this says: > > (gdb) bt full > > #0 0x00000000 in ?? () So something's going strangely wrong. Maybe gcc optimizations cause this and the real problem happens later. Try compiling auth process with optimizations: - Remove -O2 from src/auth/Makefile - cd src/auth ; make clean ; make ; make install Is the backtrace different? From tss at iki.fi Wed Nov 16 20:33:27 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:33:27 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: <1321468407.21919.588.camel@hurina> On Tue, 2011-11-15 at 20:50 -0500, Alex wrote: > > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging > > Yes, looks like permissions. I have the following now in syslog: > > Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: > setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): > Operation not permitted > Where is it trying to write that it can't? It's trying to change the process's UID/GID to mail/mail for some reason. Have you set "mail_uid=mail" or something? > Does alexuser need to be a member of group mail? No. From tss at iki.fi Wed Nov 16 20:34:14 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 20:34:14 +0200 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: References: <1321385991.21919.503.camel@hurina> Message-ID: <1321468455.21919.589.camel@hurina> On Tue, 2011-11-15 at 16:04 -0800, Mark Moseley wrote: > > The gotcha is that you have two completely independent quotas with > > independent usage/limits for the INBOX and Archive namespaces. If that > > is what you want, it should all be fine. > > Nope, that's totally fine. The idea is to put Archive on cheaper > (slower) storage and then grant more generous quotas there to make it > worth their while to use, without slowing down their Inbox. Another > application would be to put their Spam in another namespace (for > people who choose to have it put in a separate folder) with a lower > quota, again to offload it onto cheaper storage, since hardly anyone > actually looks at it. Should be fine then. > Or is this something that I could be doing more transparently in 2.1 with imapc? I don't really see how that could help. From moseleymark at gmail.com Wed Nov 16 20:40:45 2011 From: moseleymark at gmail.com (Mark Moseley) Date: Wed, 16 Nov 2011 10:40:45 -0800 Subject: [Dovecot] Glued-together private namespaces In-Reply-To: <1321468455.21919.589.camel@hurina> References: <1321385991.21919.503.camel@hurina> <1321468455.21919.589.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 10:34 AM, Timo Sirainen wrote: > On Tue, 2011-11-15 at 16:04 -0800, Mark Moseley wrote: >> > The gotcha is that you have two completely independent quotas with >> > independent usage/limits for the INBOX and Archive namespaces. If that >> > is what you want, it should all be fine. >> >> Nope, that's totally fine. The idea is to put Archive on cheaper >> (slower) storage and then grant more generous quotas there to make it >> worth their while to use, without slowing down their Inbox. Another >> application would be to put their Spam in another namespace (for >> people who choose to have it put in a separate folder) with a lower >> quota, again to offload it onto cheaper storage, since hardly anyone >> actually looks at it. > > Should be fine then. > >> Or is this something that I could be doing more transparently in 2.1 with imapc? > > I don't really see how that could help. Ah, bummer. I thought maybe 2.1 could proxy to a separate folder or namespace (but I've also barely had a chance to look at it), like so certain folders would be grabbed from a proxy. Haven't really thought that through though :) From pw at wk-serv.de Wed Nov 16 20:40:50 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Wed, 16 Nov 2011 19:40:50 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <1321467345.21919.580.camel@hurina> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> Message-ID: <4EC403B2.60005@wk-serv.de> Timo Sirainen schrieb: > On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: >> >> My mail exchangers use dovecot-lda and I think indexes will be written >> from these servers too or am I wrong with this? > > You can use LMTP and LMTP proxying. I already use lmtp:unix:private/dovecot-lmtp as transport but where is the link to the indexes? From bambero at gmail.com Wed Nov 16 21:25:10 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 20:25:10 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: <1321461498.21919.566.camel@hurina> References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 5:38 PM, Timo Sirainen wrote: > On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: >> Thanks a lot it almost helped. But now there is another problem: >> Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX >> ** test at apisoft.pl R=localuser T=local_delivery: Child process of >> local_delivery transport (running command >> "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f >> $sender_address -a $original_local_part@$original_domain") was >> terminated by signal 11 (Segmentation fault) >> >> ?But it anly appear if user has .sieve filter with mail redirection >> (uses submission_host) > > It's difficult to debug crashes without gdb backtrace. But since you're > running in a strange chrooted environment, it may be difficult to get a > core file.. > > Probably the easiest way would be if you apply the attached patch, then > try to deliver a mail that causes this crash (and no other mails), and > attach gdb into the process during the 10 second wait: > > gdb -p `pidof dovecot-lda` > cont > > bt full > > Can you see something here: gdb -p `pgrep dovecot-lda` GNU gdb 6.3 Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-openbsd5.0". Attaching to process 4118 0x030e6719 in ?? () (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. 0x0c96c825 in ?? () (gdb) bt full #0 0x0c96c825 in ?? () No symbol table info available. #1 0x7dfa61c0 in ?? () No symbol table info available. #2 0x00000001 in ?? () No symbol table info available. #3 0x00000001 in ?? () No symbol table info available. #4 0x00000002 in ?? () No symbol table info available. #5 0x2665d31c in ?? () No symbol table info available. #6 0x2665d31c in ?? () No symbol table info available. #7 0xcfbe13e8 in ?? () No symbol table info available. #8 0x0665fd79 in ?? () No symbol table info available. #9 0x00000000 in ?? () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: , process 4118 From bambero at gmail.com Wed Nov 16 21:37:32 2011 From: bambero at gmail.com (Bambero) Date: Wed, 16 Nov 2011 20:37:32 +0100 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: On Wed, Nov 16, 2011 at 8:25 PM, Bambero wrote: > On Wed, Nov 16, 2011 at 5:38 PM, Timo Sirainen wrote: >> On Wed, 2011-11-16 at 17:28 +0100, Bambero wrote: >>> Thanks a lot it almost helped. But now there is another problem: >>> Nov 16 17:13:45 zeus exim[10613]: 2011-11-16 17:13:45 1RQi7C-0001lG-BX >>> ** test at apisoft.pl R=localuser T=local_delivery: Child process of >>> local_delivery transport (running command >>> "/usr/local/libexec/dovecot/dovecot-lda -d $local_part@$domain ?-f >>> $sender_address -a $original_local_part@$original_domain") was >>> terminated by signal 11 (Segmentation fault) >>> >>> ?But it anly appear if user has .sieve filter with mail redirection >>> (uses submission_host) >> >> It's difficult to debug crashes without gdb backtrace. But since you're >> running in a strange chrooted environment, it may be difficult to get a >> core file.. >> >> Probably the easiest way would be if you apply the attached patch, then >> try to deliver a mail that causes this crash (and no other mails), and >> attach gdb into the process during the 10 second wait: >> >> gdb -p `pidof dovecot-lda` >> cont >> >> bt full >> >> > > Can you see something here: > > gdb -p `pgrep dovecot-lda` > GNU gdb 6.3 > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. ?Type "show warranty" for details. > This GDB was configured as "i386-unknown-openbsd5.0". > Attaching to process 4118 > 0x030e6719 in ?? () > (gdb) cont > Continuing. > > Program received signal SIGSEGV, Segmentation fault. > 0x0c96c825 in ?? () > (gdb) bt full > #0 ?0x0c96c825 in ?? () > No symbol table info available. > #1 ?0x7dfa61c0 in ?? () > No symbol table info available. > #2 ?0x00000001 in ?? () > No symbol table info available. > #3 ?0x00000001 in ?? () > No symbol table info available. > #4 ?0x00000002 in ?? () > No symbol table info available. > #5 ?0x2665d31c in ?? () > No symbol table info available. > #6 ?0x2665d31c in ?? () > No symbol table info available. > #7 ?0xcfbe13e8 in ?? () > No symbol table info available. > #8 ?0x0665fd79 in ?? () > No symbol table info available. > #9 ?0x00000000 in ?? () > No symbol table info available. > (gdb) quit > The program is running. ?Quit anyway (and detach it)? (y or n) y > Detaching from program: , process 4118 > I forgot to say that mails are delivered using SMTP so submission_host works, mails are strored i destination mailbox too. Only this Segmentation Fault ... From tss at iki.fi Wed Nov 16 21:45:34 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 21:45:34 +0200 Subject: [Dovecot] submission_host problem In-Reply-To: References: <1321386560.21919.509.camel@hurina> <1321454266.21919.545.camel@hurina> <1321461498.21919.566.camel@hurina> Message-ID: <1321472737.21919.596.camel@hurina> On Wed, 2011-11-16 at 20:25 +0100, Bambero wrote: > Can you see something here: No, the "??" isn't very helpful. > gdb -p `pgrep dovecot-lda` > This GDB was configured as "i386-unknown-openbsd5.0". Oh, this is OpenBSD. Try if this works instead: gdb /usr/local/libexec/dovecot/dovecot-lda `pgrep dovecot-lda` If it still gives only "??" output, that didn't work either.. From tss at iki.fi Wed Nov 16 21:47:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 16 Nov 2011 21:47:17 +0200 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4EC403B2.60005@wk-serv.de> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> Message-ID: <1321472837.21919.598.camel@hurina> On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: > Timo Sirainen schrieb: > > On Mon, 2011-11-07 at 01:08 +0100, Patrick Westenberg wrote: > >> > >> My mail exchangers use dovecot-lda and I think indexes will be written > >> from these servers too or am I wrong with this? > > > > You can use LMTP and LMTP proxying. > > I already use lmtp:unix:private/dovecot-lmtp as transport but where is > the link to the indexes? You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot LMTP proxy, which would forward the connection to the backend server which handles that user's IMAP/POP3/LMTP connections. From guenther at rudersport.de Wed Nov 16 22:08:40 2011 From: guenther at rudersport.de (Karsten =?ISO-8859-1?Q?Br=E4ckelmann?=) Date: Wed, 16 Nov 2011 21:08:40 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> Message-ID: <1321474120.4854.13.camel@monkey> On Tue, 2011-11-15 at 19:00 -0500, Alex wrote: > DELIVER=/usr/libexec/dovecot/deliver > > :0 fhW > | $DELIVER -m xspamtest Do you really want the pipe to be a (f)ilter? What do you expect deliver to pass back? And you're feeding deliver the mail (h)eaders only, dropping the body. -- char *t="\10pse\0r\0dtu\0. at ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} From sb at dod.no Thu Nov 17 00:45:28 2011 From: sb at dod.no (Steinar Bang) Date: Wed, 16 Nov 2011 23:45:28 +0100 Subject: [Dovecot] Bus factor of dovecot / GIT References: <201111091029.37963.thomas@koch.ro> <1320846789.21919.361.camel@hurina> Message-ID: <87sjlnr8jb.fsf@dod.no> >>>>> Timo Sirainen : > I'm not as much against git anymore as I was when I switched to hg, > but I don't see much benefits in switching to git either. I highly > doubt I'd get even a single patch more if I used git instead of hg. FWIW the distingushing feature of git isn't that it is a ditributed VCS. It's the ease with which it does branching and merging. What put me off git for a long time, was Linus' arrogance wrt. to existing version control systems. But after using git, I have to admit that it is... quite clever. Especially the branching and merging stuff. From tss at iki.fi Thu Nov 17 01:30:42 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 01:30:42 +0200 Subject: [Dovecot] v2.0.16 released Message-ID: <1321486243.21919.620.camel@hurina> http://dovecot.org/releases/2.0/dovecot-2.0.16.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.16.tar.gz.sig I only now noticed that the VSZ limits weren't being enforced with earlier v2.0.x releases (or they were set 1024 times too high). So if a Dovecot process was leaking memory, it wasn't being killed by kernel. Now that this enforcing is done, some installations will probably start seeing errors about reaching these limits in normal operation. The default_vsz_limit is 256 MB. You may want to increase it in larger installations to 1 GB just in case. Besides the changes listed below, a lot of smaller fixes were done. v2.0.16 2011-11-17 Timo Sirainen * VSZ limits weren't being enforced for any processes. On server with large mailboxes you may now see errors about it if the limits aren't high enough. To fix them, either increase individual service { vsz_limit } values or simply increase the default_vsz_limit setting. * Proxying: If using ssl=yes or starttls=yes with a hostname (not IP) as proxy destination, require that the certificate matches the given hostname. * LMTP: Changed default client_limit to 1. This should improve LMTP throughput with default settings. * dsync: Quota is no longer enforced (i.e. dsync can't fail because user is over quota). + Added "auto" mail storage driver, which can be used to autodetect mailbox location and format. This behavior is already the default for empty mail_location setting, so this change is mainly useful for shared namespace's location setting. + checkpassword: Export all auth %variables to AUTH_* environment. From mysqlstudent at gmail.com Thu Nov 17 02:29:36 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:29:36 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321468407.21919.588.camel@hurina> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> Message-ID: Hi, >> > deliver should have logged something, or alternatively the problem was exactly that it couldn't log anything. http://wiki.dovecot.org/LDA#logging >> >> Yes, looks like permissions. I have the following now in syslog: >> >> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >> Operation not permitted > >> Where is it trying to write that it can't? > > It's trying to change the process's UID/GID to mail/mail for some > reason. Have you set "mail_uid=mail" or something? mail_uid isn't set, but I have the following in dovecot.conf: mail_gid = mail mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_privileged_group = mail Is that not correct? Thanks, Alex From tss at iki.fi Thu Nov 17 02:34:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 02:34:31 +0200 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> Message-ID: <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> On 17.11.2011, at 2.29, Alex wrote: >>> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >>> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >>> Operation not permitted >> >>> Where is it trying to write that it can't? >> >> It's trying to change the process's UID/GID to mail/mail for some >> reason. Have you set "mail_uid=mail" or something? > > mail_uid isn't set, but I have the following in dovecot.conf: > > mail_gid = mail The error most likely happens because of this. Is there a reason why you want this? > mail_privileged_group = mail This should be enough for creating lock files to /var/spool/mail/. Although dovecot-lda can't use this setting, since it's not started as root.. From mysqlstudent at gmail.com Thu Nov 17 02:49:53 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:49:53 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: Hi, >>>> Nov 15 19:51:07 mail02 dovecot: lda(alexuser): Fatal: >>>> setresgid(12(mail),12(mail),12(mail)) failed with euid=500(alexuser): >>>> Operation not permitted >>> >>>> Where is it trying to write that it can't? >>> >>> It's trying to change the process's UID/GID to mail/mail for some >>> reason. Have you set "mail_uid=mail" or something? >> >> mail_uid isn't set, but I have the following in dovecot.conf: >> >> mail_gid = mail > > The error most likely happens because of this. Is there a reason why you want this? > >> mail_privileged_group = mail > > This should be enough for creating lock files to /var/spool/mail/. Although dovecot-lda can't use this setting, since it's not started as root.. Okay, that fixed it. It's now delivering mail properly, thanks. Just to be clear I understand, if I use the following .procmailrc, it should save all messages with the X-Spam-Status header in xspamstatus, and everything else gets delivered to /var/spool/mail/user, correct? DELIVER=/usr/libexec/dovecot/deliver :0 * ^X-Spam-Status | $DELIVER -m xspamstatus :0 | $DELIVER This is just a simple test which appears to work the way I outlined, but wanted to be sure. Thanks again for sticking with me. Best, Alex From mysqlstudent at gmail.com Thu Nov 17 02:56:37 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:56:37 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321474120.4854.13.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321474120.4854.13.camel@monkey> Message-ID: Hi, >> DELIVER=/usr/libexec/dovecot/deliver >> >> :0 fhW >> | $DELIVER -m xspamtest > > Do you really want the pipe to be a (f)ilter? What do you expect deliver > to pass back? And you're feeding deliver the mail (h)eaders only, > dropping the body. Yes, thanks. I pasted from another filter we were using, and forgot. procmail still seems to be much better than the deliver sieve language, despite how long it's been since I've used it :-) Thanks, Alex From mysqlstudent at gmail.com Thu Nov 17 02:57:12 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 19:57:12 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321474120.4854.13.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321474120.4854.13.camel@monkey> Message-ID: Hi, >> DELIVER=/usr/libexec/dovecot/deliver >> >> :0 fhW >> | $DELIVER -m xspamtest > > Do you really want the pipe to be a (f)ilter? What do you expect deliver > to pass back? And you're feeding deliver the mail (h)eaders only, > dropping the body. Yes, thanks. I pasted from another filter we were using, and forgot. procmail still seems to be much better than the deliver sieve language, despite how long it's been since I've used it :-) Thanks, Alex From mysqlstudent at gmail.com Thu Nov 17 03:42:04 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 20:42:04 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: Hi, My apologies if a previous message appeared twice. Not sure how that happened... > Okay, that fixed it. It's now delivering mail properly, thanks. > > Just to be clear I understand, if I use the following .procmailrc, it > should save all messages with the X-Spam-Status header in xspamstatus, > and everything else gets delivered to /var/spool/mail/user, correct? > > DELIVER=/usr/libexec/dovecot/deliver > > :0 > * ^X-Spam-Status > | $DELIVER -m xspamstatus I thought this was working, but it doesn't create or populate a folder named xspamstatus although it executes deliver: procmail: [30333] Wed Nov 16 20:39:36 2011 procmail: Assigning "COMSAT=no" procmail: Assigning "COMSAT=no" procmail: Assigning "DELIVER=/usr/libexec/dovecot/deliver" procmail: Match on "^X-Spam-Status" procmail: Assigning "LASTFOLDER=/usr/libexec/dovecot/deliver -m xspamstatus" >From smarchant at mydomain.com.au Wed Nov 16 20:39:36 2011 procmail: Executing "/usr/libexec/dovecot/deliver,-m,mymailbox" Subject: Riverside (97293) Folder: /usr/libexec/dovecot/deliver -m mymailbox 28400 I've set lda_mailbox_autocreate to yes, but it still doesn't create it. Is this folder relative to ~user/mail/ ? Is there an index file that dovecot-lda updates for imap? Can I just eliminate it entirely and just have procmail do all the delivery? Thanks again, Alex From sf.rique at gmail.com Thu Nov 17 03:53:29 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Wed, 16 Nov 2011 23:53:29 -0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <4EC16116.1000902@hardwarefreak.com> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> Message-ID: Jan-Frode Wich clustered filesytem do you have? I used to have ocfs2 but had problems with performance. So had to get back to ext4 and it solve the performance problem... My ocfs2 setup had some problems... but still.. Some numbers: OCFS2 1TB of maildir files. Full backup 36 Hours Incremental 15 hours Ext4 1TB of maildir files. Full backup 16 Hours Incremental 1 hour Same LUN on storage. []'sf.rique On Mon, Nov 14, 2011 at 4:42 PM, Stan Hoeppner wrote: > On 11/14/2011 4:27 AM, Jan-Frode Myklebust wrote: > > > Agree. A non-clustered fs should give you better performance, and > > probably also be more reliable, if you can live with the SPoF and > > full downtime during patching/upgrades/maintenance. But I would expect > > xfs to be a better choice than ext*. > > Depends on the workload characteristics and how well the XFS filesystem > is tuned to the storage hardware. If setup properly, using many > allocation groups with fast spindles, a decent amount of BBWC, and a > high concurrency maildir workload (dozens to hundreds of delivery and > IMAP operations), XFS will runs circles around EXTx as it can > create/write/read to every AG in parallel. Much of EXT4's operation is > still serialized. This is why XFS outruns all other filesystems in the > highly parallel mail workload benchmarks I posted previously, EXTx by a > factor of 2-3. > > For smaller hosts that don't see parallelism, for example SOHO servers, > XFS will likely be slower than EXTx as the workload will be serialized. > > -- > Stan > From guenther at rudersport.de Thu Nov 17 04:18:38 2011 From: guenther at rudersport.de (Karsten =?ISO-8859-1?Q?Br=E4ckelmann?=) Date: Thu, 17 Nov 2011 03:18:38 +0100 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> Message-ID: <1321496318.4854.62.camel@monkey> On Wed, 2011-11-16 at 20:42 -0500, Alex wrote: [...] Unrelated to other dovecot specific questions... > Is there an index file that dovecot-lda updates for imap? Yes. Which would be the advantage of using dovecot deliver, directly or called from procmail. > Can I just eliminate it entirely and just have procmail do all the > delivery? Yes, you can. In fact, that's what I usually still use. Procmail can just do much more than sieve. And procmail doesn't scare me as much as sieve. But then again, I like Perl... And I've never yet encountered a problem with dovecot IMAP updating indexes on the fly -- which it does, unless deliver does it incrementally. However, wasn't your original question about converting procmail recipes to sieve? (Yes, it was.) So what would hold you back of just not converting? -- char *t="\10pse\0r\0dtu\0. at ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} From mysqlstudent at gmail.com Thu Nov 17 04:50:52 2011 From: mysqlstudent at gmail.com (Alex) Date: Wed, 16 Nov 2011 21:50:52 -0500 Subject: [Dovecot] Getting started with sieve and conversion from procmail In-Reply-To: <1321496318.4854.62.camel@monkey> References: <4EC217B1.2040809@rename-it.nl> <4EC297FC.8050405@rename-it.nl> <1321385703.21919.501.camel@hurina> <1321468407.21919.588.camel@hurina> <63BE83DD-B953-4B91-A4A4-A59DB72400AE@iki.fi> <1321496318.4854.62.camel@monkey> Message-ID: > Unrelated to other dovecot specific questions... > >> Is there an index file that dovecot-lda updates for imap? > > Yes. Which would be the advantage of using dovecot deliver, directly or > called from procmail. > >> Can I just eliminate it entirely and just have procmail do all the >> delivery? > > Yes, you can. > > In fact, that's what I usually still use. Procmail can just do much more > than sieve. And procmail doesn't scare me as much as sieve. But then > again, I like Perl... And I've never yet encountered a problem with > dovecot IMAP updating indexes on the fly -- which it does, unless > deliver does it incrementally. > > However, wasn't your original question about converting procmail recipes > to sieve? (Yes, it was.) So what would hold you back of just not > converting? Yes, because I hadn't realized it was possible to still use procmail at all. I had said previously, although possibly not clearly, that I'd prefer to use continue to use procmail -- sieve looks entirely too convoluted, and I just don't need to do mail filtering frequently enough to have the time to figure out sieve. It also didn't support a few other things I'm currently doing with procmail. I can't figure out why "deliver -m" doesn't work as expected. It just doesn't create the mailbox. Should I just use 'c' with procmail to make a copy to forward on to deliver? DELIVER=/usr/libexec/dovecot/deliver :0 { :0c * ^X-Spam-Status xspamstatus :0 | $DELIVER } There has to be a better way... Thanks, Alex From mgoldey at goldey.net Thu Nov 17 06:48:43 2011 From: mgoldey at goldey.net (Mark Goldey) Date: Wed, 16 Nov 2011 23:48:43 -0500 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 Message-ID: <4EC4922B.5050000@goldey.net> Sorry about the HTML. I removed the string "-O2" from the /auth Makefile and, for good measure, did make clean, make and make install for all of Dovecot. Still, it crashed when trying to log in via IMAP (webmail). Core dumped, here's the backtrace. [snip] Core was generated by `dovecot/auth -w'. Program terminated with signal 11, Segmentation fault. #0 0x00000000 in ?? () (gdb) bt full #0 0x00000000 in ?? () No symbol table info available. #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, args=0x82601c8) at auth-worker-client.c:405 auth_request = 0x82776e8 userdb_id = 1 #2 0x0805b630 in auth_worker_handle_line (client=0x82754d0, line=0x82757da "2\tUSER\t1\tuser=mgoldXX at goldey.net\tservice=imap\toriginal_username=mgoldXX at goldey.net\trequested_login_user\tlip=127.0.0.1\trip=127.0.0.1\tlport=143\trport=47574\tsecured=1\tsuccessful=1\tmech=PLAIN") at auth-worker-client.c:534 args = 0x82601c0 id = 2 ret = false #3 0x0805b890 in auth_worker_input (client=0x82754d0) at auth-worker-client.c:614 _data_stack_cur_id = 3 line = 0x82757da "2\tUSER\t1\tuser=mgoldXX at goldey.net\tservice=imap\toriginal_username=mgoldXX at goldey.net\trequested_login_user\tlip=127.0.0.1\trip=127.0.0.1\tlport=143\trport=47574\tsecured=1\tsuccessful=1\tmech=PLAIN" ret = 183 #4 0xb7726021 in io_loop_call_io (io=0x8275688) at ioloop.c:384 ioloop = 0x8268398 t_id = 2 #5 0xb77270d3 in io_loop_handler_run (ioloop=0x8268398) at ioloop-epoll.c:213 ctx = 0x8273f38 event = 0x8273f78 list = 0x82756b0 io = 0xbf80049f tv = {tv_sec = 2147483, tv_usec = 0} msecs = 1 ret = 1 i = 0 j = 0 call = 192 #6 0xb7725fa0 in io_loop_run (ioloop=0x8268398) at ioloop.c:405 No locals. #7 0xb7711baa in master_service_run (service=0x82682e8, callback=0x805e466 ) at master-service.c:481 No locals. #8 0x0805e7aa in main (argc=2, argv=0x82681c0) at main.c:298 c = -1 And, for good measure: (gdb) fr 1 #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, args=0x82601c8) at auth-worker-client.c:405 405 auth_request->userdb->userdb->iface-> (gdb) p *auth_request->userdb->userdb $1 = {args = 0x8268d10 "", cache_key = 0x0, blocking = true, id = 1, init_refcount = 1, iface = 0x807b320} (gdb) p *auth_request->userdb->userdb->iface $2 = {name = 0x8074f3d "vpopmail", preinit = 0x8069a27 , init = 0, deinit = 0, lookup = 0x806975f , iterate_init = 0, iterate_next = 0, iterate_deinit = 0} This is all Greek to me, but I'm happy to do more troubleshooting. --mhg From janfrode at tanso.net Thu Nov 17 11:34:49 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 17 Nov 2011 10:34:49 +0100 Subject: [Dovecot] Performance-Tuning In-Reply-To: References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> Message-ID: <20111117093449.GA18157@dibs.tanso.net> On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > Wich clustered filesytem do you have? We use IBM GPFS, with currently 7 servers working against shared LUNs from an IBM DS4800. > > My ocfs2 setup had some problems... but still.. > Some numbers: > > OCFS2 > 1TB of maildir files. > Full backup 36 Hours > Incremental 15 hours > > Ext4 > 1TB of maildir files. > Full backup 16 Hours > Incremental 1 hour Wow, local fs's are fast! We have split the backup process up to run on 6 of the servers, with each server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). The backup processing time varies quite a bit every day, but are mostly somewhere between 14-24 hours on each server. This sounds like something between 1.5x to 2x the incremental-performance you're seeing: 15 hours/incremental of 1TB ocfs2 = 15h/TB 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB 6x 20 hours for incremental of 12 TB GPFS = 10h/TB All our backups are incremental. -jf From stan at hardwarefreak.com Thu Nov 17 12:27:50 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 17 Nov 2011 04:27:50 -0600 Subject: [Dovecot] Please advise on very fast search In-Reply-To: <1321383763.21919.484.camel@hurina> References: <4EBAA767.6060609@lazurit.com> <4EBB1DF9.4040208@hardwarefreak.com> <4EC1271B.4060604@lazurit.com> <4EC15C92.4000707@hardwarefreak.com> <43B341A1-9245-4863-932A-887E21970319@lazurit.com> <4EC2AEDE.2080306@hardwarefreak.com> <1321383763.21919.484.camel@hurina> Message-ID: <4EC4E1A6.7060101@hardwarefreak.com> On 11/15/2011 1:02 PM, Timo Sirainen wrote: > On Tue, 2011-11-15 at 12:26 -0600, Stan Hoeppner wrote: > >> This is why I recommended mbox in the first place. If your only writes >> to these mailbox files are appends of new messages, mbox is the best >> format by far. It's faster at appending than any other format, and it's >> faster for searching than any other. > > Just as long as you're not simultaneously trying to read and write the > mbox file (or just write in 2+ sessions). Then there's a lot waiting on > locks. (mdbox has no read locks, and its write locks are very short > lived.) Of course. My understanding of Alexander's workflow is that copies of all daily new mail are written to an IMAP mailbox via some MTA bcc rule or sieve rule. A nightly script moves the daily mail to another mailbox created and named by date. These named mailboxes are then used for backup and the search function, but are never written to again. So I assume there is no simultaneous read/write of the archive mailboxes he performs searches on. It's possible I don't fully understand Alexander's work flow yet. -- Stan From Mark.Zealey at webfusion.com Thu Nov 17 12:57:09 2011 From: Mark.Zealey at webfusion.com (Mark Zealey) Date: Thu, 17 Nov 2011 10:57:09 +0000 Subject: [Dovecot] Port variable in LMTP userdb lookups? In-Reply-To: <1321467929.21919.583.camel@hurina> References: <4E9FEC22.8060105@webfusion.com> <1320443764.21919.233.camel@hurina> <4EB7BF9A.9020900@webfusion.com>,<1321467929.21919.583.camel@hurina> Message-ID: Thanks so much for that have tested in 2.0.16 release now and is working fine. Mark ________________________________________ From: Timo Sirainen [tss at iki.fi] Sent: 16 November 2011 18:25 To: Mark Zealey Cc: Dovecot Mailing List Subject: Re: [Dovecot] Port variable in LMTP userdb lookups? On Mon, 2011-11-07 at 13:23 +0200, Mark Zealey wrote: > 04-11-2011 23:56, Timo Sirainen yazm??: > From my reading of the code in 2.0.15, you pass the local & remote IP > in lmtp/commands.c:427 but the struct mail_storage_service_input only > has remote_ip and local_ip fields, not port fields (unlike the > auth_user_info struct which has both ip's and port entries as well). I > think that the mail_storage_service_input struct needs the uint > local_port, remote_port fields adding in & appropriate code changes to > pass these through? Yeah, you're right, I missed that one. Added to hg now. From marco at carcano.ch Thu Nov 17 18:47:34 2011 From: marco at carcano.ch (Marco Carcano) Date: Thu, 17 Nov 2011 17:47:34 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <1320437339.21919.165.camel@hurina> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: Hello Timo, and thanks for your reply I waited to reply until got it another time as I already said, it does not happen very often, for example it happened on 12 november - the log is at the end of this mail > Enable mail_log plugin to make sure of this. > http://wiki2.dovecot.org/Plugins/MailLog I already did, but just for a few days: it does not happen very often that we loose mails, so I'm afraid I can damage the disks of the production server if I keep logging enabled for too much time - it will be a pain, years ago I had a server damaged because of logging enabled for too much time. I do not want to repeat such a painfull experience > >> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>> : saved mail to INBOX > > If Dovecot logs this, then the message definitely was saved to INBOX. it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - I suspect however that could be mine misconfiguration somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs (however I'm wondering why sometimes?!?) I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are cd /home/mailboxstore/theuser/Maildir grep "6000029222" */* |grep "RE:" grep "6000029222" .Drafts/* |grep "RE:" grep "6000029222" .Drafts/*/* |grep "RE:" grep "6000029222" .Junk/* |grep "RE:" grep "6000029222" .Posta\ eliminata/* |grep "RE:" grep "6000029222" .Posta\ indesiderata/* |grep "RE:" grep "6000029222" .Posta\ inviata/* |grep "RE:" grep "6000029222" .Sent/* |grep "RE:" grep "6000029222" .Templates/* |grep "RE:" grep "6000029222" .Trash/* |grep "RE:" and never got anything here is the log instead Nov 12 08:48:01 srv001 postfix/smtpd[1430]: connect from mail.tasnee.com[62.3.52.58] Nov 12 08:48:02 srv001 postfix/smtpd[1430]: 6C3874E4A9F: client=mail.tasnee.com[62.3.52.58] Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: warning: header Subject: RE: RFQ NO. 6000029222 - OUR OFFER NO. 2111221 from mail.tasnee.com[62.3.52.58]; from= to= proto=ESMTP helo= Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa > Nov 12 08:48:03 srv001 postfix/qmgr[4876]: 6C3874E4A9F: from=, size=9920, nrcpt=4 (queue active) Nov 12 08:48:06 srv001 postfix/smtpd[1442]: connect from localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 postfix/smtpd[1442]: 244774E4AA2: client=localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 postfix/cleanup[1434]: 244774E4AA2: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: from=, size=10323, nrcpt=4 (queue active) Nov 12 08:48:06 srv001 postfix/smtpd[1442]: disconnect from localhost.localdomain[127.0.0.1] Nov 12 08:48:06 srv001 amavis[8902]: (08902-05) Passed CLEAN, [62.3.52.58] [62.3.52.58] -> < user2 @ourdomain.local>,,,, Message-ID: <899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >, mail_id: z4aAgl2gBrfV, Hits: -0.592, size: 9919, queued_as: 244774E4AA2, 2632 ms Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 6C3874E4A9F: removed Nov 12 08:48:06 srv001 dovecot: lda(user3): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 dovecot: lda(user2): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1452]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.02/0/0.05, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 postfix/pipe[1445]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.01/0/0.06, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 dovecot: lda(theuser): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1447]: 244774E4AA2: to=, relay=dovecot, delay=0.09, delays=0.01/0.01/0/0.07, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 dovecot: lda(user4): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa >: saved mail to INBOX Nov 12 08:48:06 srv001 postfix/pipe[1450]: 244774E4AA2: to=, relay=dovecot, delay=0.26, delays=0.01/0.02/0/0.23, dsn=2.0.0, status=sent (delivered via dovecot service) Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: removed Nov 12 08:48:08 srv001 postfix/smtpd[1430]: disconnect from mail.tasnee.com[62.3.52.58] as you can see again, the mail seems delivered, but is not I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the whole config) thank you Marco Carcano From bind at enas.net Thu Nov 17 19:29:03 2011 From: bind at enas.net (Urban Loesch) Date: Thu, 17 Nov 2011 18:29:03 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: <4EC5445F.8@enas.net> Hi, On 17.11.2011 17:47, Marco Carcano wrote: > Hello Timo, and thanks for your reply > > I waited to reply until got it another time > > as I already said, it does not happen very often, for example it happened on 12 november - the log is at the end of this mail > > >> Enable mail_log plugin to make sure of this. >> http://wiki2.dovecot.org/Plugins/MailLog > > I already did, but just for a few days: it does not happen very often that we loose mails, so I'm afraid I can damage the disks of the production > server if I keep logging enabled for too much time - it will be a pain, years ago I had a server damaged because of logging enabled for too much time. > I do not want to repeat such a painfull experience I think logging is not a main reason for damaging disks. I have enabled mail_log plugin since march 2011 without problems and it helps me very often in such cases. > >> >>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>> : saved mail to INBOX >> >> If Dovecot logs this, then the message definitely was saved to INBOX. > > it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - I suspect however that could be mine misconfiguration > somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs > (however I'm wondering why sometimes?!?) > Could it be that some other person is downloading the mail via pop3 and then the client is deleting it from the server? This happens to me sometimes when a customer is accessing his account with a new client but forgot to disable the same account on his old PC. So it happens, that the old pc downloads all new mails, delete it then and the customer never see new mails on his new client. Regards Urban > I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are > > cd /home/mailboxstore/theuser/Maildir > > grep "6000029222" */* |grep "RE:" > grep "6000029222" .Drafts/* |grep "RE:" > grep "6000029222" .Drafts/*/* |grep "RE:" > grep "6000029222" .Junk/* |grep "RE:" > grep "6000029222" .Posta\ eliminata/* |grep "RE:" > grep "6000029222" .Posta\ indesiderata/* |grep "RE:" > grep "6000029222" .Posta\ inviata/* |grep "RE:" > grep "6000029222" .Sent/* |grep "RE:" > grep "6000029222" .Templates/* |grep "RE:" > grep "6000029222" .Trash/* |grep "RE:" > > and never got anything > > here is the log instead > > Nov 12 08:48:01 srv001 postfix/smtpd[1430]: connect from mail.tasnee.com[62.3.52.58] > Nov 12 08:48:02 srv001 postfix/smtpd[1430]: 6C3874E4A9F: client=mail.tasnee.com[62.3.52.58] > Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: warning: header Subject: RE: RFQ NO. 6000029222 - OUR OFFER NO. 2111221 from > mail.tasnee.com[62.3.52.58]; from= to= proto=ESMTP helo= > Nov 12 08:48:03 srv001 postfix/cleanup[1434]: 6C3874E4A9F: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa> > Nov 12 08:48:03 srv001 postfix/qmgr[4876]: 6C3874E4A9F: from=, size=9920, nrcpt=4 (queue active) > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: connect from localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: 244774E4AA2: client=localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 postfix/cleanup[1434]: 244774E4AA2: message-id=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa> > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: from=, size=10323, nrcpt=4 (queue active) > Nov 12 08:48:06 srv001 postfix/smtpd[1442]: disconnect from localhost.localdomain[127.0.0.1] > Nov 12 08:48:06 srv001 amavis[8902]: (08902-05) Passed CLEAN, [62.3.52.58] [62.3.52.58] -> > ,,,, Message-ID: > <899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>, mail_id: z4aAgl2gBrfV, Hits: -0.592, size: 9919, queued_as: 244774E4AA2, 2632 ms > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, > relay=127.0.0.1[127.0.0.1]:10024, delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: > queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/lmtp[1438]: 6C3874E4A9F: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=3.9, delays=1.2/0.01/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 244774E4AA2) > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 6C3874E4A9F: removed > Nov 12 08:48:06 srv001 dovecot: lda(user3): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 dovecot: lda(user2): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1452]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.02/0/0.05, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 postfix/pipe[1445]: 244774E4AA2: to=, relay=dovecot, delay=0.08, delays=0.01/0.01/0/0.06, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 dovecot: lda(theuser): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1447]: 244774E4AA2: to=, relay=dovecot, delay=0.09, delays=0.01/0.01/0/0.07, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 dovecot: lda(user4): msgid=<899EAB831EA7414F994704DB43677A140450EE74 at NPICMAIL.npic.com.sa>: saved mail to INBOX > Nov 12 08:48:06 srv001 postfix/pipe[1450]: 244774E4AA2: to=, relay=dovecot, delay=0.26, delays=0.01/0.02/0/0.23, dsn=2.0.0, > status=sent (delivered via dovecot service) > Nov 12 08:48:06 srv001 postfix/qmgr[4876]: 244774E4AA2: removed > Nov 12 08:48:08 srv001 postfix/smtpd[1430]: disconnect from mail.tasnee.com[62.3.52.58] > > as you can see again, the mail seems delivered, but is not > > I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the > whole config) > > > thank you > > Marco Carcano > > > > > From tss at iki.fi Thu Nov 17 19:34:01 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 19:34:01 +0200 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> Message-ID: <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> On 17.11.2011, at 18.47, Marco Carcano wrote: >>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>> : saved mail to INBOX >> >> If Dovecot logs this, then the message definitely was saved to INBOX. > > it is exactly what I told to my colleagues, but belive me, sometime some mail get lost - Most likely reason for this is that the user's client deletes the message. Possibly an automatic client side filter or some UI issue that causes user to accidentally delete a mail. The mail_log plugin's logging would have showed if this was the case. > I suspect however that could be mine misconfiguration somewhere, so that lda sometimes write the email not in the right place, but elsewhere, and just write the phrase " saved mail to INBOX in the logs (however I'm wondering why sometimes?!?) I can't think of any reason why it would randomly write to a wrong place. > I tried to find the missed email in the Maildir, but have not been able to get it - the commands used are > > cd /home/mailboxstore/theuser/Maildir > > grep "6000029222" */* |grep "RE:" > grep "6000029222" .Drafts/* |grep "RE:" > grep "6000029222" .Drafts/*/* |grep "RE:" > grep "6000029222" .Junk/* |grep "RE:" > grep "6000029222" .Posta\ eliminata/* |grep "RE:" > grep "6000029222" .Posta\ indesiderata/* |grep "RE:" > grep "6000029222" .Posta\ inviata/* |grep "RE:" > grep "6000029222" .Sent/* |grep "RE:" > grep "6000029222" .Templates/* |grep "RE:" > grep "6000029222" .Trash/* |grep "RE:" Only the grep "6000029222" .Drafts/*/* |grep "RE:" was grepping from mail files. Easier would be just: grep -r "RE:.*6000029222" . > I really think is some kind of misconfiguration of mine, may you help me, please? Just tell me what pieces of config to show (just not to flood the whole config) I doubt this is related to configuration. But you could enable http://wiki2.dovecot.org/Plugins/Lazyexpunge so that messages won't be lost if they are expunged. The next time a message is lost, you would most likely find it from the lazy-expunge namespace. (Then you could write a script that deletes e.g. >1 week old files nightly.) From sf.rique at gmail.com Thu Nov 17 19:38:18 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Thu, 17 Nov 2011 15:38:18 -0200 Subject: [Dovecot] Performance-Tuning In-Reply-To: <20111117093449.GA18157@dibs.tanso.net> References: <201111081450.25798.p.heinlein@heinlein-support.de> <4EC0534C.8070409@wk-serv.de> <201111141034.03060.p.heinlein@heinlein-support.de> <20111114102719.GA2330@dibs.tanso.net> <4EC16116.1000902@hardwarefreak.com> <20111117093449.GA18157@dibs.tanso.net> Message-ID: []'sf.rique How many servers have access to your maildir on ext4 so that you could > switch from ocfs2 to ext4? > I use ocfs2 in my test environment for four servers (2 MX and 2 IMAP) I used have 3 serves one mailman and 2 imaps. I experence that if take my loadbalancer and takes it all to just one server, itwould go faster becasue of cache. But once thsi server fails, the loadbalaner send it to the other one, but it simple does not work, clients experence timeouts, because the filesystem were too slow. On Thu, Nov 17, 2011 at 7:34 AM, Jan-Frode Myklebust wrote: On Wed, Nov 16, 2011 at 11:53:29PM -0200, Henrique Santos Fernandes wrote: > > > > Wich clustered filesytem do you have? > > We use IBM GPFS, with currently 7 servers working against shared LUNs > from an IBM DS4800. > > > > > My ocfs2 setup had some problems... but still.. > > Some numbers: > > > > OCFS2 > > 1TB of maildir files. > > Full backup 36 Hours > > Incremental 15 hours > > > > Ext4 > > 1TB of maildir files. > > Full backup 16 Hours > > Incremental 1 hour > > Wow, local fs's are fast! > Much faster! I could not run a "du" before, if i did would crash the system, now i can! > We have split the backup process up to run on 6 of the servers, with each > server processing only a part of the filesystem (/a-f, /g-l, /m-p, etc..). > The backup processing time varies quite a bit every day, but are mostly > somewhere between 14-24 hours on each server. This sounds like something > between 1.5x to 2x the incremental-performance you're seeing: > > 15 hours/incremental of 1TB ocfs2 = 15h/TB > 6x 15 hours for incremental of 12 TB GPFS = 7.5h/TB > 6x 20 hours for incremental of 12 TB GPFS = 10h/TB > > All our backups are incremental. > > > -jf > From e-frog at gmx.de Thu Nov 17 19:45:50 2011 From: e-frog at gmx.de (e-frog) Date: Thu, 17 Nov 2011 18:45:50 +0100 Subject: [Dovecot] 2.1: virtual plugin and lsub In-Reply-To: <1321466541.21919.576.camel@hurina> References: <4EB6AFE3.3020001@gmx.de> <4EBACFC0.6060703@gmx.de> <1321466541.21919.576.camel@hurina> Message-ID: <4EC5484E.1040709@gmx.de> On 16.11.2011 19:02, wroteTimo Sirainen: > On Wed, 2011-11-09 at 20:08 +0100, e-frog wrote: >> On 06.11.2011 17:03, wrote e-frog: >>> 1 lsub "" "virtual/*" >>> * LSUB () "/" "virtual" >>> * LSUB () "/" "virtual/Lists" >>> * LSUB () "/" "virtual/unread" >>> 1 OK Lsub completed. >> >> Just tested this again with 2.1beta1 with the same result. > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/16024d03f66e > Thanks, Timo. Tested and works now as expected. From marco at carcano.ch Thu Nov 17 22:50:09 2011 From: marco at carcano.ch (Marco Carcano) Date: Thu, 17 Nov 2011 21:50:09 +0100 Subject: [Dovecot] Mail lost - maybe a bug??? In-Reply-To: <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> References: <0A7F1BA2-87A8-4688-A69B-86EAA7D78A73@carcano.ch> <1320437339.21919.165.camel@hurina> <9F24E73A-7DC3-403C-B036-FC5CA989CD3E@iki.fi> Message-ID: <126C0E4E-460B-447D-A85C-9AEC956020B5@carcano.ch> Hello Timo and Urban, and thank you for the very quick reply I have already thought about these things: I have already disabled mailbox deletion from POP3 clients, that actually just leave the messages on the server however, this time noone of the 4 recipients got the email - so it is really strange that all che clients deleted the same message and left all the others we can notice this only a few days after the issue, because customers send us a reminder forwarding the lost mail Il giorno 17/nov/11, alle ore 18:34, Timo Sirainen ha scritto: > On 17.11.2011, at 18.47, Marco Carcano wrote: > >>>> Oct 27 11:20:34 srv001 dovecot: lda(user3): msgid=>>>> : saved mail to INBOX >>> >>> If Dovecot logs this, then the message definitely was saved to >>> INBOX. >> >> it is exactly what I told to my colleagues, but belive me, sometime >> some mail get lost - > > Most likely reason for this is that the user's client deletes the > message. Possibly an automatic client side filter or some UI issue > that causes user to accidentally delete a mail. The mail_log > plugin's logging would have showed if this was the case. > >> I suspect however that could be mine misconfiguration somewhere, so >> that lda sometimes write the email not in the right place, but >> elsewhere, and just write the phrase " saved mail to INBOX in the >> logs (however I'm wondering why sometimes?!?) > > I can't think of any reason why it would randomly write to a wrong > place. maybe something related to LDAP lookups? what can happen if for some reason the LDAP server does not reply to the lookup? Timo, belive to me, I use dovecot since 3 years, and I am one of the thousands of happy user, I really can't explain what is happening on this server I'll do this trial: I'll enable lda debug logging ont a file on a USB hard disk - in order to know where exactly messages are stored, and enable also maillog too I'll keep you informed on what I'll find - maybe it will take some days, as I told it happens quite seldom thank you > >> I tried to find the missed email in the Maildir, but have not been >> able to get it - the commands used are >> >> cd /home/mailboxstore/theuser/Maildir >> >> grep "6000029222" */* |grep "RE:" >> grep "6000029222" .Drafts/* |grep "RE:" >> grep "6000029222" .Drafts/*/* |grep "RE:" >> grep "6000029222" .Junk/* |grep "RE:" >> grep "6000029222" .Posta\ eliminata/* |grep "RE:" >> grep "6000029222" .Posta\ indesiderata/* |grep "RE:" >> grep "6000029222" .Posta\ inviata/* |grep "RE:" >> grep "6000029222" .Sent/* |grep "RE:" >> grep "6000029222" .Templates/* |grep "RE:" >> grep "6000029222" .Trash/* |grep "RE:" > > Only the grep "6000029222" .Drafts/*/* |grep "RE:" was grepping from > mail files. Easier would be just: > > grep -r "RE:.*6000029222" . > >> I really think is some kind of misconfiguration of mine, may you >> help me, please? Just tell me what pieces of config to show (just >> not to flood the whole config) > > I doubt this is related to configuration. > > But you could enable http://wiki2.dovecot.org/Plugins/Lazyexpunge so > that messages won't be lost if they are expunged. The next time a > message is lost, you would most likely find it from the lazy-expunge > namespace. (Then you could write a script that deletes e.g. >1 week > old files nightly.) From andy.robbins at firespring.com Thu Nov 17 23:30:07 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 15:30:07 -0600 Subject: [Dovecot] Dovecot performance issues with many writes Message-ID: We are currently experiencing performance issue with our Dovecot system which we believe is caused by excessive writes to the dovecot files. The confusing thing is that we are seeing more writes than reads on our Dovecot volume when you would assume that most of the IO should be reads from customers checking their mail. We're seeing reads vs. writes similar to the following: # iostat -d 5 -x Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 8.00 28.69 19.63 0.70 94.00 drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 8.00 111.42 94.44 0.74 99.60 We can't figure out why we would be seeing twice as many writes as we are reads. Could this have something to do with the index files? Appreciate any assistance on this issue. From micah at riseup.net Thu Nov 17 23:33:37 2011 From: micah at riseup.net (micah anderson) Date: Thu, 17 Nov 2011 16:33:37 -0500 Subject: [Dovecot] dsync problems In-Reply-To: <1321389804.21919.530.camel@hurina> References: <871ut9i3z1.fsf@algae.riseup.net> <1321389804.21919.530.camel@hurina> Message-ID: <87d3cqe8ni.fsf@algae.riseup.net> On Tue, 15 Nov 2011 22:43:24 +0200, Timo Sirainen wrote: > On Tue, 2011-11-15 at 14:24 -0500, Micah Anderson wrote: > > When a user renames their username, I am using dsync to copy their mail > > over to the new username's mail location[0]. > > > > Some of the dsyncs are failing with errors that I dont know how to work > > with, for example: > > > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > > dsync(): Error: msg iteration failed: Couldn't open mailbox 41fcd40303c8a64e432300007ef44c7a > > dsync(): Error: Trying to open a non-listed mailbox with guid=41fcd40303c8a64e432300007ef44c7a > > These shouldn't really happen. Something's going internally wrong with > dsync. Can you reproduce this reliably somehow? Well, I dont know if I can do it reliably, but its been happening a lot. One point of information that might be useful is that these users were converted from courier maildir to mdbox, and their courier bits are still around in the source mailbox (I haven't become brave enough to remove them yet). > > > The errors cause a non-zero exit code from dsync, which causes my rename > > script to bail out. What are these errors, and how can I fix them? > > Does a second dsync on error succeed? :) Before I tried it again, I looked at their mailboxes: # doveadm mailbox list -u ): Info: INBOX: only in dest (guid=14bf0409fa08c04e682900007ef44c7a) dsync(): Error: Trying to open a non-listed mailbox with guid=14bf0409fa08c04e682900007ef44c7a dsync(): Error: msg iteration failed: Couldn't open mailbox 14bf0409fa08c04e682900007ef44c7a dsync(): Error: Trying to open a non-listed mailbox with guid=14bf0409fa08c04e682900007ef44c7a and the mailboxes: # doveadm mailbox list -u Trash_084ed82bc59ca54eb53700007ef44c7a Sent Drafts INBOX_094ed82bc59ca54eb53700007ef44c7a I tried it a third time, and... it worked, no error, and now their mailbox list: # doveadm mailbox list -u Trash Sent Drafts INBOX This particular user only has one test email > > > 0. Why use dsync instead of a simple mv operation? This seems to be > > necessary for two corner cases: > > > > 1. dovecot creates the new mailbox automatically when the user logs in > > or receives a mail, so if the user changes their mail and logs in or > > receives an email before the move has been done, then the mailbox is > > created and then a move command will fail. > > > > 2. If there has been new mail created under the new name, we can't just > > simply remove the stuff that is automatically created there and replace > > it with the old things because we could potentially be removing mail > > that has been delivered in the mean time. > > You could temporarily change the permissions for the home directory so > that no new mailboxes/mails could be created during the move (e.g. 0700 > root). The problem is there are a number of users on the system and all the mail is stored under /srv/mailstorage//. So if "foo" wants to change their username to "bar" -- I dont have a deterministic way of determining that "bar" exists yet because mail could be delivered or they could login and dovecot would create it and I can't set /srv/mailstorage/ 0700 root or nobody would be able to receive mail. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From tss at iki.fi Thu Nov 17 23:51:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 17 Nov 2011 23:51:06 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> On 17.11.2011, at 23.30, Andy Robbins wrote: > We can't figure out why we would be seeing twice as many writes as we are > reads. Could this have something to do with the index files? dovecot -n output would be helpful. From andy.robbins at firespring.com Fri Nov 18 00:20:15 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 16:20:15 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> Message-ID: # dovecot -n # 1.2.9: //etc/dovecot.conf # OS: Linux 2.6.18-164.9.1.el5 x86_64 CentOS release 5.4 (Final) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log protocols: imap pop3 ssl_cert_file: /etc/postfix/ssl/smtp1.domain.com.cert ssl_key_file: /etc/postfix/ssl/smtp1.domain.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot//login login_executable(default): //libexec/dovecot/imap-login login_executable(imap): //libexec/dovecot/imap-login login_executable(pop3): //libexec/dovecot/pop3-login login_user: postfix login_greeting: Welcome login_process_per_connection: no login_processes_count: 2 mail_max_userip_connections: 50 mail_uid: 500 mail_gid: 500 mail_location: maildir:~/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /etc/dovecot/last-login-imap.sh mail_executable(imap): /etc/dovecot/last-login-imap.sh mail_executable(pop3): /etc/dovecot/last-login-pop3.sh mail_plugins: quota mail_plugin_dir(default): //lib/dovecot/imap mail_plugin_dir(imap): //lib/dovecot/imap mail_plugin_dir(pop3): //lib/dovecot/pop3 pop3_uidl_format: %f lda: postmaster_address: postmaster at domain.com auth_socket_path: /var/run/dovecot/auth-master log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver.log fsync_disable: no auth default: mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP user: vmail username_translation: %@ verbose: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: maildir:User quota quota_rule: *:storage=0 From tss at iki.fi Fri Nov 18 00:34:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 00:34:33 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> Message-ID: <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> On 18.11.2011, at 0.20, Andy Robbins wrote: > # iostat -d 5 -x > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz > avgqu-sz await svctm %util > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > 8.00 28.69 19.63 0.70 94.00 > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > 8.00 111.42 94.44 0.74 99.60 These devices don't look like NFS, but you have: > mmap_disable: yes > mail_nfs_storage: yes > mail_nfs_index: yes So what filesystem is that? Are they storing only mails, or other stuff too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing things up at least somewhat.. > lda: > fsync_disable: no This is explicitly set for LDA, but it's the default already elsewhere. Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? From janfrode at tanso.net Fri Nov 18 00:46:30 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 17 Nov 2011 23:46:30 +0100 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <20111117224630.GA1315@dibs.tanso.net> On Thu, Nov 17, 2011 at 03:30:07PM -0600, Andy Robbins wrote: > We are currently experiencing performance issue with our Dovecot system > which we believe is caused by excessive writes to the dovecot files. The > confusing thing is that we are seeing more writes than reads on our Dovecot > volume when you would assume that most of the IO should be reads from > customers checking their mail. Have you mounted the fs without noatime options? -jf From andy.robbins at firespring.com Fri Nov 18 00:38:46 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Thu, 17 Nov 2011 16:38:46 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Well, the iostat command was run from the NFS server and dovecot was run from the mail server where it is mounted, hence the discrepancy there. On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > # iostat -d 5 -x > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > avgrq-sz > > avgqu-sz await svctm %util > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 28.69 19.63 0.70 94.00 > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 111.42 94.44 0.74 99.60 > > These devices don't look like NFS, but you have: > > > mmap_disable: yes > > mail_nfs_storage: yes > > mail_nfs_index: yes > > So what filesystem is that? Are they storing only mails, or other stuff > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > things up at least somewhat.. > > > lda: > > fsync_disable: no > > This is explicitly set for LDA, but it's the default already elsewhere. > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > From tss at iki.fi Fri Nov 18 00:42:12 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 00:42:12 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Do you have more than one Dovecot server? If only one, you can still disable the mail_nfs_* settings. Also it would be interesting to see nfsstat numbers from the Dovecot server, compared to those iostat numbers.. On 18.11.2011, at 0.38, Andy Robbins wrote: > Well, the iostat command was run from the NFS server and dovecot was run from the mail server where it is mounted, hence the discrepancy there. > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > # iostat -d 5 -x > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz > > avgqu-sz await svctm %util > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 28.69 19.63 0.70 94.00 > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > 8.00 111.42 94.44 0.74 99.60 > > These devices don't look like NFS, but you have: > > > mmap_disable: yes > > mail_nfs_storage: yes > > mail_nfs_index: yes > > So what filesystem is that? Are they storing only mails, or other stuff too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing things up at least somewhat.. > > > lda: > > fsync_disable: no > > This is explicitly set for LDA, but it's the default already elsewhere. Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > From p.heinlein at heinlein-support.de Fri Nov 18 01:43:01 2011 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Fri, 18 Nov 2011 00:43:01 +0100 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: Message-ID: <201111180043.01419.p.heinlein@heinlein-support.de> Am Donnerstag, 17. November 2011, 22:30:07 schrieb Andy Robbins: > We are currently experiencing performance issue with our Dovecot > system which we believe is caused by excessive writes to the dovecot > files. The confusing thing is that we are seeing more writes than > reads on our Dovecot volume when you would assume that most of the > IO should be reads from customers checking their mail. Yes, but your read-requests are answered from your local kernel-buffer / cache and aren't really read from your hard-disk. (At least if you have enough free memory for a cache, that's large enought.) Incoming mails ARE written on the disk, but read later on just from your cache. Peer -- Heinlein Professional Linux Support GmbH Linux: Akademie - Support - Hosting http://www.heinlein-support.de Tel: 030 / 40 50 51 - 0 Fax: 030 / 40 50 51 - 19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From victormanuelo at gmail.com Fri Nov 18 05:24:53 2011 From: victormanuelo at gmail.com (Victor) Date: Thu, 17 Nov 2011 22:54:53 -0430 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 Message-ID: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> Hi, I'm new in this List, but I have 6 years using Dovecot on my debian from etch,lenny and now squeeze Package: dovecot-imapd Version: 1:1.2.15-4 Tags: squeeze -- System Information: Debian Release: 6.0 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/24 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages dovecot-imapd depends on: ii dovecot-common 1:1.2.15-4 secure mail server that supports m ii libc6 2.11.2-10 Embedded GNU C Library: Shared lib ii libldap-2.4-2 2.4.23-7 OpenLDAP libraries ii libssl0.9.8 0.9.8o-4 SSL shared libraries the quota_imap plugin not working when I use against MailquotaSize Attribute of LDAP users. this setup works perfectly on lenny and dovecot-imapd 1:1.0.15-2.3 but on squeeze dovecot-imad 1:1.2.15-7 doesn't work, the service is up, but The quota does not display. In logs nothing happens the configuration in dovecot.conf below protocol imap { mail_plugins = quota imap_quota } protocol lda { mail_plugins = quota } userdb ldap { args = /etc/dovecot/dovecot-ldap.conf } the configuration in dovecot-ldap.conf below uris = ldap://server-ldap.com.ve auth_bind = yes ldap_version = 3 base = dc=dominio,dc=com,dc=uk deref = never scope = subtree user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota=dirsize:storage user_filter = (&(objectClass=posixAccount)(uid=%u)) pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid pass_filter = (&(objectClass=posixAccount)(uid=%u)) This is the same configuration in the 6 last years. So, I would really appreciate if give me a hand which are more than 15000 Users who administered easily taking the quota in the LDAP attribute. and I want to keep the scheme. Thz Victor Onate Usuario Linux From oni-neko at gmx.net Fri Nov 18 15:34:28 2011 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Fri, 18 Nov 2011 14:34:28 +0100 Subject: [Dovecot] seting acl for master user? Message-ID: <20111118133428.63140@gmx.net> Hello! I'm sorry, but i don't really understand it the way it is written in the wiki, so i'm trying my luck here =) i followed the wiki for creating a master user (the non-sql-version) and i can login as another user via this master user. my dovecot version is 1.2.9 on an ubuntu server 10.4.3 lts. what i don't get is the acl-setting, so that the master-user can access all the other users mails and folders. i made the changes in the dovecot.conf as written in the wiki: protocol imap { mail_plugins = acl imap_acl } protocol lda { mail_plugins = sieve acl } plugin { acl = vfile:/etc/dovecot/acls } in my case I only want the master user (called cyrus) to be able to do everything with/in all mailboxes, so I made a file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". i'm pretty sure i'm missing something rather obvious here, but somehow i'm just stuck. any help would be greatly appreciated. thanks in advance =) silvana -- NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zur?ck-Garantie! Jetzt informieren: http://www.gmx.net/de/go/freephone From robert at schetterer.org Fri Nov 18 15:47:29 2011 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 18 Nov 2011 14:47:29 +0100 Subject: [Dovecot] seting acl for master user? In-Reply-To: <20111118133428.63140@gmx.net> References: <20111118133428.63140@gmx.net> Message-ID: <4EC661F1.4000201@schetterer.org> Am 18.11.2011 14:34, schrieb oni-neko at gmx.net: > Hello! > > I'm sorry, but i don't really understand it the way it is written in the wiki, so i'm trying my luck here =) > i followed the wiki for creating a master user (the non-sql-version) and i can login as another user via this master user. > my dovecot version is 1.2.9 on an ubuntu server 10.4.3 lts. > > what i don't get is the acl-setting, so that the master-user can access all the other users mails and folders. > i made the changes in the dovecot.conf as written in the wiki: > protocol imap { > mail_plugins = acl imap_acl > } > protocol lda { > mail_plugins = sieve acl > } > plugin { > acl = vfile:/etc/dovecot/acls > } > > in my case I only want the master user (called cyrus) to be able to do everything with/in all mailboxes, so I made a file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". > > i'm pretty sure i'm missing something rather obvious here, but somehow i'm just stuck. any help would be greatly appreciated. thanks in advance =) > > silvana are you sure that you dont mixed up master users with acl normally you use master user like described here http://wiki.dovecot.org/Authentication/MasterUsers -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From dave.mehler at gmail.com Fri Nov 18 16:27:42 2011 From: dave.mehler at gmail.com (David Mehler) Date: Fri, 18 Nov 2011 09:27:42 -0500 Subject: [Dovecot] setting up multiple listeners Message-ID: Hello, I'm using Dovecot v2 and am atempting to get webmail going. I'm trying to set up imap so that port 993 only is listening on the external interface, for both ipv4 and ipv6, but that on localhost only again for ipv4 and ipv6 unencrypted port 143 is active. I've tried various combinations but keep getting an error invalid number. I've googled and not come up with the missing point. Thanks. Dave. From tss at iki.fi Fri Nov 18 19:22:02 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:22:02 +0200 Subject: [Dovecot] seting acl for master user? In-Reply-To: <20111118133428.63140@gmx.net> References: <20111118133428.63140@gmx.net> Message-ID: <1321636922.11489.46.camel@hurina> On Fri, 2011-11-18 at 14:34 +0100, oni-neko at gmx.net wrote: > what i don't get is the acl-setting, so that the master-user can > access all the other users mails and folders. .. > in my case I only want the master user (called cyrus) to be able to do > everything with/in all mailboxes, so I made a > file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". Unfortunately currently there is no default ACL file. The above would create an ACL file for a mailbox called "default". I'll need to get the default ACLs implemented some day.. But if you don't need ACLs for anything else, you can simply disable the ACL plugin and the master user will have access to everything. If you do need ACLs, there are some kludges you can still do. From tss at iki.fi Fri Nov 18 19:24:31 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:24:31 +0200 Subject: [Dovecot] setting up multiple listeners In-Reply-To: References: Message-ID: <1321637071.11489.48.camel@hurina> On Fri, 2011-11-18 at 09:27 -0500, David Mehler wrote: > Hello, > > I'm using Dovecot v2 and am atempting to get webmail going. I'm trying > to set up imap so that port 993 only is listening on the external > interface, for both ipv4 and ipv6, but that on localhost only again > for ipv4 and ipv6 unencrypted port 143 is active. I've tried various > combinations but keep getting an error invalid number. I've googled > and not come up with the missing point. A full error message would have been nice, but basically you can do: service imap-login { inet_listener imap { address = 127.0.0.1 ::1 } inet_listener imaps { address = 1.2.3.4 f00::baa # your external ip addresses } } From geek at uniserve.com Fri Nov 18 19:25:35 2011 From: geek at uniserve.com (Dave Stevens) Date: Fri, 18 Nov 2011 09:25:35 -0800 Subject: [Dovecot] CCing outbound IMAP mail? Message-ID: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> For some users on my site I'd like to be able to CC a particular user on all outgoing IMAP mails. Incoming I can do but don't see how to do outgoing. Ideas? Dave -- It is told that such are the aerodynamics and wing loading of the bumblebee that, in principle, it cannot fly...if all this be true...life among bumblebees must bear a remarkable resemblance to life in the United States. -- John Kenneth Galbraith, in American Capitalism: The Concept of Countervailing Power From tss at iki.fi Fri Nov 18 19:27:34 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:27:34 +0200 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 In-Reply-To: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> References: <90A70C5D-20A2-4A69-8400-A0BAC5ADF579@gmail.com> Message-ID: <1321637255.11489.50.camel@hurina> On Thu, 2011-11-17 at 22:54 -0430, Victor wrote: > the quota_imap plugin not working when I use against MailquotaSize Attribute of LDAP users. > this setup works perfectly on lenny and dovecot-imapd 1:1.0.15-2.3 but on squeeze dovecot-imad > 1:1.2.15-7 doesn't work, the service is up, but The quota does not display. In logs nothing happens The quota configuration changed between v1.0 and v1.1. > user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mail=Maildir,mailQuotaSize=quota=dirsize:storage > user_filter = (&(objectClass=posixAccount)(uid=%u)) maildirQuotaSize=quota_rule=*:storage=%$ and to dovecot.conf add: plugin { quota = dirsize } From tss at iki.fi Fri Nov 18 19:30:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 18 Nov 2011 19:30:08 +0200 Subject: [Dovecot] CCing outbound IMAP mail? In-Reply-To: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> References: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> Message-ID: <1321637408.11489.53.camel@hurina> On Fri, 2011-11-18 at 09:25 -0800, Dave Stevens wrote: > For some users on my site I'd like to be able to CC a particular user > on all outgoing IMAP mails. Incoming I can do but don't see how to do > outgoing. Ideas? Typically the users would be sending the outgoing mails via your SMTP server and you should be able to do it the same way in MTA. If your users aren't sending mails via your SMTP server, then it depends on if they are saving the mails to the "Sent" mailbox on IMAP server. That "Sent" mailbox could be named different things, and in any case there's really no simple way to trigger sending a mail when a mail is being saved via IMAP.. From robert at schetterer.org Fri Nov 18 19:33:03 2011 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 18 Nov 2011 18:33:03 +0100 Subject: [Dovecot] CCing outbound IMAP mail? In-Reply-To: <1321637408.11489.53.camel@hurina> References: <20111118092535.22441yjs3kr1fbin@webmail.uniserve.com> <1321637408.11489.53.camel@hurina> Message-ID: <4EC696CF.3090807@schetterer.org> Am 18.11.2011 18:30, schrieb Timo Sirainen: > On Fri, 2011-11-18 at 09:25 -0800, Dave Stevens wrote: >> For some users on my site I'd like to be able to CC a particular user >> on all outgoing IMAP mails. Incoming I can do but don't see how to do >> outgoing. Ideas? > > Typically the users would be sending the outgoing mails via your SMTP > server and you should be able to do it the same way in MTA. > > If your users aren't sending mails via your SMTP server, then it depends > on if they are saving the mails to the "Sent" mailbox on IMAP server. > That "Sent" mailbox could be named different things, and in any case > there's really no simple way to trigger sending a mail when a mail is > being saved via IMAP.. > > perhaps not the right thing in postfix there is sender_bcc_maps (default: empty) Optional BCC (blind carbon-copy) address lookup tables, indexed by sender address. The BCC address (multiple results are not supported) is added when mail enters from outside of Postfix. This feature is available in Postfix 2.1 and later. The table search order is as follows: Look up the "user+extension at domain.tld" address including the optional address extension. Look up the "user at domain.tld" address without the optional address extension. Look up the "user+extension" address local part when the sender domain equals $myorigin, $mydestination, $inet_interfaces or $proxy_interfaces. Look up the "user" address local part when the sender domain equals $myorigin, $mydestination, $inet_interfaces or $proxy_interfaces. Look up the "@domain.tld" part. Specify the types and names of databases to use. After change, run "postmap /etc/postfix/sender_bcc". Note: if mail to the BCC address bounces it will be returned to the sender. -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stephan at rename-it.nl Sat Nov 19 18:34:38 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 19 Nov 2011 17:34:38 +0100 Subject: [Dovecot] Released Pigeonhole v0.2.5 for Dovecot v2.0.16 Message-ID: <4EC7DA9E.6010601@rename-it.nl> Hello Dovecot users, Before I move active development of Pigeonhole to Dovecot v2.1, I first release all the pending bug fixes and small improvements. Changelog v0.2.5: + Sieve vacation extension: made discard message for implicit delivery more verbose - The sieve-test tool: mixed up original and final envelope recipient in implementation of command line arguments. - Sieve vacation extension: resolved FIXME regarding the use of variables in the :handle argument. Variables are now handled correctly. - Sieve body extension: fixed handling of :content "message/rfc822". This now yields the headers of the embedded message as required by the specification. Handling of :content "multipart" remains to be fixed. - LDA Sieve plugin: fixed problem with recipient_delimiter configuration. Now it falls back to global recipient_delimiter setting if plugin/recipient_delimiter is not set. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.5.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.5.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From metro_domain_admin at fastmail.fm Sat Nov 19 21:05:13 2011 From: metro_domain_admin at fastmail.fm (metro_domain_admin at fastmail.fm) Date: Sat, 19 Nov 2011 14:05:13 -0500 Subject: [Dovecot] issues with 2.1b1 on OS X Message-ID: <1321729513.18027.140661001080053@webmail.messagingengine.com> Two issues came up for me when building 2.1beta1 on OS X (10.5.8 on PPC). The first was already mentioned in an earlier thread by a BSD user (http://dovecot.org/list/dovecot/2011-September/061201.html). I just wanted to point out that the error is still occurring in spite of a new test for GNU ld in the configure script. I needed to remove "-no-undefined" specifically: #NOPLUGIN_LDFLAGS="-no-undefined" if test "$with_gnu_ld" = yes; then NOPLUGIN_LDFLAGS="$NOPLUGIN_LDFLAGS -Wl,--as-needed" fi More important, Dovecot aborts on launch with the following error: Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument I configured using the same args as I had used successfully on 2.0.15: --with-gssapi --with-ldap --with-sql --prefix=/opt/dovecot-2.X.X. I also tried an exact copy of my known-good config from 2.0.15, to no avail. Thanks, Tony From tss at iki.fi Sat Nov 19 22:34:36 2011 From: tss at iki.fi (Timo Sirainen) Date: Sat, 19 Nov 2011 22:34:36 +0200 Subject: [Dovecot] issues with 2.1b1 on OS X In-Reply-To: <1321729513.18027.140661001080053@webmail.messagingengine.com> References: <1321729513.18027.140661001080053@webmail.messagingengine.com> Message-ID: On 19.11.2011, at 21.05, metro_domain_admin at fastmail.fm wrote: > Two issues came up for me when building 2.1beta1 on OS X (10.5.8 on > PPC). The first was already mentioned in an earlier thread by a BSD user > (http://dovecot.org/list/dovecot/2011-September/061201.html). I just > wanted to point out that the error is still occurring in spite of a new > test for GNU ld in the configure script. I needed to remove > "-no-undefined" specifically: This error? Undefined symbols: "_environ", referenced from: _env_clean in env-util.o http://hg.dovecot.org/dovecot-2.1/rev/b2c9298e981 and http://hg.dovecot.org/dovecot-2.1/rev/9d022d3fba42 fixes it. > More important, Dovecot aborts on launch with the following error: > > Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, > READ, 16) failed: Invalid argument > I configured using the same args as I had used successfully on 2.0.15: > --with-gssapi --with-ldap --with-sql --prefix=/opt/dovecot-2.X.X. I also > tried an exact copy of my known-good config from 2.0.15, to no avail. Works in 10.6. What is the gdb backtrace of the crash with the attached patch? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 577 bytes Desc: not available URL: From davide at cavion.eu Sun Nov 20 11:08:20 2011 From: davide at cavion.eu (Davide Cavion) Date: Sun, 20 Nov 2011 10:08:20 +0100 Subject: [Dovecot] Using google 2 step authentication Message-ID: Google authenticator has a PAM interface: http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions Have someone tried implementing it onto Dovecot? It would be really great if you consider your emails important! From dovecot.user at seibercom.net Sun Nov 20 18:09:22 2011 From: dovecot.user at seibercom.net (Jerry) Date: Sun, 20 Nov 2011 11:09:22 -0500 Subject: [Dovecot] Using google 2 step authentication In-Reply-To: References: Message-ID: <20111120110922.3488d46d@scorpio> On Sun, 20 Nov 2011 10:08:20 +0100 Davide Cavion articulated: > Google authenticator has a PAM interface: > > http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions > > Have someone tried implementing it onto Dovecot? It would be really > great if you consider your emails important! Isn't this an oxymoron; ie "Security" & "Google"? -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From davide at cavion.eu Sun Nov 20 18:18:01 2011 From: davide at cavion.eu (Davide Cavion) Date: Sun, 20 Nov 2011 17:18:01 +0100 Subject: [Dovecot] Using google 2 step authentication In-Reply-To: <20111120110922.3488d46d@scorpio> References: <20111120110922.3488d46d@scorpio> Message-ID: <924E1EE7-6487-443E-AE88-FBD3AF7AA62E@cavion.eu> Haha, you are right =) However this is open source and don't speak with google's servers ;) Il giorno 20/nov/2011, alle ore 17:09, Jerry ha scritto: > On Sun, 20 Nov 2011 10:08:20 +0100 > Davide Cavion articulated: > >> Google authenticator has a PAM interface: >> >> http://code.google.com/p/google-authenticator/wiki/PamModuleInstructions >> >> Have someone tried implementing it onto Dovecot? It would be really >> great if you consider your emails important! > > Isn't this an oxymoron; ie "Security" & "Google"? > > -- > Jerry ? > Dovecot.user at seibercom.net > > Disclaimer: off-list followups get on-list replies or get ignored. > Please do not ignore the Reply-To header. > __________________________________________________________________ > From victormanuelo at gmail.com Mon Nov 21 02:05:27 2011 From: victormanuelo at gmail.com (Victor) Date: Sun, 20 Nov 2011 19:35:27 -0430 Subject: [Dovecot] Mail_quota plugin and LDAP on Dovecot 1.2 Message-ID: <9BD55AF0-B054-4AD3-AC94-E6490E1B495D@gmail.com> Thank a lot, your were completely in the truth Victor Onate From tss at iki.fi Mon Nov 21 04:06:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 04:06:18 +0200 Subject: [Dovecot] issues with 2.1b1 on OS X In-Reply-To: <1321729513.18027.140661001080053@webmail.messagingengine.com> References: <1321729513.18027.140661001080053@webmail.messagingengine.com> Message-ID: <1321841181.24607.23.camel@hurina> On Sat, 2011-11-19 at 14:05 -0500, metro_domain_admin at fastmail.fm wrote: > More important, Dovecot aborts on launch with the following error: > > Nov 19 10:23:52 kerio2 dovecot[33553]: master: Fatal: kevent(EV_ADD, > READ, 16) failed: Invalid argument This probably helps: service stats { fifo_listener stats-mail { mode = 0 } } Since this fails only in OSX 10.5 (not 10.6), I don't know if I'll bother adding any special code to work around it. In any case it means that kevent() can't be used on a FIFO, and that means you can't use v2.1's stats tracking feature. Or possibly you could try if it works with configure --with-ioloop=poll or select instead of kqueue. From preen at informatik.uni-freiburg.de Mon Nov 21 11:18:02 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Mon, 21 Nov 2011 10:18:02 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4DACA6E3.5020301@bio.umass.edu> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> Message-ID: <4ECA174A.1020909@informatik.uni-freiburg.de> Hello, after serveral tests (and reading a lot of howto's) I'm now fairly convinced that the Solaris HW-crypto engine is not automatically used. Even when dovecot ist compiled with the OpenSSL version provided by Solaris. Currently I have only a T1-CPU available for testing (Sun Fire T2000) and after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with ENGINE *e; ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); ENGINE_set_default_ciphers(e); in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() the crypto device gets used. I'm sure that this is not the whole story since this only seems to affect the IMAP login. One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in the various articles). Is there any chance that Dovecot gets updated/patched for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. Regards. Martin Chris Hoogendyk wrote: > > On 4/18/11 6:30 AM, Martin Preen wrote: >> Hello, >> I tried to find out how about to use the hardware crypto engines under >> Solaris (Sun Fire T2000). It seems, that its not just a compilation >> issue: >> >>> For operations that are to be offloaded, it is necessary to restrict >>> use to subset >> > of OpenSSL functions (the EVP_ functions) and explicitly indicate >> the use of the PKCS11 >> > engine; something like the following works for bulk ciphers (the >> process for RSA is similar): >>> >>> ENGINE *e; >>> ENGINE_load_builtin_engines(); >>> e = ENGINE_by_id("pkcs11"); >>> ENGINE_set_default_ciphers(e); >>> EVP_CIPHER_CTX_init (&ctx); >>> EVP_EncryptInit (&ctx, EVP_des_cbc (), key, iv); >>> EVP_EncryptUpdate (.....); >> >> Since I'm not familiar with SSL programming, I wonder if it's possible to >> modify the dovecot 1.2.x source code. Maybe someone has already tried >> this >> or can give me hint. > > You don't want to do that. > > The objective is for the ssl engine to be low level and basically > invisible to layers of application above that use it. So, build openssl > properly and it just works for everything else. > > I'm not at work today, so I don't have reference to my notes. Ah, just > found an email I sent to my colleagues -- copied it below. --------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/~preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6337 bytes Desc: S/MIME Cryptographic Signature URL: From hummel at pasteur.fr Mon Nov 21 12:19:42 2011 From: hummel at pasteur.fr (Thomas Hummel) Date: Mon, 21 Nov 2011 11:19:42 +0100 Subject: [Dovecot] Dovecot Log analysis Message-ID: <20111121101941.GA25724@parmesan.sis.pasteur.fr> Hello, Before writing them myself, I'd like to figure out if some tools exist (anvil ? ...) and how to use them to extract from dovecot logs things like who is doing what and how hard or how fast in order to do some rate limiting. For instance, I'd like to be able to identify, slow or block users who . read or write to much / slice of time . login/logout too often / slice of time Note : I'm still using the 1.2.x branch. Thanks -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From pareshchalke at yahoo.com Mon Nov 21 08:52:44 2011 From: pareshchalke at yahoo.com (Paresh Chalke) Date: Mon, 21 Nov 2011 06:52:44 +0000 (UTC) Subject: [Dovecot] uid=error Message-ID: Hi Timo, We are facing this issue on our mail server 1. Two email id's are mapped to a same account 2. If I send a mail to both the account simultaneously i get below error "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" 3. Due to this the mail is delivered only to 1 email id. 4. I am loosing all my mails. Kindly help in the same ASAP. Regards, Paresh Chalke From wgrcunha at gmail.com Mon Nov 21 16:06:16 2011 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 21 Nov 2011 12:06:16 -0200 Subject: [Dovecot] Iterate over domain Message-ID: Hello, We need to use doveadm commands on all accounts of a single domain, for now i need to query the database using scripts and for each one user, run doveadm. I can use iterate but, only for all server accounts, have any way to use doveadm iterate over a single domain? Thanks From duihi77 at gmail.com Mon Nov 21 16:20:54 2011 From: duihi77 at gmail.com (Duane Hill) Date: Mon, 21 Nov 2011 14:20:54 +0000 Subject: [Dovecot] Iterate over domain In-Reply-To: References: Message-ID: <1178881680.20111121142054@gmail.com> On Monday, November 21, 2011 at 14:06:16 UTC, wgrcunha at gmail.com confabulated: > Hello, > We need to use doveadm commands on all accounts of a single domain, for now > i need to query the database using scripts and for each one user, run > doveadm. I can use iterate but, only for all server accounts, have any way > to use doveadm iterate over a single domain? Instead of using the -A option for all, use: -u '*@thedomain.tld' -- There are 10 kinds of people in the world... Those who understand binary, and those who don't. From jgiles at cybermesa.com Mon Nov 21 17:44:13 2011 From: jgiles at cybermesa.com (jgiles at cybermesa.com) Date: Mon, 21 Nov 2011 08:44:13 -0700 (MST) Subject: [Dovecot] Question about imaps Message-ID: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> Hi List! Had a questions about imap vs. imaps. I read the ssl article at http://wiki2.dovecot.org/SSL and from what I am reading, imaps is no longer needed as imap with initiate SSL/TLS on its own if STARTTLS is started and I have disable_plaintext_auth=yes and ssl=required settings. I do have SSL up and working, and can connect using imap (143) to my server without issue, but was not able to connect using imaps (993) using Kmail as my client. I Was getting a client error stating that the authentication failed using my password. However, there was nothing in the mail.err or mail.log file even when I enable debugging. If I specify imaps as my only protocol, then dovecot fails to even start without any error loging at all. As I stated, imap works just fine. So, is imaps deprecated as mentioned in the article and imaps no longer needed? I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. This server is simply an imap serer and not a MTA (No postfix or sendmail installed.) Thanks! Joe From jgiles at cybermesa.com Mon Nov 21 18:40:02 2011 From: jgiles at cybermesa.com (jgiles at cybermesa.com) Date: Mon, 21 Nov 2011 09:40:02 -0700 (MST) Subject: [Dovecot] Question about imaps In-Reply-To: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> Message-ID: <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > Hi List! > > Had a questions about imap vs. imaps. > > I read the ssl article at http://wiki2.dovecot.org/SSL and from what I am > reading, imaps is no longer needed as imap with initiate SSL/TLS on its > own if STARTTLS is started and I have disable_plaintext_auth=yes and > ssl=required settings. > > I do have SSL up and working, and can connect using imap (143) to my > server without issue, but was not able to connect using imaps (993) using > Kmail as my client. > > I Was getting a client error stating that the authentication failed using > my password. However, there was nothing in the mail.err or mail.log file > even when I enable debugging. > > If I specify imaps as my only protocol, then dovecot fails to even start > without any error loging at all. > > As I stated, imap works just fine. > > So, is imaps deprecated as mentioned in the article and imaps no longer > needed? > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > This server is simply an imap serer and not a MTA (No postfix or sendmail > installed.) > > Thanks! > Joe > > > > LOL, replying to my own e-mail... I just discovered that the problem is most likely in my Kmail 2 client. Tried to connect using the following: openssl s_client -connect mymailserver.com:993 I was able to connect, log in, and get information about my mail folders. Will try Thunderbird when I get home from work. Kmail 2 is still a WIP I guess. That said, I am still reading that imap with starttls is the way to go, so might be a moot issue. Thanks! Joe From tss at iki.fi Mon Nov 21 20:06:19 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 20:06:19 +0200 Subject: [Dovecot] uid=error In-Reply-To: References: Message-ID: On 21.11.2011, at 8.52, Paresh Chalke wrote: > We are facing this issue on our mail server > 1. Two email id's are mapped to a same account > 2. If I send a mail to both the account simultaneously i get below error > > "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, > uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. From dovecot.user at seibercom.net Mon Nov 21 20:20:55 2011 From: dovecot.user at seibercom.net (Jerry) Date: Mon, 21 Nov 2011 13:20:55 -0500 Subject: [Dovecot] Question about imaps In-Reply-To: <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> Message-ID: <20111121132055.15af04cf@scorpio> On Mon, 21 Nov 2011 09:40:02 -0700 (MST) jgiles at cybermesa.com articulated: > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > > Hi List! > > > > Had a questions about imap vs. imaps. > > > > I read the ssl article at http://wiki2.dovecot.org/SSL and from > > what I am reading, imaps is no longer needed as imap with initiate > > SSL/TLS on its own if STARTTLS is started and I have > > disable_plaintext_auth=yes and ssl=required settings. > > > > I do have SSL up and working, and can connect using imap (143) to my > > server without issue, but was not able to connect using imaps (993) > > using Kmail as my client. > > > > I Was getting a client error stating that the authentication failed > > using my password. However, there was nothing in the mail.err or > > mail.log file even when I enable debugging. > > > > If I specify imaps as my only protocol, then dovecot fails to even > > start without any error loging at all. > > > > As I stated, imap works just fine. > > > > So, is imaps deprecated as mentioned in the article and imaps no > > longer needed? > > > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > > This server is simply an imap serer and not a MTA (No postfix or > > sendmail installed.) > > > > Thanks! > > Joe > > LOL, replying to my own e-mail... > > I just discovered that the problem is most likely in my Kmail 2 > client. > > Tried to connect using the following: > > openssl s_client -connect mymailserver.com:993 > > I was able to connect, log in, and get information about my mail > folders. > > Will try Thunderbird when I get home from work. Kmail 2 is still a > WIP I guess. > > That said, I am still reading that imap with starttls is the way to > go, so might be a moot issue. > > Thanks! > Joe Try this: (substitute your actual host for "remote.host") openssl s_client -connect remote.host:143 -starttls imap See if you get a connection. If so, then the server offers STARTTLS. -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From andy.robbins at firespring.com Mon Nov 21 20:34:45 2011 From: andy.robbins at firespring.com (Andy Robbins) Date: Mon, 21 Nov 2011 12:34:45 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: Actually, some of the information I provided previously was incorrect. We have four mail servers in our cluster which each have the NFS share mounted. These four servers handle all SMTP and sending of mail. However, it was thought that we could boost performance by simply running POP3 and IMAP directly from the NFS server, since nothing besides Dovecot is currently running there. So, here is the dovecot -n output from our IMAP/POP3 server where the file system is local storage and is where we believe the performance issues are originating. # dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 2304). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.32-24-server x86_64 Ubuntu 10.04.1 LTS protocols: imap pop3 ssl_cert_file: /etc/dovecot/smtp1.domain.com.cert ssl_key_file: /etc/dovecot/smtp1.domain.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Welcome login_process_per_connection: no login_processes_count: 2 max_mail_processes: 2048 mail_max_userip_connections: 50 mail_uid: 500 mail_gid: 500 mail_location: maildir:~/Maildir fsync_disable: yes maildir_very_dirty_syncs: yes mbox_write_locks: fcntl dotlock mail_executable(default): /etc/dovecot/last-login-imap.sh mail_executable(imap): /etc/dovecot/last-login-imap.sh mail_executable(pop3): /etc/dovecot/last-login-pop3.sh mail_plugins: quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format: %f auth default: mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP username_translation: %@ verbose: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf plugin: quota: maildir:User quota quota_rule: *:storage=0 So you can see that we have already tried a number of performance modifications such as setting fsync_disable: yes with little or no effect. If I run nfsstat on the email servers where the volumes are mounted and only running SMTP I get: # nfsstat Client rpc stats: calls retrans authrefrsh 51762532 72668 0 Client nfs v3: null getattr setattr lookup access readlink 0 0% 9639844 18% 3846547 7% 10495207 20% 11930633 23% 8 0% read write create mkdir symlink mknod 7011062 13% 5372278 10% 1593131 3% 95 0% 0 0% 0 0% remove rmdir rename link readdir readdirplus 830662 1% 0 0% 865484 1% 2576 0% 110845 0% 24963 0% fsstat fsinfo pathconf commit 2074 0% 8 0% 0 0% 37113 0% My apologies for the somewhat garbled output, there, but I it seems to me that we're not witnessing the same higher IO writes as compared to reads. Thanks for your help so far. Hopefully, this sheds more light on your previous questions. On Thu, Nov 17, 2011 at 4:42 PM, Timo Sirainen wrote: > Do you have more than one Dovecot server? If only one, you can still > disable the mail_nfs_* settings. Also it would be interesting to see > nfsstat numbers from the Dovecot server, compared to those iostat numbers.. > > On 18.11.2011, at 0.38, Andy Robbins wrote: > > > Well, the iostat command was run from the NFS server and dovecot was run > from the mail server where it is mounted, hence the discrepancy there. > > > > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > > > # iostat -d 5 -x > > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > avgrq-sz > > > avgqu-sz await svctm %util > > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > 8.00 28.69 19.63 0.70 94.00 > > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > 8.00 111.42 94.44 0.74 99.60 > > > > These devices don't look like NFS, but you have: > > > > > mmap_disable: yes > > > mail_nfs_storage: yes > > > mail_nfs_index: yes > > > > So what filesystem is that? Are they storing only mails, or other stuff > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > things up at least somewhat.. > > > > > lda: > > > fsync_disable: no > > > > This is explicitly set for LDA, but it's the default already elsewhere. > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > > > > > > From jgiles at cybermesa.com Mon Nov 21 21:17:57 2011 From: jgiles at cybermesa.com (Joe Giles) Date: Mon, 21 Nov 2011 12:17:57 -0700 Subject: [Dovecot] Question about imaps In-Reply-To: <20111121132055.15af04cf@scorpio> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> <20111121132055.15af04cf@scorpio> Message-ID: <21698362.YkgfMOa32X@linuxbox> On Monday, November 21, 2011 01:20:55 PM Jerry wrote: > On Mon, 21 Nov 2011 09:40:02 -0700 (MST) > jgiles at cybermesa.com articulated: > > > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: > > > Hi List! > > > > > > Had a questions about imap vs. imaps. > > > > > > I read the ssl article at http://wiki2.dovecot.org/SSL and from > > > what I am reading, imaps is no longer needed as imap with initiate > > > SSL/TLS on its own if STARTTLS is started and I have > > > disable_plaintext_auth=yes and ssl=required settings. > > > > > > I do have SSL up and working, and can connect using imap (143) to my > > > server without issue, but was not able to connect using imaps (993) > > > using Kmail as my client. > > > > > > I Was getting a client error stating that the authentication failed > > > using my password. However, there was nothing in the mail.err or > > > mail.log file even when I enable debugging. > > > > > > If I specify imaps as my only protocol, then dovecot fails to even > > > start without any error loging at all. > > > > > > As I stated, imap works just fine. > > > > > > So, is imaps deprecated as mentioned in the article and imaps no > > > longer needed? > > > > > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. > > > This server is simply an imap serer and not a MTA (No postfix or > > > sendmail installed.) > > > > > > Thanks! > > > Joe > > > > LOL, replying to my own e-mail... > > > > I just discovered that the problem is most likely in my Kmail 2 > > client. > > > > Tried to connect using the following: > > > > openssl s_client -connect mymailserver.com:993 > > > > I was able to connect, log in, and get information about my mail > > folders. > > > > Will try Thunderbird when I get home from work. Kmail 2 is still a > > WIP I guess. > > > > That said, I am still reading that imap with starttls is the way to > > go, so might be a moot issue. > > > > Thanks! > > Joe > > Try this: (substitute your actual host for "remote.host") > > openssl s_client -connect remote.host:143 -starttls imap > > See if you get a connection. If so, then the server offers STARTTLS. > > Hi, Jerry, Thanks for the information. Yes, I ran that command and got my cert and other information proving to me that TLS is working. I was also able to log in and get mailbox information. Thanks again! Joe From victormanuelo at gmail.com Mon Nov 21 21:42:56 2011 From: victormanuelo at gmail.com (=?UTF-8?Q?Victor_O=C3=B1ate?=) Date: Mon, 21 Nov 2011 15:12:56 -0430 Subject: [Dovecot] Question about imaps In-Reply-To: <21698362.YkgfMOa32X@linuxbox> References: <51156.15.219.153.73.1321890253.squirrel@webmail.cybermesa.com> <49624.15.219.153.73.1321893602.squirrel@webmail.cybermesa.com> <20111121132055.15af04cf@scorpio> <21698362.YkgfMOa32X@linuxbox> Message-ID: Try to conect from shell # telnet server 993 2011/11/21 Joe Giles : > On Monday, November 21, 2011 01:20:55 PM Jerry wrote: >> On Mon, 21 Nov 2011 09:40:02 -0700 (MST) >> jgiles at cybermesa.com articulated: >> >> > On Mon, November 21, 2011 8:44 am, jgiles at cybermesa.com wrote: >> > > Hi List! >> > > >> > > Had a questions about imap vs. imaps. >> > > >> > > I read the ssl article at http://wiki2.dovecot.org/SSL and from >> > > what I am reading, imaps is no longer needed as imap with initiate >> > > SSL/TLS on its own if STARTTLS is started and I have >> > > disable_plaintext_auth=yes and ssl=required settings. >> > > >> > > I do have SSL up and working, and can connect using imap (143) to my >> > > server without issue, but was not able to connect using imaps (993) >> > > using Kmail as my client. >> > > >> > > I Was getting a client error stating that the authentication failed >> > > using my password. However, there was nothing in the mail.err or >> > > mail.log file even when I enable debugging. >> > > >> > > If I specify imaps as my only protocol, then dovecot fails to even >> > > start without any error loging at all. >> > > >> > > As I stated, imap works just fine. >> > > >> > > So, is imaps deprecated as mentioned in the article and imaps no >> > > longer needed? >> > > >> > > I am running Dovecot 2.0.13 on an Ubuntu 11.10 64bit server. >> > > This server is simply an imap serer and not a MTA (No postfix or >> > > sendmail installed.) >> > > >> > > Thanks! >> > > Joe >> > >> > LOL, replying to my own e-mail... >> > >> > I just discovered that the problem is most likely in my Kmail 2 >> > client. >> > >> > Tried to connect using the following: >> > >> > openssl s_client -connect mymailserver.com:993 >> > >> > I was able to connect, log in, and get information about my mail >> > folders. >> > >> > Will try Thunderbird when I get home from work. Kmail 2 is still a >> > WIP I guess. >> > >> > That said, I am still reading that imap with starttls is the way to >> > go, so might be a moot issue. >> > >> > Thanks! >> > Joe >> >> Try this: (substitute your actual host for "remote.host") >> >> openssl s_client -connect remote.host:143 -starttls imap >> >> See if you get a connection. If so, then the server offers STARTTLS. >> >> > > Hi, Jerry, > > Thanks for the information. > > Yes, I ran that command and got my cert and other information proving to me that TLS is working. I was also able to log in and get mailbox information. > > Thanks again! > Joe > From stan at hardwarefreak.com Mon Nov 21 21:45:54 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 21 Nov 2011 13:45:54 -0600 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: <4ECAAA72.4040802@hardwarefreak.com> On 11/21/2011 12:34 PM, Andy Robbins wrote: > Warning: fd limit 1024 is lower than what Dovecot can use under full load > (more than 2304). Either grow the limit or change login_max_processes_count > and max_mail_processes settings Why haven't you increased the file descriptor limit? -- Stan From achekalin at lazurit.com Mon Nov 21 22:02:39 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 21 Nov 2011 23:02:39 +0300 Subject: [Dovecot] Unique message IDs? Message-ID: <4ECAAE5F.1040507@lazurit.com> Hello, I'm working with indexing my mail box. What I need is to have index in which mailing specified header string is located. I work with the messages via IMAP. I know that sounds like FTS can help me but no, I don't want to index whole message nor I want to patch FTS source to make it to index headers only. I need a way to identify message across my mail box. The idea is that I can move messages between IMAP folders and the index still should be able to identify it. What I want to know, will UID strings be unique across all my messages no matter which IMAP folder it belongs now to, or the UID can change if I move message? If I remove dovecot-uidlist from the folder will UIDs of messages in the folder be changed? What happened if I put a message to the given IMAP folder (and dovecot-uidlist will record UID for it), then shut down dovecot, and (given maildir storage model) move the message's file info another folder in the same mail box - after I start dovecot, will the UID for the message be the same or not? Thank you in advance, Alexander Chekalin From robert at schetterer.org Mon Nov 21 22:14:53 2011 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 21 Nov 2011 21:14:53 +0100 Subject: [Dovecot] uid=error In-Reply-To: References: Message-ID: <4ECAB13D.3050501@schetterer.org> Am 21.11.2011 19:06, schrieb Timo Sirainen: > On 21.11.2011, at 8.52, Paresh Chalke wrote: > >> We are facing this issue on our mail server >> 1. Two email id's are mapped to a same account >> 2. If I send a mail to both the account simultaneously i get below error >> >> "Nov 21 12:13:37 blade8 dovecot: lda(kapil.gupta at qlc.in): save: box=INBOX, >> uid=error, msgid=<1321857820.4ec9f31ce638c at SwiftMailer.generated>, size=10921" > > This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. > Hi Timo, just for info i also have sporadic error like this Nov 21 20:36:10 mail01 dovecot: lmtp(10398, user at user.de): save: box=INBOX, uid=error, msgid=<20111121193610.69B4D200A2F at server.de>, size=6356 as mails get delivered i dont care, i see this happen sometimes during rsync backuptimes but i have not really an idea what exact cause this -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Mon Nov 21 22:22:58 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 22:22:58 +0200 Subject: [Dovecot] uid=error In-Reply-To: <4ECAB13D.3050501@schetterer.org> References: <4ECAB13D.3050501@schetterer.org> Message-ID: On 21.11.2011, at 22.14, Robert Schetterer wrote: >> This is the "info" message from mail_log plugin about what happened to mail. Besides this one, there should be another error message from lda saying what actually happened. The above informational message doesn't really say anything, except that a mail was first saved but then the save transaction was rollbacked. >> > Hi Timo, > just for info > i also have sporadic error like this > Nov 21 20:36:10 mail01 dovecot: lmtp(10398, user at user.de): save: > box=INBOX, uid=error, msgid=<20111121193610.69B4D200A2F at server.de>, > size=6356 > > as mails get delivered i dont care, i see > this happen sometimes during rsync backuptimes > but i have not really an idea what exact cause this Oh, I was wrong. The uid=error simply means that the UID is unknown for some reason. Possibly an internal bug.. The mail got saved just fine. Some backends like saving to a virtual mailbox will also give uid=error. So anyway, back to the original question: This log message doesn't indicate any error. The problem of some mail not being saved is somewhere else, possibly in MTA configuration? From tss at iki.fi Mon Nov 21 22:27:59 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 22:27:59 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECAAE5F.1040507@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> Message-ID: On 21.11.2011, at 22.02, Alexander Chekalin wrote: > I need a way to identify message across my mail box. The idea is that I can move messages between IMAP folders and the index still should be able to identify it. Message GUIDs are pretty good for that. > What I want to know, will UID strings be unique across all my messages no matter which IMAP folder it belongs now to, or the UID can change if I move message? If I remove dovecot-uidlist from the folder will UIDs of messages in the folder be changed? What happened if I put a message to the given IMAP folder (and dovecot-uidlist will record UID for it), then shut down dovecot, and (given maildir storage model) move the message's file info another folder in the same mail box - after I start dovecot, will the UID for the message be the same or not? With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). Assuming you're using Dovecot v2.x, when mail is copied to another mailbox its filename is preserved. So deleting dovecot* files won't lose the GUID. The only problem is that if you copy the same mail twice to another mailbox, it can't of course have the same filename twice, so Dovecot will assign it a new filename. But in a new enough version (probably v2.0.something) it still preserves the GUID by writing it to dovecot-uidlist file. In this situation if you delete the uidlist, the GUID changes to its filename. From tss at iki.fi Mon Nov 21 23:41:57 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:41:57 +0200 Subject: [Dovecot] Dovecot performance issues with many writes In-Reply-To: References: <306AC465-186A-4DDD-ACDC-AF1B892D0BD6@iki.fi> <742F699A-1ECE-4FBF-B12D-22DDDEB612E6@iki.fi> Message-ID: <1321911718.24607.51.camel@hurina> I don't see anything obviously bad in your setup. And since you said you're running Dovecot in the IMAP server, perhaps it's not even Dovecot at all that is causing the writes?.. Here are anyway some thoughts: * Log files? Make sure syslog isn't fsyncing each Dovecot log line to disk (or just set log_path = /var/log/dovecot.log and it definitely won't fsync) * Try if disabling index files lowers writes (I don't see why it would make a big difference): mail_location = maildir:~/Maildir:INDEX=MEMORY * Try if temporarily disabling Maildir++ quota makes any difference. There aren't really any other potential writers in Dovecot.. On Mon, 2011-11-21 at 12:34 -0600, Andy Robbins wrote: > Actually, some of the information I provided previously was incorrect. We > have four mail servers in our cluster which each have the NFS share > mounted. These four servers handle all SMTP and sending of mail. However, > it was thought that we could boost performance by simply running POP3 and > IMAP directly from the NFS server, since nothing besides Dovecot is > currently running there. So, here is the dovecot -n output from our > IMAP/POP3 server where the file system is local storage and is where we > believe the performance issues are originating. > > # dovecot -n > # 1.2.9: /etc/dovecot/dovecot.conf > Warning: fd limit 1024 is lower than what Dovecot can use under full load > (more than 2304). Either grow the limit or change login_max_processes_count > and max_mail_processes settings > # OS: Linux 2.6.32-24-server x86_64 Ubuntu 10.04.1 LTS > protocols: imap pop3 > ssl_cert_file: /etc/dovecot/smtp1.domain.com.cert > ssl_key_file: /etc/dovecot/smtp1.domain.com.key > disable_plaintext_auth: no > login_dir: /var/run/dovecot/login > login_executable(default): /usr/lib/dovecot/imap-login > login_executable(imap): /usr/lib/dovecot/imap-login > login_executable(pop3): /usr/lib/dovecot/pop3-login > login_greeting: Welcome > login_process_per_connection: no > login_processes_count: 2 > max_mail_processes: 2048 > mail_max_userip_connections: 50 > mail_uid: 500 > mail_gid: 500 > mail_location: maildir:~/Maildir > fsync_disable: yes > maildir_very_dirty_syncs: yes > mbox_write_locks: fcntl dotlock > mail_executable(default): /etc/dovecot/last-login-imap.sh > mail_executable(imap): /etc/dovecot/last-login-imap.sh > mail_executable(pop3): /etc/dovecot/last-login-pop3.sh > mail_plugins: quota > mail_plugin_dir(default): /usr/lib/dovecot/modules/imap > mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap > mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 > pop3_uidl_format: %f > auth default: > mechanisms: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 APOP > username_translation: %@ > verbose: yes > passdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > userdb: > driver: ldap > args: /etc/dovecot/dovecot-ldap.conf > plugin: > quota: maildir:User quota > quota_rule: *:storage=0 > > So you can see that we have already tried a number of performance > modifications such as setting fsync_disable: yes with little or no effect. > If I run nfsstat on the email servers where the volumes are mounted and > only running SMTP I get: > > # nfsstat > Client rpc stats: > calls retrans authrefrsh > 51762532 72668 0 > > Client nfs v3: > null getattr setattr lookup access > readlink > 0 0% 9639844 18% 3846547 7% 10495207 20% 11930633 23% 8 > 0% > read write create mkdir symlink > mknod > 7011062 13% 5372278 10% 1593131 3% 95 0% 0 0% 0 > 0% > remove rmdir rename link readdir > readdirplus > 830662 1% 0 0% 865484 1% 2576 0% 110845 0% 24963 > 0% > fsstat fsinfo pathconf commit > 2074 0% 8 0% 0 0% 37113 0% > > My apologies for the somewhat garbled output, there, but I it seems to me > that we're not witnessing the same higher IO writes as compared to reads. > > Thanks for your help so far. Hopefully, this sheds more light on your > previous questions. > > > > On Thu, Nov 17, 2011 at 4:42 PM, Timo Sirainen wrote: > > > Do you have more than one Dovecot server? If only one, you can still > > disable the mail_nfs_* settings. Also it would be interesting to see > > nfsstat numbers from the Dovecot server, compared to those iostat numbers.. > > > > On 18.11.2011, at 0.38, Andy Robbins wrote: > > > > > Well, the iostat command was run from the NFS server and dovecot was run > > from the mail server where it is mounted, hence the discrepancy there. > > > > > > > > > On Thu, Nov 17, 2011 at 4:34 PM, Timo Sirainen wrote: > > > On 18.11.2011, at 0.20, Andy Robbins wrote: > > > > > > > # iostat -d 5 -x > > > > Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s > > avgrq-sz > > > > avgqu-sz await svctm %util > > > > dm-4 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > > 8.00 28.69 19.63 0.70 94.00 > > > > drbd0 0.00 0.00 485.80 865.80 3886.40 6926.40 > > > > 8.00 111.42 94.44 0.74 99.60 > > > > > > These devices don't look like NFS, but you have: > > > > > > > mmap_disable: yes > > > > mail_nfs_storage: yes > > > > mail_nfs_index: yes > > > > > > So what filesystem is that? Are they storing only mails, or other stuff > > too (like logs)? The mail_nfs_*=yes are probably not needed and are slowing > > things up at least somewhat.. > > > > > > > lda: > > > > fsync_disable: no > > > > > > This is explicitly set for LDA, but it's the default already elsewhere. > > Maybe you wanted to try fsync_disable=yes elsewhere except in LDA? > > > > > > > > > > > > > From tss at iki.fi Mon Nov 21 23:45:39 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:45:39 +0200 Subject: [Dovecot] Dovecot Log analysis In-Reply-To: <20111121101941.GA25724@parmesan.sis.pasteur.fr> References: <20111121101941.GA25724@parmesan.sis.pasteur.fr> Message-ID: <1321911939.24607.55.camel@hurina> On Mon, 2011-11-21 at 11:19 +0100, Thomas Hummel wrote: > Hello, > > Before writing them myself, I'd like to figure out if some tools exist (anvil ? > ...) and how to use them to extract from dovecot logs things like who is doing > what and how hard or how fast in order to do some rate limiting. For instance, > I'd like to be able to identify, slow or block users who > > . read or write to much / slice of time > > . login/logout too often / slice of time > > Note : I'm still using the 1.2.x branch. The reading/writing rate limiting could be a bit tricky. I guess with Linux you could look into /proc//io files for all imap/pop3 processes and determine from there which processes are doing too much. For pop3 there's also http://dovecot.org/patches/1.2/pop3-throttle-plugin.c Other than that, I don't know of any existing stuff for v1.2. For v2.1 there's the new "stats" service and you can ask all kinds of statistics from it, including what users/processes have used the most disk IO. From pw at wk-serv.de Mon Nov 21 23:45:49 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Mon, 21 Nov 2011 22:45:49 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <1321472837.21919.598.camel@hurina> References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> Message-ID: <4ECAC68D.3060007@wk-serv.de> Timo Sirainen schrieb: > On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: >> I already use lmtp:unix:private/dovecot-lmtp as transport but where is >> the link to the indexes? > > You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot > LMTP proxy, which would forward the connection to the backend server > which handles that user's IMAP/POP3/LMTP connections. I don't know if we're talking about the same :) From achekalin at lazurit.com Mon Nov 21 23:47:15 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 00:47:15 +0300 Subject: [Dovecot] Unique message IDs? In-Reply-To: References: <4ECAAE5F.1040507@lazurit.com> Message-ID: <4ECAC6E3.9090307@lazurit.com> > Message GUIDs are pretty good for that. Oh, thank you! Nice news! > With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). But what if I one day decide to convert my maildir's to mbox'es? I really plan to do such conversion in a while (as soon as I finish the indexing system). Yours, Alexander From tss at iki.fi Mon Nov 21 23:52:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:52:28 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4ECA174A.1020909@informatik.uni-freiburg.de> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> Message-ID: <1321912348.24607.58.camel@hurina> On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: > Hello, > after serveral tests (and reading a lot of howto's) I'm now fairly convinced > that the Solaris HW-crypto engine is not automatically used. Even when dovecot > ist compiled with the OpenSSL version provided by Solaris. > > Currently I have only a T1-CPU available for testing (Sun Fire T2000) and > after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with > > ENGINE *e; > ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); Does "openssl engine" return this "pkcs11" string? > ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); > ENGINE_set_default_ciphers(e); > > in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() > the crypto device gets used. I'm sure that this is not the whole story since > this only seems to affect the IMAP login. It should work for POP3 as well, all of the SSL code is shared. > One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in > the various articles). Is there any chance that Dovecot gets updated/patched > for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. I guess I could add ssl_crypto_device setting for this. But I'll need to figure out proper ifdefs to avoid compile failures with older OpenSSL versions. From tss at iki.fi Mon Nov 21 23:56:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Mon, 21 Nov 2011 23:56:46 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECAC6E3.9090307@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> Message-ID: <1321912606.24607.61.camel@hurina> On Tue, 2011-11-22 at 00:47 +0300, Alexander Chekalin wrote: > > With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). > > But what if I one day decide to convert my maildir's to mbox'es? > I really plan to do such conversion in a while (as soon as I finish the > indexing system). mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of fakes it by returning MD5 of specific headers as GUIDs (so not 100% reliable). With mdbox GUIDs work even better than with Maildir, the GUID is always stored in the message's metadata. From hoogendyk at bio.umass.edu Tue Nov 22 00:26:58 2011 From: hoogendyk at bio.umass.edu (Chris Hoogendyk) Date: Mon, 21 Nov 2011 17:26:58 -0500 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1321912348.24607.58.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> Message-ID: <4ECAD032.6060902@bio.umass.edu> On 11/21/11 4:52 PM, Timo Sirainen wrote: > On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: >> Hello, >> after serveral tests (and reading a lot of howto's) I'm now fairly convinced >> that the Solaris HW-crypto engine is not automatically used. Even when dovecot >> ist compiled with the OpenSSL version provided by Solaris. >> >> Currently I have only a T1-CPU available for testing (Sun Fire T2000) and >> after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with >> >> ENGINE *e; >> ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); > Does "openssl engine" return this "pkcs11" string? > >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >> ENGINE_set_default_ciphers(e); >> >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >> the crypto device gets used. I'm sure that this is not the whole story since >> this only seems to affect the IMAP login. > It should work for POP3 as well, all of the SSL code is shared. > >> One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in >> the various articles). Is there any chance that Dovecot gets updated/patched >> for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. > I guess I could add ssl_crypto_device setting for this. But I'll need to > figure out proper ifdefs to avoid compile failures with older OpenSSL > versions. Just a quick comment -- the Sun instructions for setting up Apache specifically noted the importance of specifying "SSLCryptoDevice pkcs11" in httpd.conf. So, the OpenSSL provided by Solaris will do what it should, IFF pkcs11 is specified by the program that is calling it. I had an email exchange with a Solaris developer at some point, who said that GnuPG simply could not use the crypto device, because it was built with libcrypt, did not use the pkcs11 engine, and it would take a significant rewrite to make it do so. -- --------------- Chris Hoogendyk - O__ ---- Systems Administrator c/ /'_ --- Biology& Geology Departments (*) \(*) -- 140 Morrill Science Center ~~~~~~~~~~ - University of Massachusetts, Amherst --------------- Erd?s 4 From preen at informatik.uni-freiburg.de Tue Nov 22 00:49:28 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Mon, 21 Nov 2011 23:49:28 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1321912348.24607.58.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> Message-ID: <4ECAD578.4060304@informatik.uni-freiburg.de> Timo Sirainen wrote: > On Mon, 2011-11-21 at 10:18 +0100, Martin Preen wrote: >> Hello, >> after serveral tests (and reading a lot of howto's) I'm now fairly convinced >> that the Solaris HW-crypto engine is not automatically used. Even when dovecot >> ist compiled with the OpenSSL version provided by Solaris. >> >> Currently I have only a T1-CPU available for testing (Sun Fire T2000) and >> after patching src/login-common/ssl-proxy-openssl.c (Dovecot 1.2.17) with >> >> ENGINE *e; >> ENGINE_load_builtin_engines(); ENGINE_init((e=ENGINE_by_id("pkcs11"))); > > Does "openssl engine" return this "pkcs11" string? This requires the Solaris OpenSSL version (or another version using the pkcs11 patch). # /usr/sfw/bin/openssl engine (pkcs11) PKCS #11 engine support >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >> ENGINE_set_default_ciphers(e); >> >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >> the crypto device gets used. I'm sure that this is not the whole story since >> this only seems to affect the IMAP login. > > It should work for POP3 as well, all of the SSL code is shared. I couldn't find the EncryptUpdate call which has to be changed too (due to the howto documents). Maybe some other call needs e patch. But I don't know which. Martin >> One has to use the specific SSL-engine and the ENGINE/EVP calls (as stated in >> the various articles). Is there any chance that Dovecot gets updated/patched >> for this ? E.g. Like the SSLCryptoDevice setting in Apache's mod_ssl. > > I guess I could add ssl_crypto_device setting for this. But I'll need to > figure out proper ifdefs to avoid compile failures with older OpenSSL > versions. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6350 bytes Desc: S/MIME Cryptographic Signature URL: From Damon_Atkins at yahoo.com.au Tue Nov 22 04:17:39 2011 From: Damon_Atkins at yahoo.com.au (Damon Atkins) Date: Tue, 22 Nov 2011 13:17:39 +1100 Subject: [Dovecot] Solaris hardware crypto engines Message-ID: <4ECB0643.7070308@yahoo.com.au> Here are some blogs on the topic. http://wikis.sun.com/display/CryptoPerf/Using+the+UltraSPARC+cryptographic+accelerators Solaris 10 # /usr/sfw/bin/openssl engine -c -t # cc -fast*-I /usr/sfw/include -L /usr/sfw/lib -lcrypto* aes_test.c -o aes_test.out http://blogs.oracle.com/DanX/entry/sparc_t4_openssl_engine http://blogs.oracle.com/DanX/entry/where_s_the_crypto_libraries http://blogs.oracle.com/DanX/entry/solaris_x86_aesni_openssl_engine http://blogs.oracle.com/chichang1/entry/rsa_performance_of_sun_fire Here is some info from my intel box Solaris 11 # /usr/bin/openssl engine -c -t (aesni) Intel AES-NI engine (no-aesni) % no-aesni means no aes H/W acceleration [ available ] (dynamic) Dynamic engine loading support [ unavailable ] (pkcs11) PKCS #11 engine support [RSA, DSA, DH, RAND, DSA] [ available ] $ isainfo -v # My cpu does not have 'aes' support 64-bit amd64 applications cx16 sse3 sse2 sse fxsr mmx cmov amd_sysc cx8 tsc fpu 32-bit i386 applications ahf cx16 sse3 sse2 sse fxsr mmx cmov sep cx8 tsc fpu # ldd /opt/dovecot/libexec/dovecot/ssl-build-param **** libssl.so.1.0.0 => /lib/libssl.so.1.0.0 *** *** libcrypto.so.1.0.0 => /lib/libcrypto.so.1.0.0* *** libc.so.1 => /lib/libc.so.1 libnsl.so.1 => /lib/libnsl.so.1 libsocket.so.1 => /lib/libsocket.so.1 librt.so.1 => /lib/librt.so.1 libsendfile.so.1 => /lib/libsendfile.so.1 libmp.so.2 => /lib/libmp.so.2 libmd.so.1 => /lib/libmd.so.1 libm.so.2 => /lib/libm.so.2 ./configure --prefix=/opt/dovecot --with-ldap=yes --with-gssapi --with-ssldir=/etc/openssl Install prefix . : /opt/dovecot File offsets ... : 64bit I/O polling .... : poll I/O notifys .... : none SSL ............ : yes (OpenSSL) GSSAPI ......... : yes passdbs ........ : passwd passwd-file shadow pam checkpassword ldap : -bsdauth -sia -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword ldap : -sql -vpopmail -nss SQL drivers .... : : -pgsql -mysql -sqlite Note Under OpenSolaris I did the following: CPPFLAGS=-I/usr/sfw/include LDFLAGS=-R/usr/sfw/lib ./configure --prefix=/opt/dovecot --with-ldap=yes --with-gssapi --with-ssldir=/etc/openssl (most likely Solaris 10 is the same as above, openssl may look old but I believe it is patched with compatible *fixes* from current openssl so application do not break. Apparently it took 5mths to update Solaris 11 to OpenSSL 1.0 and test everything) Cheers Damon. From achekalin at lazurit.com Tue Nov 22 07:39:30 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 08:39:30 +0300 Subject: [Dovecot] Unique message IDs? In-Reply-To: <1321912606.24607.61.camel@hurina> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> Message-ID: <4ECB3592.1010500@lazurit.com> >>> With Maildir the message GUID is typically the same as the Maildir base filename (i.e. everything before ':' character). >> >> But what if I one day decide to convert my maildir's to mbox'es? >> I really plan to do such conversion in a while (as soon as I finish the >> indexing system). > > mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of > fakes it by returning MD5 of specific headers as GUIDs (so not 100% > reliable). With mdbox GUIDs work even better than with Maildir, the GUID > is always stored in the message's metadata. I'd like to use the best optimized one (mdbox), but there is a reason not to do that is when I use mbox or maildir I can see where given IMAP folder mails are stored, so, say, if I want to copy only one IMAP folder to some remote site, I just copy know dir or file. With mdbox this is different, I simply can not guess where my messages exactly are. If it be possible to have per-IMAP-folders mdboxes, I love to use it. But from what I know and tried this is not way mdbox used to work, right? Yours, Alexander Chekalin From janfrode at tanso.net Tue Nov 22 09:16:24 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 22 Nov 2011 08:16:24 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4ECAC68D.3060007@wk-serv.de> References: <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> Message-ID: <20111122071624.GA6110@dibs.tanso.net> On Mon, Nov 21, 2011 at 10:45:49PM +0100, Patrick Westenberg wrote: > Timo Sirainen schrieb: > >On Wed, 2011-11-16 at 19:40 +0100, Patrick Westenberg wrote: > >>I already use lmtp:unix:private/dovecot-lmtp as transport but where is > >>the link to the indexes? > > > >You can switch to lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be Dovecot > >LMTP proxy, which would forward the connection to the backend server > >which handles that user's IMAP/POP3/LMTP connections. > > I don't know if we're talking about the same :) > I wondered that too. It looked to me like you tried to ask where the lmtp-service picks up the path to indexes, right? AFAIU it picks that up from the /var/run/dovecot/auth-master socket. -jf From achekalin at lazurit.com Tue Nov 22 11:10:12 2011 From: achekalin at lazurit.com (Alexander Chekalin) Date: Tue, 22 Nov 2011 12:10:12 +0300 Subject: [Dovecot] No duplicates on imap copy? Message-ID: <4ECB66F4.7060204@lazurit.com> Hello, when from my script I try to copy messages from one imap folder into another own (within the same imap mailbox). That works perfectly, but... If I copy the same message N times from one folder into another one, in the destination folder I see N copies of the same message, each with its own UID. I wonder if there any way to deduplicate folder content (so in the destination folder I'll get only one copy of each message)? I'd implement something like that myself in my script but I just don't know which part of the messages should I compare - probably md5 of headers? Thank you in advance! Yours, Alexander Chekalin From tss at iki.fi Tue Nov 22 11:58:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 11:58:46 +0200 Subject: [Dovecot] No duplicates on imap copy? In-Reply-To: <4ECB66F4.7060204@lazurit.com> References: <4ECB66F4.7060204@lazurit.com> Message-ID: <74E4AA7F-089C-4FF4-BE74-F80E8D1848D3@iki.fi> On 22.11.2011, at 11.10, Alexander Chekalin wrote: > when from my script I try to copy messages from one imap folder into another own (within the same imap mailbox). That works perfectly, but... If I copy the same message N times from one folder into another one, in the destination folder I see N copies of the same message, each with its own UID. Right. > I wonder if there any way to deduplicate folder content (so in the destination folder I'll get only one copy of each message)? Not any easy way currently. > I'd implement something like that myself in my script but I just don't know which part of the messages should I compare - probably md5 of headers? What Dovecot version and what mailbox format? From tss at iki.fi Tue Nov 22 12:01:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 12:01:28 +0200 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECB3592.1010500@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> <4ECB3592.1010500@lazurit.com> Message-ID: <275436D2-0816-47E2-877C-FC55AC43B67D@iki.fi> On 22.11.2011, at 7.39, Alexander Chekalin wrote: > I'd like to use the best optimized one (mdbox), but there is a reason not to do that is when I use mbox or maildir I can see where given IMAP folder mails are stored, so, say, if I want to copy only one IMAP folder to some remote site, I just copy know dir or file. With mdbox this is different, I simply can not guess where my messages exactly are. > > If it be possible to have per-IMAP-folders mdboxes, I love to use it. But from what I know and tried this is not way mdbox used to work, right? sdbox would work like that. The reason mdbox doesn't work like that is because copying messages would be rather slow then. The idea with mdbox is anyway that you'd use Dovecot's tools to manage the mailboxes rather than access them directly through filesystem. So if you want to copy one IMAP folder, you'd use either dsync or doveadm import to do it. From pw at wk-serv.de Tue Nov 22 12:17:12 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 22 Nov 2011 11:17:12 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <20111122071624.GA6110@dibs.tanso.net> References: <4EB0624F.90307@wk-serv.de> <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> Message-ID: <4ECB76A8.80208@wk-serv.de> Jan-Frode Myklebust schrieb: > I wondered that too. It looked to me like you tried to ask where the > lmtp-service picks up the path to indexes, right? AFAIU it picks that up > from the /var/run/dovecot/auth-master socket. No. I want to know if dovecot writes to the indexes if it receives a mail via lmtp. Someone proposed to store the index files on a locally installed SSD on a frontend (imap) machine and stick the users to that machine but if the lmtp-service writes to the indexes (and I think he does), that machine needs access to the indexes too which will bring us back to shared storage. From janfrode at tanso.net Tue Nov 22 12:45:47 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 22 Nov 2011 11:45:47 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <4ECB76A8.80208@wk-serv.de> References: <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> Message-ID: <20111122104547.GA10680@dibs.tanso.net> On Tue, Nov 22, 2011 at 11:17:12AM +0100, Patrick Westenberg wrote: > > No. I want to know if dovecot writes to the indexes if it receives a > mail via lmtp. > > Someone proposed to store the index files on a locally installed SSD > on a frontend (imap) machine and stick the users to that machine but > if the lmtp-service writes to the indexes (and I think he does), > that machine needs access to the indexes too which will bring us > back to shared storage. Ah, then Timo's reply was right. He suggested you do the lmtp-deliveries to the same server that you would send you imap-user to. You can do this trough dovecot director and lmtp-proxying. So instead of: lmtp:unix:private/dovecot-lmtp you should use: lmtp:tcp:1.2.3.4:24 where 1.2.3.4 would be the Dovecot LMTP proxy that proxies to the same machine as you would use for imap for this particular recipient. -jf From stan at hardwarefreak.com Tue Nov 22 19:00:08 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 22 Nov 2011 11:00:08 -0600 Subject: [Dovecot] 1.2.15 can't get zlib working Message-ID: <4ECBD518.9020206@hardwarefreak.com> Following these instructions: http://wiki.dovecot.org/Plugins/Zlib I enabled zlib and restarted Dovecot. I made a backup copy of a 68MB mbox file and moved it to a non mail directory. The mbox file is an existing TBird IMAP folder in my UNIX user mail directory. It is an archive of a defunct mailing list. It was a regular IMAP folder prior to attempting this. I was able to access all emails in the folder. It worked fine. I gzipped the original mbox file down to 14MB and removed the write flag with chmod. I launched TBird and received the following error in Activity Manager when accessing this IMAP folder: The current operation on '1-Spam-l' did not succeed. The mail server for account stan at hardwarefreak.com responded: Mailbox doesn't exist: 1-Spam-l I deleted the Dovecot index files and the TBird 1-Spam-l.msf file which didn't help. Curiously, there are no entries in either dovecot.err or dovecot.log, giving not even the slightest hint as to where I should troubleshoot. Kinda sucks when one follows instructions exactly, simple instructions at that, and it doesn't work, and generates zero log errors. A bit frustrating... Any ideas? /$ la /home/stan/mail/1-Spam-l.gz -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_plugins: fts fts_squat zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster at hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10 -- Stan From mrvjtod at gmail.com Tue Nov 22 20:24:15 2011 From: mrvjtod at gmail.com (Chris Young) Date: Tue, 22 Nov 2011 13:24:15 -0500 Subject: [Dovecot] disable imap info logging Message-ID: Is there a way to disable the imap info logging? My logs are filling with tons of login/logout messages Nov 22 18:23:25 imap-login: Info: Login: user=, method={method}, rip=10.86.189.79, lip=10.86.189.82, mpid=22488 Nov 22 18:23:25 imap(USER1): Info: Disconnected: Logged out bytes=54/726 From tss at iki.fi Tue Nov 22 20:28:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 20:28:56 +0200 Subject: [Dovecot] disable imap info logging In-Reply-To: References: Message-ID: <4A94BBF7-5704-4542-9796-942B50940367@iki.fi> On 22.11.2011, at 20.24, Chris Young wrote: > Is there a way to disable the imap info logging? > > My logs are filling with tons of login/logout messages > > Nov 22 18:23:25 imap-login: Info: Login: user=, method={method}, > rip=10.86.189.79, lip=10.86.189.82, mpid=22488 > Nov 22 18:23:25 imap(USER1): Info: Disconnected: Logged out bytes=54/726 Rotate the logs and have it delete old ones? Or simply: info_log_path = /dev/null From tss at iki.fi Tue Nov 22 20:30:33 2011 From: tss at iki.fi (Timo Sirainen) Date: Tue, 22 Nov 2011 20:30:33 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECBD518.9020206@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> Message-ID: On 22.11.2011, at 19.00, Stan Hoeppner wrote: > The current operation on '1-Spam-l' did not succeed. The mail server > for account stan at hardwarefreak.com responded: Mailbox doesn't exist: > 1-Spam-l .. > /$ la /home/stan/mail/1-Spam-l.gz > -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) From Juergen.Obermann at hrz.uni-giessen.de Tue Nov 22 22:11:02 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Tue, 22 Nov 2011 21:11:02 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC Message-ID: Hello, compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC with Sun Studio 11 stops with the following error: Making all in lib-imap-client gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-imap-client' source='imapc-client.c' object='imapc-client.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-client.lo imapc-client.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-client.c -KPIC -DPIC -o .libs/imapc-client.o "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-client.c -o imapc-client.o >/dev/null 2>&1 source='imapc-connection.c' object='imapc-connection.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ /bin/bash ../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-connection.lo imapc-connection.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-ssl-iostream -I../../src/lib-mail -I../../src/lib-imap -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-connection.c -KPIC -DPIC -o .libs/imapc-connection.o "../../src/lib-imap/imap-util.h", line 6: warning: useless declaration "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) "imapc-connection.c", line 1446: operand cannot have void type: op "==" "imapc-connection.c", line 1446: operands have incompatible types: const void "==" pointer to struct imapc_command {pointer to struct pool {..} pool, pointer to struct buffer {..} data, unsigned int send_pos, unsigned int tag, enum imapc_command_flags {IMAPC_COMMAND_FLAG_RETRIABLE(4), IMAPC_COMMAND_FLAG_PRELOGIN(2), IMAPC_COMMAND_FLAG_SELECT(1)} flags, pointer to struct imapc_connection {..} conn, pointer to struct imapc_client_mailbox {..} box, union {..} streams, pointer to function(pointer to const struct imapc_command_reply {..}, pointer to void) returning void callback, pointer to void context, unsigned int idle :1, unsigned int wait_for_literal :1} cc: acomp failed for imapc-connection.c gmake[3]: *** [imapc-connection.lo] Error 1 gmake[3]: Leaving directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-imap-client' Viele Gr??e, J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From ramiblanco at gmail.com Tue Nov 22 22:52:38 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Tue, 22 Nov 2011 17:52:38 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search Message-ID: Hi, I found that doing a wildcard search with doveadm doesn't expand the %d global variable when doing a wildcard search like: doveadm search *@domain ALL LARGER 10000k doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users In /var/log/dovecot.log: Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 22 20:48:07 auth-worker: Error: ldap(?): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object Is this a bug? Configs: #file /usr/local/etc/dovecot/dovecot-ldap.conf.ext: hosts = x.x.x.x dn = cn=dovecot,dc=domain dnpass = mypass auth_bind = yes auth_bind_userdn = mail=%u,vd=%d,dc=domain ldap_version = 3 base = vd=%d,dc=domain deref = never scope = subtree user_attrs = mailbox=mail=maildir:/home/vmail/domains/%$,quota=quota_rule=*:storage=%$,vdHome=home=%$/%d/%1u/%1.1u/%n,=uid=504,=gid=12 user_filter = (&(&(objectClass=VirtualMailAccount)(mail=%u))(accountActive=TRUE)) pass_attrs = userPassword=password,mail=user pass_filter = (&(&(objectClass=VirtualMailAccount)(mail=%u))(accountActive=TRUE)) iterate_attrs = mail=user iterate_filter = (&(objectClass=VirtualMailAccount)(accountActive=TRUE)) default_pass_scheme = CRYPT # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.3.1.el5 x86_64 CentOS release 5.5 (Final) ext3 auth_mechanisms = plain login dict { expire = pgsql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } hostname = localhost lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot.log mail_gid = 12 mail_location = maildir:/home/vmail/domains/%d/%1u/%1.1u/%n mail_plugins = quota autocreate expire mail_privileged_group = mail mail_uid = 504 managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { autocreate = Enviados autocreate2 = Papelera autocreate3 = Borrador autocreate4 = Spam autosubscribe = Enviados autosubscribe2 = Papelera autosubscribe3 = Borrador autosubscribe4 = Spam expire = Papelera expire2 = Papelera/* expire3 = Spam expire4 = Spam/* expire_dict = proxy::expire quota = maildir:User quota quota_rule = *:storage=100M quota_rule2 = Papelera:storage=+100M sieve = ~/sieve/dovecot.sieve sieve_before = /usr/local/etc/sieve/spam.sieve sieve_dir = ~/sieve sieve_global_dir = /usr/local/etc/sieve/ } postmaster_address = postmaster at d omain protocols = imap sieve pop3 service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } vsz_limit = 64 M } ssl_cert = References: <4ECBD518.9020206@hardwarefreak.com> Message-ID: <4ECC2149.4070403@hardwarefreak.com> On 11/22/2011 12:30 PM, Timo Sirainen wrote: > On 22.11.2011, at 19.00, Stan Hoeppner wrote: > >> The current operation on '1-Spam-l' did not succeed. The mail server >> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >> 1-Spam-l > .. >> /$ la /home/stan/mail/1-Spam-l.gz >> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz > > The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) Aha. That was it. Thanks Timo. For some reason my read of the wiki page made me think this was handled transparently--just zip the file and everything works as it did before. Apparently it's not as simple as the wiki leads one (me anyway) to believe. This bit of the wiki caused me some confusion as well: "Compressed mbox files can be accessed only as read-only" Thus I chmod'ed the .gz file to read-only. This creates a problem. It appears that when Dovecot creates the .imap folder of the same name it inherits the permissions of the zipped mbox file. Thus it can't create the indexes: 2011-11-22 14:59:23 IMAP(stan): Error: file_dotlock_open(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log) failed: Permission denied (euid=1000(stan) egid=1000(stan) access(/home/stan/mail/.imap/1-Spam-l.gz/dovecot.index.log, 4) failed: No such file or directory) Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki could be reworded in a way that doesn't prompt some folks to manually make the zipped files read-only. It took a while for Dovecot to index the 15K+ messages. With that finished, accessing the folder is similar to before, but there's a small lag when opening messages. As this is an archive folder the contents won't change, so Squat FTS should be very fast after the first search, just as before. Interestingly, it appears my squat indexes aren't updating--for any folder. I've searched 4 folders via Tbird body search with xyzzyx (took forever) and I see no changes to the dates or sizes of existing indexes. I deleted the squat indexes for one folder and ran the search again. No new squat indexes were created. No errors in the logs. Any ideas why the squat indexes aren't updating? IIRC this happened once before and I was able to fix it. Don't recall how I did it though.... Once again: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.6 i686 Debian 6.0.3 xfs log_path: /var/log/dovecot.err info_log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap ssl_parameters_regenerate: 0 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login login_process_per_connection: no login_process_size: 16 login_processes_count: 1 login_max_processes_count: 1 login_max_connections: 8 max_mail_processes: 4 mail_privileged_group: mail mail_location: mbox:/home/%u/mail:INBOX=/var/mail/%u mailbox_idle_check_interval: 15 mbox_write_locks: fcntl mbox_very_dirty_syncs: yes mbox_lazy_writes: no mail_process_size: 320 mail_plugins: zlib imap_client_workarounds: tb-extra-mailbox-sep lda: postmaster_address: postmaster at hardwarefreak.com mail_plugins: sieve auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam args: max_requests=1 userdb: driver: passwd plugin: fts: squat fts_squat: partial=4 full=10 Thanks. -- Stan From dovecot at knutejohnson.com Wed Nov 23 00:53:30 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Tue, 22 Nov 2011 14:53:30 -0800 Subject: [Dovecot] How to disable pop3 in version 2? Message-ID: <4ECC27EA.2000200@knutejohnson.com> I'm about to bring up a new mail server running Ubuntu 11.10 with Dovecot 2. I've been using 1.2 all along and there are a bunch more config files in 2. In 1.2 I just edited the protocols statement but I can't find one in version 2. Is there a way to disable pop3 or do I just firewall the ports? And a related question, I've been using imaps on port 993 but I hear that's not the best solution anymore, I should be using imap with STARTTLS? How do I disable imaps? Thanks, -- Knute Johnson From tss at iki.fi Wed Nov 23 00:56:10 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 00:56:10 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: Message-ID: On 22.11.2011, at 22.52, Ramiro Blanco wrote: > Hi, I found that doing a wildcard search with doveadm doesn't expand the %d > global variable when doing a wildcard search like: > > Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain > scope=subtree Iteration is supposed to iterate through all users.. > Is this a bug? More like a missing feature I guess. I guess it could make it expand the %d if it's known.. From tss at iki.fi Wed Nov 23 01:01:44 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:01:44 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC In-Reply-To: References: Message-ID: On 22.11.2011, at 22.11, J?rgen Obermann wrote: > "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) What? No it doesn't.. Does changing the 0x400000000 in line 20 to 0x200000000 help? > "imapc-connection.c", line 1446: operand cannot have void type: op "==" > "imapc-connection.c", line 1446: operands have incompatible types: Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1cdd39d11ce4 From tss at iki.fi Wed Nov 23 01:03:35 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:03:35 +0200 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <4ECC27EA.2000200@knutejohnson.com> References: <4ECC27EA.2000200@knutejohnson.com> Message-ID: <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> On 23.11.2011, at 0.53, Knute Johnson wrote: > I'm about to bring up a new mail server running Ubuntu 11.10 with Dovecot 2. I've been using 1.2 all along and there are a bunch more config files in 2. In 1.2 I just edited the protocols statement but I can't find one in version 2. Is there a way to disable pop3 or do I just firewall the ports? In the example dovecot.conf there's a protocols setting. If you don't see any, you can just add your own wherever. "protocols=imap" should work. > And a related question, I've been using imaps on port 993 but I hear that's not the best solution anymore, I should be using imap with STARTTLS? How do I disable imaps? Well, http://wiki2.dovecot.org/SSL explains this more. I don't think you really need to disable imaps, but if you want to, you can do it with: service imap-login { inet_listener imaps { port = 0 } } From tss at iki.fi Wed Nov 23 01:07:28 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 01:07:28 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECC2149.4070403@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> Message-ID: <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> On 23.11.2011, at 0.25, Stan Hoeppner wrote: > On 11/22/2011 12:30 PM, Timo Sirainen wrote: >> On 22.11.2011, at 19.00, Stan Hoeppner wrote: >> >>> The current operation on '1-Spam-l' did not succeed. The mail server >>> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >>> 1-Spam-l >> .. >>> /$ la /home/stan/mail/1-Spam-l.gz >>> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz >> >> The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) > > Aha. That was it. Thanks Timo. For some reason my read of the wiki > page made me think this was handled transparently--just zip the file and > everything works as it did before. Apparently it's not as simple as the > wiki leads one (me anyway) to believe. I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :) > This bit of the wiki caused me some confusion as well: > "Compressed mbox files can be accessed only as read-only" > > Thus I chmod'ed the .gz file to read-only. This creates a problem. It > appears that when Dovecot creates the .imap folder of the same name it > inherits the permissions of the zipped mbox file. Thus it can't create > the indexes: I've fixed this in some version. I guess in v2.0. > Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki > could be reworded in a way that doesn't prompt some folks to manually > make the zipped files read-only. Well, I don't really care that much about v1.x anymore. > It took a while for Dovecot to index the 15K+ messages. With that > finished, accessing the folder is similar to before, but there's a small > lag when opening messages. Yeah, it's uncompressing the entire file until it finds the message you're opening. > As this is an archive folder the contents > won't change, so Squat FTS should be very fast after the first search, > just as before. Interestingly, it appears my squat indexes aren't > updating--for any folder. I've searched 4 folders via Tbird body search > with xyzzyx (took forever) and I see no changes to the dates or sizes of > existing indexes. I deleted the squat indexes for one folder and ran > the search again. No new squat indexes were created. No errors in the > logs. > > Any ideas why the squat indexes aren't updating? IIRC this happened > once before and I was able to fix it. Don't recall how I did it though.... > mail_plugins: zlib Doesn't look like fts, fts_squat is enabled? From dovecot at knutejohnson.com Wed Nov 23 02:40:31 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Tue, 22 Nov 2011 16:40:31 -0800 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> Message-ID: <4ECC40FF.8020103@knutejohnson.com> On 11/22/2011 3:03 PM, Timo Sirainen wrote: > On 23.11.2011, at 0.53, Knute Johnson wrote: > >> I'm about to bring up a new mail server running Ubuntu 11.10 with >> Dovecot 2. I've been using 1.2 all along and there are a bunch >> more config files in 2. In 1.2 I just edited the protocols >> statement but I can't find one in version 2. Is there a way to >> disable pop3 or do I just firewall the ports? > > In the example dovecot.conf there's a protocols setting. If you don't > see any, you can just add your own wherever. "protocols=imap" should > work. > >> And a related question, I've been using imaps on port 993 but I >> hear that's not the best solution anymore, I should be using imap >> with STARTTLS? How do I disable imaps? > > Well, http://wiki2.dovecot.org/SSL explains this more. I don't think > you really need to disable imaps, but if you want to, you can do it > with: > > service imap-login { inet_listener imaps { port = 0 } } > > Thanks for that. The service imap-login line above is in the file /etc/dovecot/conf.d/10-master.conf. I don't know if that is standard for version 2 or something that Ubuntu created. The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; !include_try /usr/share/dovecot/protocols.d/*.protocol That directory contains one file; dovecot-imapd.protocol and it contains one line; protocols = $protocols imap So I am assuming that it is adding imap to the protocols in the environment variable $protocols somehow? dovecot -n yields; protocols = " imap" I did get TBird to retrieve mail from port 993 using SSL however. Will it work without a protocols = imaps? Thanks, -- Knute Johnson From stan at hardwarefreak.com Wed Nov 23 02:56:29 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 22 Nov 2011 18:56:29 -0600 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> Message-ID: <4ECC44BD.3080806@hardwarefreak.com> On 11/22/2011 5:07 PM, Timo Sirainen wrote: > On 23.11.2011, at 0.25, Stan Hoeppner wrote: > >> On 11/22/2011 12:30 PM, Timo Sirainen wrote: >>> On 22.11.2011, at 19.00, Stan Hoeppner wrote: >>> >>>> The current operation on '1-Spam-l' did not succeed. The mail server >>>> for account stan at hardwarefreak.com responded: Mailbox doesn't exist: >>>> 1-Spam-l >>> .. >>>> /$ la /home/stan/mail/1-Spam-l.gz >>>> -r-------- 1 stan stan 14M Oct 28 2010 /home/stan/mail/1-Spam-l.gz >>> >>> The name is now "1-Spam-l.gz", not "1-Spam-l". (Subscription file not updated?) >> >> Aha. That was it. Thanks Timo. For some reason my read of the wiki >> page made me think this was handled transparently--just zip the file and >> everything works as it did before. Apparently it's not as simple as the >> wiki leads one (me anyway) to believe. > > I thought about doing something smarter, but then I thought "no one uses compressed mboxes for anything important anyway" :) Out of curiosity, what (or who) prompted the development of the compressed mbox feature? Or was it that you wanted to do it for maildir, and then figured you should for mbox as well? The implementation seems to work ok. The instructions just seem a bit...thin. :) >> This bit of the wiki caused me some confusion as well: >> "Compressed mbox files can be accessed only as read-only" >> >> Thus I chmod'ed the .gz file to read-only. This creates a problem. It >> appears that when Dovecot creates the .imap folder of the same name it >> inherits the permissions of the zipped mbox file. Thus it can't create >> the indexes: > > I've fixed this in some version. I guess in v2.0. Yeah, I'm waiting for Debian to get a backport of 2.0.x. For many reasons the only thing I'm comfortable installing from source is the Linux kernel. >> Reverting with 'chmod +w' fixed this. Maybe that sentence in the wiki >> could be reworded in a way that doesn't prompt some folks to manually >> make the zipped files read-only. > > Well, I don't really care that much about v1.x anymore. Understandable. >> It took a while for Dovecot to index the 15K+ messages. With that >> finished, accessing the folder is similar to before, but there's a small >> lag when opening messages. > > Yeah, it's uncompressing the entire file until it finds the message you're opening. It's pretty damn fast at it. I haven't seen anything more than a couple of seconds lag while randomly accessing mail all over the folder. The original gzip of the file took >45 seconds. >> As this is an archive folder the contents >> won't change, so Squat FTS should be very fast after the first search, >> just as before. Interestingly, it appears my squat indexes aren't >> updating--for any folder. I've searched 4 folders via Tbird body search >> with xyzzyx (took forever) and I see no changes to the dates or sizes of >> existing indexes. I deleted the squat indexes for one folder and ran >> the search again. No new squat indexes were created. No errors in the >> logs. >> >> Any ideas why the squat indexes aren't updating? IIRC this happened >> once before and I was able to fix it. Don't recall how I did it though.... > >> mail_plugins: zlib > > Doesn't look like fts, fts_squat is enabled? Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins had to be declared in a single line directive. I did this as part of my troubleshooting when zlib wasn't working, thinking putting it on it's own line may help--not. -- Stan From Juergen.Obermann at hrz.uni-giessen.de Wed Nov 23 12:01:25 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Wed, 23 Nov 2011 11:01:25 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10on SPARC In-Reply-To: References: Message-ID: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> ----- Nachricht von tss at iki.fi --------- Datum: Wed, 23 Nov 2011 01:01:44 +0200 Von: Timo Sirainen Betreff: Re: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10 on SPARC An: J?rgen Obermann Cc: dovecot at dovecot.org > On 22.11.2011, at 22.11, J?rgen Obermann wrote: > >> "imapc-client.h", line 21: warning: enumerator value overflows >> INT_MAX (2147483647) > > What? No it doesn't.. Does changing the 0x400000000 in line 20 to > 0x200000000 help? No, but stripping off the last zero helps, because INT_MAX is 0x7FFFFFFF. >> "imapc-connection.c", line 1446: operand cannot have void type: op "==" >> "imapc-connection.c", line 1446: operands have incompatible types: > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1cdd39d11ce4 > > > OK now. ----- Ende der Nachricht von tss at iki.fi ----- One more compilation problem occurs: gmake[5]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/lib-storage/index/imapc' source='imapc-storage.c' object='imapc-storage.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../../../../depcomp \ /bin/bash ../../../../libtool --tag=CC --mode=compile cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c -o imapc-storage.lo imapc-storage.c libtool: compile: cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/lib -I../../../../src/lib-test -I../../../../src/lib-settings -I../../../../src/lib-mail -I../../../../src/lib-imap -I../../../../src/lib-imap-client -I../../../../src/lib-index -I../../../../src/lib-storage -I../../../../src/lib-storage/list -I../../../../src/lib-storage/index -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c imapc-storage.c -KPIC -DPIC -o .libs/imapc-storage.o "../../../../src/lib-storage/mailbox-list.h", line 13: warning: useless declaration "../../../../src/lib-storage/mailbox-list-private.h", line 14: warning: useless declaration "imapc-sync.h", line 4: warning: useless declaration "imapc-storage.c", line 500: void function cannot return value cc: acomp failed for imapc-storage.c gmake[5]: *** [imapc-storage.lo] Error 1 Greetings, J?rgen Obermann -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From patrickdk at patrickdk.com Wed Nov 23 14:24:39 2011 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 23 Nov 2011 07:24:39 -0500 Subject: [Dovecot] Unique message IDs? In-Reply-To: <4ECB3592.1010500@lazurit.com> References: <4ECAAE5F.1040507@lazurit.com> <4ECAC6E3.9090307@lazurit.com> <1321912606.24607.61.camel@hurina> <4ECB3592.1010500@lazurit.com> Message-ID: <20111123072439.Horde.FvMFLZLnE6FOzOYHNPlAn_A@kishi.patrickdk.com> Quoting Alexander Chekalin : >>>> With Maildir the message GUID is typically the same as the >>>> Maildir base filename (i.e. everything before ':' character). >>> >>> But what if I one day decide to convert my maildir's to mbox'es? >>> I really plan to do such conversion in a while (as soon as I finish the >>> indexing system). >> >> mbox? or mdbox? With mbox there are no proper GUIDs, but Dovecot kind of >> fakes it by returning MD5 of specific headers as GUIDs (so not 100% >> reliable). With mdbox GUIDs work even better than with Maildir, the GUID >> is always stored in the message's metadata. > > I'd like to use the best optimized one (mdbox), but there is a > reason not to do that is when I use mbox or maildir I can see where > given IMAP folder mails are stored, so, say, if I want to copy only > one IMAP folder to some remote site, I just copy know dir or file. > With mdbox this is different, I simply can not guess where my > messages exactly are. > > If it be possible to have per-IMAP-folders mdboxes, I love to use > it. But from what I know and tried this is not way mdbox used to > work, right? > You can always use the info from the wiki, I took some code from it to create this little script, that dumps my spam folder and uses it to learn. Basically dumps a mailfolder back into maildir format. doveadm search -u $useraccount mailbox Spam | while read guid uid; do doveadm fetch -u $useraccount text mailbox-guid $guid uid $uid > msg.$uid; done If your using mdbox on the other end, you could reimport them I suppose, I haven't looked into doing that, since I haven't needed that yet. From elhijo at 0lim.net Wed Nov 23 16:19:34 2011 From: elhijo at 0lim.net (David LEROUX) Date: Wed, 23 Nov 2011 15:19:34 +0100 Subject: [Dovecot] dovecot as an imap "client" Message-ID: <4ECD00F6.4060908@0lim.net> Hi, I need to get emails from one of my users imap account on a server which is not managed by us to his local imap mailbox. I'm almost sure that I've read some times that dovecot would be able to do that, kind of imap relay, but I can't find it again. I'm not sure to be very clear. Thanks, -- David From frank at moltke28.B.Shuttle.DE Wed Nov 23 16:53:18 2011 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 23 Nov 2011 15:53:18 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: <4ECD00F6.4060908@0lim.net> References: <4ECD00F6.4060908@0lim.net> Message-ID: (auto-added) On Wed, 23 Nov 2011 15:19:34 +0100 David LEROUX wrote: > Hi, > I need to get emails from one of my users imap account on a server which > is not managed by us to his local imap mailbox. imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner From micah at riseup.net Wed Nov 23 16:54:21 2011 From: micah at riseup.net (Micah Anderson) Date: Wed, 23 Nov 2011 09:54:21 -0500 Subject: [Dovecot] doveadm import assertion failed Message-ID: <87vcqax52a.fsf@algae.riseup.net> Restoring a user's mailbox yesterday resulted in 'doveadm import' panic'ing with an assertion failure and giving a backtrace: /usr/bin/doveadm import -u mdbox:/maildir/riseup.net/a//.daily.1/mdbox restored_from_backups/daily1 all doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac tion log sequence (3) doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac tion log sequence (3) doveadm(): Panic: file mail-transaction-log-file.c: line 187 (mail_transaction_log_file_add_to_list): assertion failed: ((*p)->hdr.file_se q < file->hdr.file_seq) doveadm(): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd1a) [0x7f05152bbd1a] -> /usr/lib/dovecot/libdovecot.so.0(default_f atal_handler+0x32) [0x7f05152bbe02] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f051529519f] -> /usr/lib/dovecot/libdovecot-storage.so .0(+0xa596a) [0x7f05159e696a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transaction_log_file_open+0x1f8) [0x7f05159e7bc8] -> /usr/lib/ dovecot/libdovecot-storage.so.0(mail_transaction_log_find_file+0xd0) [0x7f05159e46e0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_transa ction_log_view_set+0xcb) [0x7f05159e8bdb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0xbe) [0x7f05159dff6e] -> /usr/lib/ dovecot/libdovecot-storage.so.0(mail_index_map+0x86) [0x7f05159d1816] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9a6da) [0x7f05159db6da] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x56) [0x7f05159db956] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_ index_sync_begin+0x1e) [0x7f05159dc21e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x121) [0x7f05159bbb41] -> /usr/lib/dove cot/libdovecot-storage.so.0(mdbox_sync+0x46) [0x7f05159bc1f6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x43) [0x7f 05159bc293] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7f0515974031] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7f05159748b7] -> /usr/bin/doveadm(doveadm_mail_iter_init+0x8b) [0x40c9db] -> /usr/bin/doveadm() [0x40c284] -> /usr/bin/doveadm() [0x40a673] -> /usr/bin/doveadm(doveadm_mail_single_user+0x61) [0x40a731] -> /usr/bin/doveadm() [0x40a95d] -> /usr/bin/doveadm(doveadm_mail_try_run+0x141) [0x40ad71] -> /usr/bin/doveadm(main+0x381) [0x4107e1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f0514b2dc4d] -> /usr/bin/doveadm() [0x409e59] This is with 2.0.15. Micah -- From tss at iki.fi Wed Nov 23 16:58:41 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 16:58:41 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris 10on SPARC In-Reply-To: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> Message-ID: <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> On 23.11.2011, at 12.01, J?rgen Obermann wrote: >> On 22.11.2011, at 22.11, J?rgen Obermann wrote: >> >>> "imapc-client.h", line 21: warning: enumerator value overflows INT_MAX (2147483647) >> >> What? No it doesn't.. Does changing the 0x400000000 in line 20 to 0x200000000 help? > > No, but stripping off the last zero helps, because INT_MAX is 0x7FFFFFFF. Oops :) Looks like gcc automatically increases the enum's size to 64bit in that case. > "imapc-storage.c", line 500: void function cannot return value http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e From elhijo at 0lim.net Wed Nov 23 17:07:25 2011 From: elhijo at 0lim.net (David LEROUX) Date: Wed, 23 Nov 2011 16:07:25 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: (auto-added) References: <4ECD00F6.4060908@0lim.net> (auto-added) Message-ID: <4ECD0C2D.8080901@0lim.net> On 11/23/2011 03:53 PM, Frank Elsner wrote: > imapsync comes to my mind. - http://imapsync.lamiral.info/ - > http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync > --Frank Elsner Thanks, Thought that dovecot would be able to do that. David. From info at fduerr.de Wed Nov 23 10:16:30 2011 From: info at fduerr.de (Frank) Date: Wed, 23 Nov 2011 08:16:30 +0000 (UTC) Subject: [Dovecot] dsync strange duplicate directories Message-ID: Hello, i'm testing whether dsync will help me in setting up a redundant master - master imap setup and experience this: 1. Master 1 dovecot is running, master 2 dovecot is down 2. I use rsync to create an exact copy of the master 1 maildir on master 2 3. I start up master 2 dovecot 4. I do a dsync on master 2: dsync -D -u test at example.com mirror ssh root at master1 dsync -D -u test at example.com Now what i see is this: Each mail directory is duplicated on master 1 and master 2 to a directory name with some MD5 hash value (as it seems) e.g. before dsync: ls -a /home/vmail/example.com/test/Maildir cur new .Drafts .Junk .Sent ... after dsync: ls -a /home/vmail/example.com/test/Maildir cur new .Drafts .Drafts_af61060f6f9fcb4efc0a00002691362e .Junk .Junk_b261060f6f9fcb4efc0a00002691362e .Sent .Sent_b261060f6f9fcb4efc0a00002691362e ... The new directories contain the same files as the original directories. What am i doing wrong? Thanks for the help Frank From tss at iki.fi Wed Nov 23 17:29:16 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 17:29:16 +0200 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: <4ECD0C2D.8080901@0lim.net> References: <4ECD00F6.4060908@0lim.net> (auto-added) <4ECD0C2D.8080901@0lim.net> Message-ID: On 23.11.2011, at 17.07, David LEROUX wrote: > On 11/23/2011 03:53 PM, Frank Elsner wrote: >> imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner > Thanks, > Thought that dovecot would be able to do that. v2.1 has "imapc" backend, which you could use with dsync to do a migration, but for now it's probably easier to just use imapsync. From Juergen.Obermann at hrz.uni-giessen.de Wed Nov 23 19:07:09 2011 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Wed, 23 Nov 2011 18:07:09 +0100 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris10on SPARC In-Reply-To: <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> Message-ID: Am 23.11.2011 15:58, schrieb Timo Sirainen: > On 23.11.2011, at 12.01, J?rgen Obermann wrote: > >>> On 22.11.2011, at 22.11, J?rgen Obermann wrote: >>> >>>> "imapc-client.h", line 21: warning: enumerator value overflows >>>> INT_MAX (2147483647) >>> >>> What? No it doesn't.. Does changing the 0x400000000 in line 20 to >>> 0x200000000 help? >> >> No, but stripping off the last zero helps, because INT_MAX is >> 0x7FFFFFFF. > > Oops :) Looks like gcc automatically increases the enum's size to > 64bit in that case. > >> "imapc-storage.c", line 500: void function cannot return value > > http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e There seems to be one more similar problem: gmake[3]: Entering directory `/net/fileserv/export/sunsrc/src/dovecot-2.1.beta1/src/anvil' source='anvil-connection.c' object='anvil-connection.o' libtool=no \ DEPDIR=.deps depmode=none /bin/bash ../../depcomp \ cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-master -I/usr/local/include -fast -xarch=v8plusa -I/usr/sfw/include -c anvil-connection.c "anvil-connection.c", line 156: void function cannot return value cc: acomp failed for anvil-connection.c gmake[3]: *** [anvil-connection.o] Error 2 -- J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From tss at iki.fi Wed Nov 23 19:11:55 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 19:11:55 +0200 Subject: [Dovecot] Problem compiling dovecot 2.1 Beta1 under Solaris10on SPARC In-Reply-To: References: <20111123110125.20044ckf9t81e8ow@webmail.hrz.uni-giessen.de> <48632BC1-4D21-4D0B-93CA-8E96F8AE49DC@iki.fi> Message-ID: <1322068318.24607.64.camel@hurina> On Wed, 2011-11-23 at 18:07 +0100, J?rgen Obermann wrote: > > http://hg.dovecot.org/dovecot-2.1/rev/bfcd0bed5a9e > > There seems to be one more similar problem: > > "anvil-connection.c", line 156: void function cannot return value Fixed: http://hg.dovecot.org/dovecot-2.1/rev/4f1d20b57c04 From pch0317 at gmail.com Wed Nov 23 21:55:58 2011 From: pch0317 at gmail.com (pch0317) Date: Wed, 23 Nov 2011 19:55:58 +0000 Subject: [Dovecot] Problem with Outlook 2010 Message-ID: <4ECD4FCE.4000400@gmail.com> Hi, Maybe this time somebody help. Thanks -------- Original Message -------- Subject: Problem with Outlook 2010 Date: Tue, 07 Jun 2011 20:20:56 +0100 From: pch0317 To: Dovecot Mailing List Hi, I use dovecot 2.0. I have problem with Outlook 2010. This application freezes for about 2 minutes when I move or delete messages. Other applications such as Outlook Express and Thunderbird work correctly. Has anyone had similar problem? Thanks From tss at iki.fi Wed Nov 23 22:09:18 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 22:09:18 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: Message-ID: <1322078958.24607.76.camel@hurina> On Wed, 2011-11-23 at 00:56 +0200, Timo Sirainen wrote: > > Hi, I found that doing a wildcard search with doveadm doesn't expand the %d > > global variable when doing a wildcard search like: > > > > Nov 22 20:48:07 auth-worker: Debug: ldap: iterate: base=vd=%d,dc=domain > > scope=subtree > > Iteration is supposed to iterate through all users.. > > > Is this a bug? > > > More like a missing feature I guess. I guess it could make it expand the %d if it's known.. hg version of v2.1 now supports this. From dovecot.user at seibercom.net Wed Nov 23 22:28:00 2011 From: dovecot.user at seibercom.net (Jerry) Date: Wed, 23 Nov 2011 15:28:00 -0500 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <20111123152800.6504a2ba@scorpio> On Wed, 23 Nov 2011 19:55:58 +0000 pch0317 articulated: > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work > correctly. > > Has anyone had similar problem? Seriously, do you have any log messages, etcetera to diagnose this problem with? -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From dovecot.user at seibercom.net Wed Nov 23 22:31:56 2011 From: dovecot.user at seibercom.net (Jerry) Date: Wed, 23 Nov 2011 15:31:56 -0500 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <20111123153156.0c02bb74@scorpio> On Wed, 23 Nov 2011 19:55:58 +0000 pch0317 articulated: > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work > correctly. You could start here to get some useful information as to what Outlook 2010 is doing. http://www.theemailadmin.com/2010/08/troubleshooting-outlook-2010-connections/ -- Jerry ? Dovecot.user at seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Wed Nov 23 22:57:50 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 22:57:50 +0200 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <4E754D10.5010404@babelmonkeys.de> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> Message-ID: <1322081870.24607.78.camel@hurina> On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: > Am 16.09.2011 14:40, schrieb Timo Sirainen: > > On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: > > > > Looks pretty good. Below are a few things I noticed. I could fix these > > myself next week also, or you can do them during weekend if you want > > to. :) > > > I decided to do it myself, hope this fixes all issues. Committed finally to http://hg.dovecot.org/dovecot-2.1 with some cleanups. From tss at iki.fi Wed Nov 23 23:02:30 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:02:30 +0200 Subject: [Dovecot] Can't delete sub-folder In-Reply-To: <4EBAFFD7.8020201@gmail.com> References: <4EBAFFD7.8020201@gmail.com> Message-ID: <1322082151.24607.79.camel@hurina> On Wed, 2011-11-09 at 22:33 +0000, pch0317 wrote: > I use thunderbird 8.0 as a client and dovecot 2.0.9 with imap protocol .. > When I try to delete 'parent' folder which contain 'messages only' > folder, error message appear: [ALREADYEXISTS] Mailbox has children, > delete them first. Ok, so I delete 'message only' folder first and try > to delete parent folder, but the same error appear: [ALREADYEXISTS] > Mailbox has children, delete them first. > When I list mailboxes from console I see inside 'parent' folder ".imap" > folder. When I delete .imap folder from console and then delete parent > folder from thunderbird, parent folder is deleted correctly. You can reproduce this? I tried with v2.0.16 and it works ok, so I have probably fixed this since v2.0.9. From stephan at rename-it.nl Wed Nov 23 23:07:14 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 23 Nov 2011 22:07:14 +0100 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <1322081870.24607.78.camel@hurina> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> <1322081870.24607.78.camel@hurina> Message-ID: <4ECD6082.4070406@rename-it.nl> On 11/23/2011 9:57 PM, Timo Sirainen wrote: > On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: >> Am 16.09.2011 14:40, schrieb Timo Sirainen: >>> On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: >>> >>> Looks pretty good. Below are a few things I noticed. I could fix these >>> myself next week also, or you can do them during weekend if you want >>> to. :) >>> >> I decided to do it myself, hope this fixes all issues. > Committed finally to http://hg.dovecot.org/dovecot-2.1 with some > cleanups. Great, the Pigeonhole ManageSieve implementation is now suddenly fully RFC compliant, without any effort on my part! :) Regards, Stephan. From tss at iki.fi Wed Nov 23 23:11:53 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:11:53 +0200 Subject: [Dovecot] [PATCH] SCRAM-SHA-1 authentication In-Reply-To: <4ECD6082.4070406@rename-it.nl> References: <4E729CBC.5080600@babelmonkeys.de> <1316176851.12936.91.camel@hurina> <4E754D10.5010404@babelmonkeys.de> <1322081870.24607.78.camel@hurina> <4ECD6082.4070406@rename-it.nl> Message-ID: <1322082713.24607.81.camel@hurina> On Wed, 2011-11-23 at 22:07 +0100, Stephan Bosch wrote: > On 11/23/2011 9:57 PM, Timo Sirainen wrote: > > On Sun, 2011-09-18 at 03:44 +0200, Florian Zeitz wrote: > >> Am 16.09.2011 14:40, schrieb Timo Sirainen: > >>> On Fri, 2011-09-16 at 02:47 +0200, Florian Zeitz wrote: > >>> > >>> Looks pretty good. Below are a few things I noticed. I could fix these > >>> myself next week also, or you can do them during weekend if you want > >>> to. :) > >>> > >> I decided to do it myself, hope this fixes all issues. > > Committed finally to http://hg.dovecot.org/dovecot-2.1 with some > > cleanups. > > Great, the Pigeonhole ManageSieve implementation is now suddenly fully > RFC compliant, without any effort on my part! :) I'm not actually sure about that :) The final replying is probably wrong, since ManageSieve supports sending it to client, unlike IMAP/POP3.. From elhijo at 0lim.net Wed Nov 23 23:13:20 2011 From: elhijo at 0lim.net (elhijo) Date: Wed, 23 Nov 2011 22:13:20 +0100 Subject: [Dovecot] dovecot as an imap "client" In-Reply-To: References: <4ECD00F6.4060908@0lim.net> (auto-added) <4ECD0C2D.8080901@0lim.net> Message-ID: <4ECD61F0.1010806@0lim.net> On 23/11/2011 16:29, Timo Sirainen wrote: > On 23.11.2011, at 17.07, David LEROUX wrote: > >> On 11/23/2011 03:53 PM, Frank Elsner wrote: >>> imapsync comes to my mind. - http://imapsync.lamiral.info/ - http://www.howtoforge.com/how-to-migrate-mailboxes-between-imap-servers-with-imapsync --Frank Elsner >> Thanks, >> Thought that dovecot would be able to do that. > v2.1 has "imapc" backend, which you could use with dsync to do a migration, but for now it's probably easier to just use imapsync. > I'll stick to imapsync which seems to prefectly fit to my needs. Thanks for your help, David From stephan at rename-it.nl Wed Nov 23 23:15:00 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 23 Nov 2011 22:15:00 +0100 Subject: [Dovecot] v2.1.beta1 released In-Reply-To: <4EB9BD21.6060501@rename-it.nl> References: <1320791738.21919.350.camel@hurina> <4EB9BD21.6060501@rename-it.nl> Message-ID: <4ECD6254.2020606@rename-it.nl> On 11/9/2011 12:37 AM, Stephan Bosch wrote: > On 11/8/2011 11:35 PM, Timo Sirainen wrote: >> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz >> http://dovecot.org/releases/2.1/beta/dovecot-2.1.beta1.tar.gz.sig >> >> Here's the first beta release of Dovecot v2.1. This version has already >> been tested quite a lot, so I'm not expecting any major bugs. So please >> upgrade and see if you can find any problems. I'm optimistic about >> getting rc1 released this year and perhaps even v2.1.0. >> > > Ok, I guess it is time for a Pigeonhole release for Dovecot v2.1. I'm > a bit swamped this week, but next weekend I should be able to compose > one. For now you can use the dovecot-2.0-pigeonhole hg and patch it > with > http://hg.rename-it.nl/pigeonhole-0.2-dovecot-2.1-patches/file/tip/pigeonhole-0.2-dovecot-2.1.patch > to make it compile against v2.1. I have an update on this. Although there is no tarball release so far, I did start a separate repository with Pigeonhole (v0.3.x) for Dovecot v2.1: http://hg.rename-it.nl/dovecot-2.1-pigeonhole This means that the patch procedure mentioned above is no longer necessary. From this point onwards, new features are only implemented for Pigeonhole v0.3.x, unless a back-port can be achieved with little risk and effort. First in line is support for the editheader extension, which I expect to finish next week. Regards, Stephan. From ramiblanco at gmail.com Wed Nov 23 23:19:35 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 18:19:35 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322078958.24607.76.camel@hurina> References: <1322078958.24607.76.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > > hg version of v2.1 now supports this. > Does it need some extra configuration? I've upgraded to hg ver. of 2.1 and still the same error... Thanks for the quick response! -- Ramiro Blanco From tss at iki.fi Wed Nov 23 23:27:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 23 Nov 2011 23:27:47 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> Message-ID: <1322083667.24607.82.camel@hurina> On Wed, 2011-11-23 at 18:19 -0300, Ramiro Blanco wrote: > 2011/11/23 Timo Sirainen : > > > > hg version of v2.1 now supports this. > > > > Does it need some extra configuration? No. > I've upgraded to hg ver. of 2.1 and still the same error... Does it still show "%d" in the error messages, or is it "" now? From ramiblanco at gmail.com Thu Nov 24 00:08:20 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 19:08:20 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322083667.24607.82.camel@hurina> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > > Does it still show "%d" in the error messages, or is it "" now? > It still shows "%d" Nov 23 18:42:59 auth-worker(729): Debug: ldap: iterate: base=vd=%d,dc=domain scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 23 18:42:59 auth-worker(729): Error: ldap(): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object -- Ramiro Blanco From tss at iki.fi Thu Nov 24 01:13:06 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:13:06 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> Message-ID: <1322089987.24607.84.camel@hurina> On Wed, 2011-11-23 at 19:08 -0300, Ramiro Blanco wrote: > 2011/11/23 Timo Sirainen : > > > > Does it still show "%d" in the error messages, or is it "" now? > > > It still shows "%d" > > Nov 23 18:42:59 auth-worker(729): Debug: ldap: iterate: > base=vd=%d,dc=domain scope=subtree Actually only the debug message was wrong. The expansion should have worked. Anyway, fixed the debug message: http://hg.dovecot.org/dovecot-2.1/rev/1fed3c1c166e From tss at iki.fi Thu Nov 24 01:21:04 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:21:04 +0200 Subject: [Dovecot] doveadm import assertion failed In-Reply-To: <87vcqax52a.fsf@algae.riseup.net> References: <87vcqax52a.fsf@algae.riseup.net> Message-ID: <1322090464.24607.87.camel@hurina> On Wed, 2011-11-23 at 09:54 -0500, Micah Anderson wrote: > Restoring a user's mailbox yesterday resulted in 'doveadm import' > panic'ing with an assertion failure and giving a backtrace: > > /usr/bin/doveadm import -u mdbox:/maildir/riseup.net/a//.daily.1/mdbox restored_from_backups/daily1 all > doveadm(): Error: Transaction log /maildir/riseup.net/a//.daily.1/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.log: duplicate transac > tion log sequence (3) So this path points to the backed up mailbox you're importing, not the destination mailbox? And you can reproduce this crash reliably? Could you send me these files: mailboxes/INBOX/dbox-Mails/dovecot.index mailboxes/INBOX/dbox-Mails/dovecot.index.log files? storage/dovecot.map.index storage/dovecot.map.index.log None of them contain anything sensitive about the user's mailbox. From tss at iki.fi Thu Nov 24 01:23:26 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:23:26 +0200 Subject: [Dovecot] 1.2.15 can't get zlib working In-Reply-To: <4ECC44BD.3080806@hardwarefreak.com> References: <4ECBD518.9020206@hardwarefreak.com> <4ECC2149.4070403@hardwarefreak.com> <8A6A95A8-CC2B-4247-AF35-AA930BFCB1A6@iki.fi> <4ECC44BD.3080806@hardwarefreak.com> Message-ID: <1322090606.24607.89.camel@hurina> On Tue, 2011-11-22 at 18:56 -0600, Stan Hoeppner wrote: > Out of curiosity, what (or who) prompted the development of the > compressed mbox feature? Or was it that you wanted to do it for > maildir, and then figured you should for mbox as well? The > implementation seems to work ok. The instructions just seem a > bit...thin. :) I think it was one of my friends saying he can't start using Dovecot until it supports compressed mboxes. > >> mail_plugins: zlib > > > > Doesn't look like fts, fts_squat is enabled? > > Stupid me. When I enabled zlib I created a 2nd mail_plugins line. So > 'mail_plugins fts fts_squat' got ignored. Didn't realize all plugins > had to be declared in a single line directive. I did this as part of my > troubleshooting when zlib wasn't working, thinking putting it on it's > own line may help--not. With v2.0 you can do: mail_plugins = $mail_plugins zlib mail_plugins = $mail_plugins fts fts_squat From tss at iki.fi Thu Nov 24 01:25:03 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:25:03 +0200 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <4ECC40FF.8020103@knutejohnson.com> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> <4ECC40FF.8020103@knutejohnson.com> Message-ID: <1322090703.24607.91.camel@hurina> On Tue, 2011-11-22 at 16:40 -0800, Knute Johnson wrote: > The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; > > !include_try /usr/share/dovecot/protocols.d/*.protocol > > That directory contains one file; > > dovecot-imapd.protocol > > and it contains one line; > > protocols = $protocols imap > > So I am assuming that it is adding imap to the protocols in the > environment variable $protocols somehow? Not environment variable $protocols, but the previous "protocols" setting's value in dovecot.conf. > I did get TBird to retrieve mail from port 993 using SSL however. Will > it work without a protocols = imaps? Yeah, there's no such thing as "imaps protocol". I should never have included it as a protocol in previous Dovecot versions either.. From tss at iki.fi Thu Nov 24 01:55:08 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 01:55:08 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <4ECAD578.4060304@informatik.uni-freiburg.de> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> Message-ID: <1322092508.24607.93.camel@hurina> On Mon, 2011-11-21 at 23:49 +0100, Martin Preen wrote: > # /usr/sfw/bin/openssl engine > (pkcs11) PKCS #11 engine support > > >> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); > >> ENGINE_set_default_ciphers(e); > >> > >> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() > >> the crypto device gets used. I'm sure that this is not the whole story since > >> this only seems to affect the IMAP login. > > > > It should work for POP3 as well, all of the SSL code is shared. > > I couldn't find the EncryptUpdate call which has to be changed too > (due to the howto documents). Maybe some other call needs e patch. > But I don't know which. What EncryptUpdate?.. I've anyway added the engine init/deinit calls in your email to v2.1 hg. Lets hope it works :) At least it didn't break when I tried it with "dynamic" value (which is the only engine my OpenSSL supports). From tss at iki.fi Thu Nov 24 02:08:23 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:08:23 +0200 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1322092508.24607.93.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> <1322092508.24607.93.camel@hurina> Message-ID: <1322093304.24607.94.camel@hurina> On Thu, 2011-11-24 at 01:55 +0200, Timo Sirainen wrote: > What EncryptUpdate?.. I've anyway added the engine init/deinit calls in > your email to v2.1 hg. Lets hope it works :) At least it didn't break > when I tried it with "dynamic" value (which is the only engine my > OpenSSL supports). Oh, and in Solaris "pkcs11" value also works. But I don't know if it's actually using hardware crypto. From tss at iki.fi Thu Nov 24 02:25:17 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:25:17 +0200 Subject: [Dovecot] Seg fault in dovecot/auth 2.0.15 In-Reply-To: <4EC4922B.5050000@goldey.net> References: <4EC4922B.5050000@goldey.net> Message-ID: <1322094317.24607.96.camel@hurina> On Wed, 2011-11-16 at 23:48 -0500, Mark Goldey wrote: > (gdb) bt full > #0 0x00000000 in ?? () > No symbol table info available. > #1 0x0805b0c7 in auth_worker_handle_user (client=0x82754d0, id=2, > args=0x82601c8) at auth-worker-client.c:405 Still doesn't make any sense.. Try what it logs with attached patch. And perhaps also try running via valgrind: service auth-worker { executable = /usr/bin/valgrind -q /usr/local/libexec/dovecot/auth -w } -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: text/x-patch Size: 800 bytes Desc: not available URL: From nanitous at xs4all.nl Thu Nov 24 02:44:44 2011 From: nanitous at xs4all.nl (Nanitous) Date: Thu, 24 Nov 2011 01:44:44 +0100 Subject: [Dovecot] Excluding some system users after configuration Message-ID: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> Dear readers, Consider an installed Postfix configuration (2.3.3) on CentOS. Now I want to install Dovecot (alas only version 1.0.7.7 is available) to provide IMAP services for a few system users, but not all. Most importantly the mail delivery by postfix for some of these system user /must/ not change after the installation of Dovecot. How can I prevent the installation of Dovecot to touch some specific system user accounts? Thanks in advance, /Twan From dovecot at knutejohnson.com Thu Nov 24 02:45:10 2011 From: dovecot at knutejohnson.com (Knute Johnson) Date: Wed, 23 Nov 2011 16:45:10 -0800 Subject: [Dovecot] How to disable pop3 in version 2? In-Reply-To: <1322090703.24607.91.camel@hurina> References: <4ECC27EA.2000200@knutejohnson.com> <03AAF9D5-7AB3-47F7-9B8B-ABCB163769E0@iki.fi> <4ECC40FF.8020103@knutejohnson.com> <1322090703.24607.91.camel@hurina> Message-ID: <4ECD9396.1050703@knutejohnson.com> On 11/23/2011 3:25 PM, Timo Sirainen wrote: > On Tue, 2011-11-22 at 16:40 -0800, Knute Johnson wrote: > >> The Ubuntu installation has this line in the /etc/dovecot/dovecot.conf file; >> >> !include_try /usr/share/dovecot/protocols.d/*.protocol >> >> That directory contains one file; >> >> dovecot-imapd.protocol >> >> and it contains one line; >> >> protocols = $protocols imap >> >> So I am assuming that it is adding imap to the protocols in the >> environment variable $protocols somehow? > > Not environment variable $protocols, but the previous "protocols" > setting's value in dovecot.conf. > >> I did get TBird to retrieve mail from port 993 using SSL however. Will >> it work without a protocols = imaps? > > Yeah, there's no such thing as "imaps protocol". I should never have > included it as a protocol in previous Dovecot versions either.. > Thanks very much Timo! -- Knute Johnson From tss at iki.fi Thu Nov 24 02:46:56 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:46:56 +0200 Subject: [Dovecot] v2.1.rc1 released Message-ID: <1322095616.24607.108.camel@hurina> http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz.sig I'll probably release v2.1.0 pretty soon, unless people report some new bugs. I know v2.1 is already being used to serve mails to tens (or hundreds?) of thousands of users, so it should be pretty stable. Most of the changes since v2.1.beta1 have been for rather small issues. Many of the fixed bugs exist also in v2.0, but since they are so minor I didn't bother backporting the fixes. As new features since v2.1.beta1 there are the things I did also for v2.0.16, and: * ssl_client_cert/key: Proxying can now send SSL certificate to backend server * ssl_crypto_device for enabling hardware encryption * SCRAM-SHA-1 authentication mechanism support by Florian Zeitz * passdb/userdb checkpassword: Export all auth %variables to AUTH_* environment. * maildir_broken_filename_sizes setting for people upgrading from other maildir servers that wrote tons of broken S= values And as a reminder, here are the biggest new things since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now From jayw at interoceansystems.com Thu Nov 24 02:54:16 2011 From: jayw at interoceansystems.com (Jay Welch) Date: Wed, 23 Nov 2011 16:54:16 -0800 Subject: [Dovecot] Random Loss of Connection with Thunderbird and Dovecot Message-ID: <4ECD95B8.8070807@interoceansystems.com> Hi, Dovecot Version: 1.2.16 OS: CENTOS 5.5 I am having issues with Thunderbird connecting to Dovecot at random times. Usually an error comes up and states that Thunderbird cannot connect to the server. The only way I have been able to fix this is reset/repair the network connection or reboot the computer. Is there any parameters I can change in Dovecot to fix this issue? Does anyone know what is likely going on? I have been reading forums and I cannot find anything. Thanks From tss at iki.fi Thu Nov 24 02:51:48 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:51:48 +0200 Subject: [Dovecot] Excluding some system users after configuration In-Reply-To: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> References: <016F0556-1746-4B69-8BBE-D1B6A855DD52@xs4all.nl> Message-ID: <1322095908.24607.110.camel@hurina> On Thu, 2011-11-24 at 01:44 +0100, Nanitous wrote: > Dear readers, > > Consider an installed Postfix configuration (2.3.3) on CentOS. > Now I want to install Dovecot (alas only version 1.0.7.7 is available) to provide IMAP services for a few system users, but not all. Most importantly the mail delivery by postfix for some of these system user /must/ not change after the installation of Dovecot. > > How can I prevent the installation of Dovecot to touch some specific system user accounts? As long as you don't configure Postfix to deliver mails via Dovecot's "deliver", there is no change to how mails are delivered. If you want to restrict IMAP access to only some users, you can do it with pam_listfile: http://wiki.dovecot.org/Authentication/RestrictAccess From tss at iki.fi Thu Nov 24 02:56:07 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 02:56:07 +0200 Subject: [Dovecot] Random Loss of Connection with Thunderbird and Dovecot In-Reply-To: <4ECD95B8.8070807@interoceansystems.com> References: <4ECD95B8.8070807@interoceansystems.com> Message-ID: <1322096168.24607.114.camel@hurina> On Wed, 2011-11-23 at 16:54 -0800, Jay Welch wrote: > Dovecot Version: 1.2.16 > > > I am having issues with Thunderbird connecting to Dovecot at random > times. Usually an error comes up and states that Thunderbird cannot > connect to the server. The only way I have been able to fix this is > reset/repair the network connection or reboot the computer. Is there any > parameters I can change in Dovecot to fix this issue? Does anyone know > what is likely going on? I have been reading forums and I cannot find > anything. Dovecot always logs the reason for logout, e.g.: > Nov 24 02:53:24 imap(tss): Info: Connection closed in=0 out=291 This means that the connection got disconnected (i.e. Dovecot didn't do the disconnection) > Nov 24 02:52:47 imap(tss): Info: Disconnected: Logged out in=8 out=334 And this means that the client used a LOGOUT command. and so on. Find the log entries about the times when you're having problems. Dovecot logs about every single connection it sees. If you don't see a log message about some connection, it means that Dovecot never saw it, and the problem is outside Dovecot. I'd first look into any antivirus/software firewalls you've installed. They're usually the reason for connection problems with Windows. From ramiblanco at gmail.com Thu Nov 24 04:00:28 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 23:00:28 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <1322089987.24607.84.camel@hurina> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> Message-ID: 2011/11/23 Timo Sirainen : > Actually only the debug message was wrong. The expansion should have > worked. Anyway, fixed the debug message: > http://hg.dovecot.org/dovecot-2.1/rev/1fed3c1c166e Still failing: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users Now "%d" it's empty, it shows "vd=" instead of "vd=%d": Nov 23 22:50:46 auth-worker(26556): Debug: ldap: iterate: base=vd=,dc=uvq,dc=edu,dc=ar scope=subtree filter=(&(objectClass=VirtualMailAccount)(accountActive=TRUE)) fields=mail Nov 23 22:50:46 auth-worker(26556): Error: ldap(): ldap_search((&(objectClass=VirtualMailAccount)(accountActive=TRUE))) failed: No such object It is supposed to parse domain part from input in "doveadm search *@domain ..", right? -- Ramiro Blanco From tss at iki.fi Thu Nov 24 04:20:25 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 04:20:25 +0200 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> Message-ID: <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> On 24.11.2011, at 4.00, Ramiro Blanco wrote: > Now "%d" it's empty, it shows "vd=" instead of "vd=%d": > > It is supposed to parse domain part from input in "doveadm search > *@domain ..", right? Too tired to test search myself now, but this is what I used to test it myself: doveadm user '*@domain' If that works, then the problem is with doveadm and it needs more patching.. From ramiblanco at gmail.com Thu Nov 24 04:47:24 2011 From: ramiblanco at gmail.com (Ramiro Blanco) Date: Wed, 23 Nov 2011 23:47:24 -0300 Subject: [Dovecot] "%d" does not expand to domain on wildcard search In-Reply-To: <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> References: <1322078958.24607.76.camel@hurina> <1322083667.24607.82.camel@hurina> <1322089987.24607.84.camel@hurina> <8349C93D-250A-4668-BE0E-224966C8B51F@iki.fi> Message-ID: 2011/11/23 Timo Sirainen : > Too tired to test search myself now, but this is what I used to test it myself: > You've done too much already! > doveadm user '*@domain' > > If that works, then the problem is with doveadm and it needs more patching.. That works just fine. I guess it needs some patching then. -- Ramiro Blanco From robert at schetterer.org Thu Nov 24 08:05:47 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 24 Nov 2011 07:05:47 +0100 Subject: [Dovecot] Problem with Outlook 2010 In-Reply-To: <4ECD4FCE.4000400@gmail.com> References: <4ECD4FCE.4000400@gmail.com> Message-ID: <4ECDDEBB.6000608@schetterer.org> Am 23.11.2011 20:55, schrieb pch0317: > Hi, > > Maybe this time somebody help. > > Thanks > > -------- Original Message -------- > Subject: Problem with Outlook 2010 > Date: Tue, 07 Jun 2011 20:20:56 +0100 > From: pch0317 > To: Dovecot Mailing List > > > > Hi, > > I use dovecot 2.0. > > I have problem with Outlook 2010. This application freezes for about 2 > minutes when I move or delete messages. > Other applications such as Outlook Express and Thunderbird work correctly. > > Has anyone had similar problem? > > > Thanks > > no problem with any outlook version what are your logs about the problem? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From oni-neko at gmx.net Thu Nov 24 09:10:27 2011 From: oni-neko at gmx.net (oni-neko at gmx.net) Date: Thu, 24 Nov 2011 08:10:27 +0100 Subject: [Dovecot] seting acl for master user? In-Reply-To: <1321636922.11489.46.camel@hurina> References: <20111118133428.63140@gmx.net> <1321636922.11489.46.camel@hurina> Message-ID: <20111124071027.28190@gmx.net> thank you very much for the reply, it works now, after i removed all the acl-stuff. =) i don't really know why it didn't work directly after i set up the master user (but before trying around with acls), but hey, I'm not complaning that it's working now =) thanks again and sorry for the late re =) bye sil -------- Original-Nachricht -------- > Datum: Fri, 18 Nov 2011 19:22:02 +0200 > Von: Timo Sirainen > An: oni-neko at gmx.net > CC: dovecot at dovecot.org > Betreff: Re: [Dovecot] seting acl for master user? > On Fri, 2011-11-18 at 14:34 +0100, oni-neko at gmx.net wrote: > > > what i don't get is the acl-setting, so that the master-user can > > access all the other users mails and folders. > .. > > in my case I only want the master user (called cyrus) to be able to do > > everything with/in all mailboxes, so I made a > > file /etc/dovecot/acls/default that reads "user=cyrus lrwstipekxa". > > Unfortunately currently there is no default ACL file. The above would > create an ACL file for a mailbox called "default". I'll need to get the > default ACLs implemented some day.. > > But if you don't need ACLs for anything else, you can simply disable the > ACL plugin and the master user will have access to everything. If you do > need ACLs, there are some kludges you can still do. > > -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de From preen at informatik.uni-freiburg.de Thu Nov 24 10:45:26 2011 From: preen at informatik.uni-freiburg.de (Martin Preen) Date: Thu, 24 Nov 2011 09:45:26 +0100 Subject: [Dovecot] Solaris hardware crypto engines In-Reply-To: <1322092508.24607.93.camel@hurina> References: <4DAC12C3.5060503@informatik.uni-freiburg.de> <4DACA6E3.5020301@bio.umass.edu> <4ECA174A.1020909@informatik.uni-freiburg.de> <1321912348.24607.58.camel@hurina> <4ECAD578.4060304@informatik.uni-freiburg.de> <1322092508.24607.93.camel@hurina> Message-ID: <4ECE0426.7020805@informatik.uni-freiburg.de> Timo Sirainen wrote: > On Mon, 2011-11-21 at 23:49 +0100, Martin Preen wrote: > >> # /usr/sfw/bin/openssl engine >> (pkcs11) PKCS #11 engine support >> >>>> ENGINE_set_default_RSA(e); ENGINE_set_default_DSA(e); >>>> ENGINE_set_default_ciphers(e); >>>> >>>> in ssl_proxy_init() and inserting ENGINE_cleanup(); in ssl_proxy_deinit() >>>> the crypto device gets used. I'm sure that this is not the whole story since >>>> this only seems to affect the IMAP login. >>> It should work for POP3 as well, all of the SSL code is shared. >> I couldn't find the EncryptUpdate call which has to be changed too >> (due to the howto documents). Maybe some other call needs e patch. >> But I don't know which. > > What EncryptUpdate?.. I've anyway added the engine init/deinit calls in > your email to v2.1 hg. Lets hope it works :) At least it didn't break > when I tried it with "dynamic" value (which is the only engine my > OpenSSL supports). Probably I'm wrong (I have no experience with SSL programming). I thught the EncryptUpdate was necessary for the encoding of the SSL data stream. But maybe there has to be a link between engine initialization and the SSL contexts ? Martin --------------------------------------------------------------- Martin Preen, Universit?t Freiburg, Institut f?r Informatik Georges-Koehler-Allee 52, Raum EG-006, 79110 Freiburg, Germany phone: ++49 761 203-8250 preen at informatik.uni-freiburg.de fax: ++49 761 203-8242 swt.informatik.uni-freiburg.de/~preen -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 6337 bytes Desc: S/MIME Cryptographic Signature URL: From olli.raisanen at telemail.fi Thu Nov 24 11:22:20 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-1?q?R=E4is=E4nen?=) Date: Thu, 24 Nov 2011 11:22:20 +0200 Subject: [Dovecot] How to create home directories for virtual users? Message-ID: <201111241122.21131.olli.raisanen@telemail.fi> Hello, I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home directory should not be the same as mail directory. When I create new user accounts with PostfixAdmin the mail directory is not a problem but is there some smart way to create the home directory as well (locating it for instance in '/srv/vmail/%d/%n/home')? Regards, Olli From pw at wk-serv.de Thu Nov 24 12:04:25 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 11:04:25 +0100 Subject: [Dovecot] Indexes to MLC-SSD In-Reply-To: <20111122104547.GA10680@dibs.tanso.net> References: <4EB2B15D.1010005@wildgooses.com> <4EB2C704.1010107@wk-serv.de> <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> Message-ID: <1d5f057161f05918ab010a979c455d9c@wk-serv.de> On Tue, 22 Nov 2011 11:45:47 +0100, Jan-Frode Myklebust wrote: > Ah, then Timo's reply was right. He suggested you do the > lmtp-deliveries > to the same server that you would send you imap-user to. You can do > this > trough dovecot director and lmtp-proxying. > > So instead of: > > lmtp:unix:private/dovecot-lmtp > > you should use: > > lmtp:tcp:1.2.3.4:24 > > where 1.2.3.4 would be the Dovecot LMTP proxy that proxies to the > same > machine as you would use for imap for this particular recipient. I see. So as far as I understood: - I set up a new server as LMTP proxy for my two MX 10 to connect to - the proxy redirects to my backend imap servers which will then store the mails on my shared storage and the index files to a local disk (so I have to enable LMTP additionally to enable this servers to store the mails) - I set up a frontend imap server for my users to connect to which will redirect them to the backend servers Am I right so far? Patrick From johannes at sipsolutions.net Thu Nov 24 15:11:06 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 14:11:06 +0100 Subject: [Dovecot] modules directory Message-ID: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Hi! I'm a little confused with how plugin loading works in dovecot 2, I can't seem to make it load from $(moduledir)/imap/. Is that intended to work? Just a configuration issue? Reason I ask is that I ported my antispam plugin (I know there's a fork, but I still like mine better) to dovecot 2 and the default "make install" no longer loads properly with just "mail_plugins = antispam". So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, but I still see subdirectories for at least auth. Should I install into just $(moduledir)/ now? johannes From pw at wk-serv.de Thu Nov 24 15:35:55 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 14:35:55 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <20111124104709.GB24297@dibs.tanso.net> References: <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> Message-ID: <4ECE483B.5090208@wk-serv.de> Jan-Frode Myklebust schrieb: > Yes, and in ascii-art this becomes :-) > > MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > \---(lmtp)---> backend-server2 > > > IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > \--(imap) --> backend-server2 I failed creating a nice ascii view ;-) However, I have some more questions: - What happens if the MTA accepted a message but the backend-server fails and is unable to store it? - What about load-balancing and/or failover? Maybe all users on backend-server1 are fetching their mails causing high load while backend-server2 is idle. Is it possible to specify more than one backend-server IPs as I can do for the database hosts for userdb queries? From pw at wk-serv.de Thu Nov 24 15:38:43 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 14:38:43 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <20111124104709.GB24297@dibs.tanso.net> References: <4EB59802.9040205@wildgooses.com> <4EB72177.8090801@wk-serv.de> <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> Message-ID: <4ECE48E3.1070806@wk-serv.de> Jan-Frode Myklebust schrieb: > MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > \---(lmtp)---> backend-server2 > > > IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > \--(imap) --> backend-server2 In this scenario it should be possible to use a non cluster filesystem to store my mails, isn't it? Patrick From tss at iki.fi Thu Nov 24 16:02:24 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 16:02:24 +0200 Subject: [Dovecot] modules directory In-Reply-To: <1322140266.5366.8.camel@jlt3.sipsolutions.net> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Message-ID: On 24.11.2011, at 15.11, Johannes Berg wrote: > I'm a little confused with how plugin loading works in dovecot 2, I > can't seem to make it load from $(moduledir)/imap/. Is that intended to > work? Just a configuration issue? > > Reason I ask is that I ported my antispam plugin (I know there's a fork, > but I still like mine better) to dovecot 2 and the default "make > install" no longer loads properly with just "mail_plugins = antispam". > > So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, > but I still see subdirectories for at least auth. There's no /imap/ directory anymore, because it was annoying having to create symlinks to most of the plugins. Then with v2.0 there came new binaries and it wouldn't have made much sense adding lots of more directories with lots of more symlinks.. > Should I install into just $(moduledir)/ now? Yep. From janfrode at tanso.net Thu Nov 24 16:21:55 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 24 Nov 2011 15:21:55 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <4ECE48E3.1070806@wk-serv.de> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE48E3.1070806@wk-serv.de> Message-ID: <20111124142155.GA28600@dibs.tanso.net> On Thu, Nov 24, 2011 at 02:38:43PM +0100, Patrick Westenberg wrote: > Jan-Frode Myklebust schrieb: > > >MTA -(lmtp)--> dovecot-director ---(lmtp)---> backend-server1 > > \---(lmtp)---> backend-server2 > > > > > >IMAP-user -(imap)--> dovecot-director --(imap) --> backend-server1 > > \--(imap) --> backend-server2 > > In this scenario it should be possible to use a non cluster filesystem > to store my mails, isn't it? Yes, that should work fine -- assuming you have acceptable uptime on the backend server that is hosting the messages. You also need to configure a user<->backend-server mapping trough proxy extrafield to make sure that the user lands on the correct server: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy -jf From johannes at sipsolutions.net Thu Nov 24 16:13:40 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 15:13:40 +0100 Subject: [Dovecot] modules directory In-Reply-To: References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> Message-ID: <1322144020.5366.20.camel@jlt3.sipsolutions.net> On Thu, 2011-11-24 at 16:02 +0200, Timo Sirainen wrote: > On 24.11.2011, at 15.11, Johannes Berg wrote: > > > I'm a little confused with how plugin loading works in dovecot 2, I > > can't seem to make it load from $(moduledir)/imap/. Is that intended to > > work? Just a configuration issue? > > > > Reason I ask is that I ported my antispam plugin (I know there's a fork, > > but I still like mine better) to dovecot 2 and the default "make > > install" no longer loads properly with just "mail_plugins = antispam". > > > > So is $(moduledir)/imap/ no longer used? I also notice acl/imap_acl now, > > but I still see subdirectories for at least auth. > > There's no /imap/ directory anymore, because it was annoying having to > create symlinks to most of the plugins. Then with v2.0 there came new > binaries and it wouldn't have made much sense adding lots of more > directories with lots of more symlinks.. > > > Should I install into just $(moduledir)/ now? > > Yep. Alright, thanks! Do you remember if that would be compatible with older versions as well, or do those require it being in /imap/? johannes From tss at iki.fi Thu Nov 24 16:15:58 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 16:15:58 +0200 Subject: [Dovecot] modules directory In-Reply-To: <1322144020.5366.20.camel@jlt3.sipsolutions.net> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> <1322144020.5366.20.camel@jlt3.sipsolutions.net> Message-ID: <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> On 24.11.2011, at 16.13, Johannes Berg wrote: >>> Should I install into just $(moduledir)/ now? >> >> Yep. > > Alright, thanks! Do you remember if that would be compatible with older > versions as well, or do those require it being in /imap/? What changed was that the default mail_plugin_dir changed from being $moduledir/imap to just plain $moduledir. So older versions require it being in imap/ by default. From johannes at sipsolutions.net Thu Nov 24 16:19:41 2011 From: johannes at sipsolutions.net (Johannes Berg) Date: Thu, 24 Nov 2011 15:19:41 +0100 Subject: [Dovecot] modules directory In-Reply-To: <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> References: <1322140266.5366.8.camel@jlt3.sipsolutions.net> <1322144020.5366.20.camel@jlt3.sipsolutions.net> <1025F181-A647-4E9B-BEFB-C8634091FFD8@iki.fi> Message-ID: <1322144381.5366.21.camel@jlt3.sipsolutions.net> On Thu, 2011-11-24 at 16:15 +0200, Timo Sirainen wrote: > On 24.11.2011, at 16.13, Johannes Berg wrote: > > >>> Should I install into just $(moduledir)/ now? > >> > >> Yep. > > > > Alright, thanks! Do you remember if that would be compatible with older > > versions as well, or do those require it being in /imap/? > > What changed was that the default mail_plugin_dir changed from being > $moduledir/imap to just plain $moduledir. So older versions require it > being in imap/ by default. Ok, thanks. I'll find a way to make it depend on the version. johannes From janfrode at tanso.net Thu Nov 24 16:33:21 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 24 Nov 2011 15:33:21 +0100 Subject: [Dovecot] Questions about Proxy/Director (was: Re: Indexes to MLC-SSD) In-Reply-To: <4ECE483B.5090208@wk-serv.de> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> Message-ID: <20111124143321.GB28600@dibs.tanso.net> On Thu, Nov 24, 2011 at 02:35:55PM +0100, Patrick Westenberg wrote: > > - What happens if the MTA accepted a message but the backend-server > fails and is unable to store it? Your MTA will be notified about the failure trough SMTP/LMTP error codes, and retry if it was a temp-failure. > - What about load-balancing and/or failover? > Maybe all users on backend-server1 are fetching their mails causing > high load while backend-server2 is idle. > Is it possible to specify more than one backend-server IPs as I can > do for the database hosts for userdb queries? We use a clusterfs (so it's not very important which backend server the user is directed to), and have configured the dovecot director with: director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 passdb { args = proxy=y nopassword=y driver = static } which gives us loadbalancing of new logins, and also failover should a backend server fail. If you want to work with at local index and maildirs on NFS, you'll need to configure something smarter for selecting backend server. Maybe use proxy extrafield to specify a host for the user, and point this at a failover IP that will hang on the preferred server by default, but fail over to another working server if the preferred server fails.. http://wiki2.dovecot.org/PasswordDatabase/ExtraField -jf From simon.brereton at buongiorno.com Thu Nov 24 16:42:49 2011 From: simon.brereton at buongiorno.com (Simon Brereton) Date: Thu, 24 Nov 2011 09:42:49 -0500 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111241122.21131.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: On Nov 24, 2011 4:22 AM, "Olli R?is?nen" wrote: > > Hello, > > I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. > Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > directory should not be the same as mail directory. When I create new user > accounts with PostfixAdmin the mail directory is not a problem but is there > some smart way to create the home directory as well (locating it for instance > in '/srv/vmail/%d/%n/home')? Why do you need a home directory for virtual users? Simon From JTRUTWIN at CSBSJU.EDU Thu Nov 24 16:46:57 2011 From: JTRUTWIN at CSBSJU.EDU (Trutwin, Joshua) Date: Thu, 24 Nov 2011 14:46:57 +0000 Subject: [Dovecot] POP - preventing re-download with server move Message-ID: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> Hi, Curious if anyone has any good suggestions to handle this. I try to get my hosting users to use IMAP but most use Outhouse and the IMAP support is somewhat annoying for many so I typically have them change their settings when using POP to leave copies on the server. That way when they get the inevitable virus or wanna use webmail they have a backup copy and webmail sees their saved mail. So now I have to move my sites to a new server and I'm trying to avoid having all these messages re-downloaded by POP clients. Some of the sites I've moved with low email usage have just dealt with the re-download and cleaned up afterwards. I have a few other accounts though that I'm more concerned about that have massive amounts of stored email. The mail is stored in Maildirs - a sample file: 1280409166.23580.foo:2,S If I move this to server "bar" would renaming the file to 1280409166.23580.bar:2,S prevent this somehow? Old server uses qmail-pop, new server uses dovecot 2.0. Thanks for any tips, Josh From robert at schetterer.org Thu Nov 24 17:07:44 2011 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 24 Nov 2011 16:07:44 +0100 Subject: [Dovecot] POP - preventing re-download with server move In-Reply-To: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> References: <710C58696EA3BC42B425E4DBB39C1D5E48552AAE@MAIL-MBX2.ad.csbsju.edu> Message-ID: <4ECE5DC0.4000503@schetterer.org> Am 24.11.2011 15:46, schrieb Trutwin, Joshua: > Hi, > > Curious if anyone has any good suggestions to handle this. > > I try to get my hosting users to use IMAP but most use Outhouse and > the IMAP support is somewhat annoying for many so I typically have > them change their settings when using POP to leave copies on the > server. That way when they get the inevitable virus or wanna use > webmail they have a backup copy and webmail sees their saved mail. > > So now I have to move my sites to a new server and I'm trying to > avoid having all these messages re-downloaded by POP clients. Some > of the sites I've moved with low email usage have just dealt with the > re-download and cleaned up afterwards. I have a few other accounts > though that I'm more concerned about that have massive amounts of > stored email. > > The mail is stored in Maildirs - a sample file: 1280409166.23580.foo:2,S > > If I move this to server "bar" would renaming the file to > 1280409166.23580.bar:2,S prevent this somehow? > > Old server uses qmail-pop, new server uses dovecot 2.0. > > Thanks for any tips, > > Josh perhaps you can do it i.e like this use imapsync with diff between the two servers then block pop3 with firewall for public on the old one, run a last sync session, change dns and/or ip etc , users use new server i think this is the most secured way, i did it like this the problem by redownload may stay cause the pop3 clients have their own list of mail downloaded etc build by i.e with some one hashes i.e. http://kb.mozillazine.org/Popstate.dat and i dont know some way to press them to download anything -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Thu Nov 24 17:37:53 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 17:37:53 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: On 24.11.2011, at 16.42, Simon Brereton wrote: >> Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > > Why do you need a home directory for virtual users? It says right there in the wiki. From joseba.torre at ehu.es Thu Nov 24 17:38:55 2011 From: joseba.torre at ehu.es (Joseba Torre) Date: Thu, 24 Nov 2011 16:38:55 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: <4ECE650F.2070700@ehu.es> El 24/11/11 15:42, Simon Brereton escribi?: > On Nov 24, 2011 4:22 AM, "Olli R?is?nen" wrote: >> >> Hello, >> >> I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. >> Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home >> directory should not be the same as mail directory. When I create new user >> accounts with PostfixAdmin the mail directory is not a problem but is > there >> some smart way to create the home directory as well (locating it for > instance >> in '/srv/vmail/%d/%n/home')? > > Why do you need a home directory for virtual users? For storing files wich are not mails. Sieve rules, subscription files and so on. Aaaaaaaaaaaagur. From mlists at edicom.eu Thu Nov 24 18:11:11 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 17:11:11 +0100 Subject: [Dovecot] Problem with lmtp proxy Message-ID: <201111241711.11504.mlists@edicom.eu> Hello everyone, I have set up postfix to deliver mails to dovecot (2.0.16) using LMTP. On the other hand, I've successfully configured the IMAP proxy setting in dovecot in order to be able to distribute mailboxes among different servers. I wanted to do the same proxy at LMTP level, but it's not working. If I put lmtp_proxy = no, then everything works ok (assuming the mailbox is local), but when I set lmtp_proxy = yes then the user is not found when deliverying the message via LMTP, so the mail remains in the postfix queue. Users are validated through active directory. However, this AD hasn't the SFU installed, thus its LDAP schema doesn't provide me with the required uid, gid, etc. To solve this, I have winbind configured in the system, so I'm doing this: - To validate users *and* be able to set the proxy extra fields, I use ldap as "passdb" (I believe it's not possible to use proxies with PAM). I'm using some LDAP field to store the host that has the user's mailbox. - In order to get the user account data that is not available in the AD, I use passwd as "userdb". I know that when using LMTP with proxy, a passdb needs to be configured. I assume this is needed for the service to lookup for the appropriate host to send the message. So, initially I set ldap as the passdb for LMTP, but just because it didn't work I put both ldap and passwd, and even a userdb (passwd), but the problems remain the same. Anyway the ldap should be the right one as it is there where I have the host information for each user. I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! This is my current config: ************* dovecot -n ************** # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37-gentoo-edicom-1104 x86_64 Gentoo Base System release 1.12.14 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_username_format = %n auth_verbose = yes base_dir = /var/run/dovecot/ listen = * lmtp_proxy = yes login_trusted_networks = 127.0.0.1 mail_debug = yes mail_location = maildir:~/maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at domain.com protocols = imap pop3 sieve lmtp quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = root mode = 0600 user = root } } service imap { vsz_limit = 512 M } service lmtp { inet_listener lmtp { address = 192.168.0.90 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } ssl_cert = , res_matched: <> dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_free_request (origid 1, msgid 1) dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_parse_result dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_msgfree dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_result ld 0x16518d0 msgid -1 dovecot [2011-11-24 15:53:50] [err] auth: Error: wait4msg ld 0x16518d0 msgid -1 (timeout 0 usec) dovecot [2011-11-24 15:53:50] [err] auth: Error: wait4msg continue ld 0x16518d0 msgid -1 all 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Connections: dovecot [2011-11-24 15:53:50] [err] auth: Error: * host: domain.com port: 389 (default) dovecot [2011-11-24 15:53:50] [err] auth: Error: refcnt: 1 status: Connected dovecot [2011-11-24 15:53:50] [err] auth: Error: last used: Thu Nov 24 15:53:50 2011 dovecot [2011-11-24 15:53:50] [err] auth: Error: dovecot [2011-11-24 15:53:50] [err] auth: Error: dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Outstanding Requests: dovecot [2011-11-24 15:53:50] [err] auth: Error: Empty dovecot [2011-11-24 15:53:50] [err] auth: Error: ld 0x16518d0 request count 0 (abandoned 0) dovecot [2011-11-24 15:53:50] [err] auth: Error: ** ld 0x16518d0 Response Queue: dovecot [2011-11-24 15:53:50] [err] auth: Error: Empty dovecot [2011-11-24 15:53:50] [err] auth: Error: ld 0x16518d0 response count 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_chkResponseList ld 0x16518d0 msgid -1 all 0 dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_chkResponseList returns ld 0x16518d0 NULL dovecot [2011-11-24 15:53:50] [err] auth: Error: ldap_int_select dovecot [2011-11-24 15:53:50] [info] lmtp(10506): Disconnect from local: Client quit (in reset) **************************************************** From tom at whyscream.net Thu Nov 24 18:13:47 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 24 Nov 2011 17:13:47 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111241122.21131.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> Message-ID: <4ECE6D3B.6020208@whyscream.net> On 24-11-11 10:22, Olli R?is?nen wrote: > Hello, > > I,m using Dovecot 1.2.9, now finishing a migration from Courier IMAP. > Documentation (http://wiki.dovecot.org/VirtualUsers/Home) says that home > directory should not be the same as mail directory. When I create new user > accounts with PostfixAdmin the mail directory is not a problem but is there > some smart way to create the home directory as well (locating it for instance > in '/srv/vmail/%d/%n/home')? > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail as mailstore? Otherwise, postfixadmin has a post-create hook that is able to execute arbitrary scripts, creating homedirs should be among the included examples. -- Tom From mlists at edicom.eu Thu Nov 24 18:27:38 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 17:27:38 +0100 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241711.11504.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> Message-ID: <201111241727.38458.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 17:11:11 Miguel Tormo escribi?: > dn = cn=binduser,cn=Users,dc=edicom,dc=es Obviously this part should be "dn = cn=binduser,cn=Users,dc=domain,dc=com" to be coherent with the rest. I replace the real domain and user names used, but forgot this part :S. From ecasarero at gmail.com Thu Nov 24 18:30:03 2011 From: ecasarero at gmail.com (Eduardo Casarero) Date: Thu, 24 Nov 2011 13:30:03 -0300 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241727.38458.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> <201111241727.38458.mlists@edicom.eu> Message-ID: 2011/11/24 Miguel Tormo > El Jueves, 24 de Noviembre de 2011 17:11:11 Miguel Tormo escribi?: > > dn = cn=binduser,cn=Users,dc=edicom,dc=es > > Obviously this part should be "dn = cn=binduser,cn=Users,dc=domain,dc=com" > to be coherent with the rest. I replace the real domain and user names > used, but forgot this part :S. > > From tss at iki.fi Thu Nov 24 18:58:40 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 18:58:40 +0200 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <201111241711.11504.mlists@edicom.eu> References: <201111241711.11504.mlists@edicom.eu> Message-ID: <1322153921.24607.124.camel@hurina> On Thu, 2011-11-24 at 17:11 +0100, Miguel Tormo wrote: > I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! The problem is: > ************** cat dovecot-ldap.conf.ext ************** > auth_bind = yes auth_bind=yes requires a user authentication, but LMTP of course can't authenticate a user. So it fails: > dovecot [2011-11-24 15:53:50] [debug] auth: Debug: password(myuser): passdb doesn't support credential lookups Either try to get auth_bind=no working, or I think you can also set up a separate passdb for lmtp: protocol lmtp { passdb { driver = ldap args = some-other-ldap.conf } } And in this other ldap.conf have auth_bind=no, and possibly return password field always as something like "foo". From mlists at edicom.eu Thu Nov 24 19:26:07 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 18:26:07 +0100 Subject: [Dovecot] Problem with lmtp proxy In-Reply-To: <1322153921.24607.124.camel@hurina> References: <201111241711.11504.mlists@edicom.eu> <1322153921.24607.124.camel@hurina> Message-ID: <201111241826.07847.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 17:58:40 Timo Sirainen escribi?: > On Thu, 2011-11-24 at 17:11 +0100, Miguel Tormo wrote: > > I think I'm missing something but I can't find what it is. To summarize: with lmtp_proxy = no it does work, with lmtp_proxy = yes it doesn't. Thank you for your help! > > The problem is: > > > ************** cat dovecot-ldap.conf.ext ************** > > auth_bind = yes > > auth_bind=yes requires a user authentication, but LMTP of course can't > authenticate a user. So it fails: > > > dovecot [2011-11-24 15:53:50] [debug] auth: Debug: password(myuser): passdb doesn't support credential lookups > > Either try to get auth_bind=no working, or I think you can also set up a > separate passdb for lmtp: > > protocol lmtp { > passdb { > driver = ldap > args = some-other-ldap.conf > } > } > > And in this other ldap.conf have auth_bind=no, and possibly return > password field always as something like "foo". > I just configured your suggestion (set a separate passdb for lmtp with ldap and auth_bind = no ) and it works perfectly. Thank you very much! From lists at luigirosa.com Thu Nov 24 20:04:12 2011 From: lists at luigirosa.com (Luigi Rosa) Date: Thu, 24 Nov 2011 19:04:12 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322095616.24607.108.camel@hurina> References: <1322095616.24607.108.camel@hurina> Message-ID: <4ECE871C.3000804@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timo Sirainen said the following on 24/11/11 01:46: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz Works as advertised. I noticed this warining: Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < 4096), because of service auth { client_limit } What does exactly means? The server we are talking about is my home server, a CentOS/64 with a dozen of mailboxes. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Computers make excellent and efficient servants, but I have no wish to serve under them. Captain, a starship also runs on loyalty to one man. And nothing can replace it or him. --Spock, "The Ultimate Computer" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7OhxwACgkQ3kWu7Tfl6ZSJ0gCeIaS5ksyVHY3O9qOSCEyWUH2V By8AoMbUNSHBBc4102vScrozTOOkOeo+ =AZaL -----END PGP SIGNATURE----- From mlists at edicom.eu Thu Nov 24 20:23:00 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 19:23:00 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <4ECE871C.3000804@luigirosa.com> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> Message-ID: <201111241923.00215.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 19:04:12 Luigi Rosa escribi?: > Timo Sirainen said the following on 24/11/11 01:46: > > > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc1.tar.gz > > Works as advertised. > > I noticed this warining: > > Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < > 4096), because of service auth { client_limit } > > What does exactly means? > > > The server we are talking about is my home server, a CentOS/64 with a dozen of > mailboxes. > The fd limit is the maximum number of concurrently open file descriptors per process. It can be configured on a per-user basis. You can check the actual value running 'ulimit -n'. It can be increased up to the value set in /proc/sys/fs/file-max. You can set it editing /etc/security/limits.conf. What the above comment means is that under max load the auth service could need up to 4096 file descriptors open, that's more than the default in most linux systems (1024). However, in your case with just a dozen of mailboxes I don't think you'll need to change it. From lists at luigirosa.com Thu Nov 24 20:29:41 2011 From: lists at luigirosa.com (Luigi Rosa) Date: Thu, 24 Nov 2011 19:29:41 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111241923.00215.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> Message-ID: <4ECE8D15.6050200@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Miguel Tormo said the following on 24/11/11 19:23: > The fd limit is the maximum number of concurrently open file descriptors > per process. It can be configured on a per-user basis. You can check the > actual value running 'ulimit -n'. It can be increased up to the value set > in /proc/sys/fs/file-max. You can set it editing > /etc/security/limits.conf. What the above comment means is that under max > load the auth service could need up to 4096 file descriptors open, that's > more than the default in most linux systems (1024). > > However, in your case with just a dozen of mailboxes I don't think you'll > need to change it. Thank you for the explaination. Ciao, luigi - -- / +--[Luigi Rosa]-- \ A woman was in love with fourteen soldiers. It was clearly platoonic. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk7OjRUACgkQ3kWu7Tfl6ZQXAwCgyD6LAUYQX7/x1PIFda5CoFjx IYsAniqFYvdk6y982XjvWsWc4G5t9zEC =70Em -----END PGP SIGNATURE----- From olli.raisanen at telemail.fi Thu Nov 24 21:23:33 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-1?q?R=E4is=E4nen?=) Date: Thu, 24 Nov 2011 21:23:33 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <4ECE6D3B.6020208@whyscream.net> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> Message-ID: <201111242123.34156.olli.raisanen@telemail.fi> Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: > > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail > as mailstore? Well, yes - at least the home directory will be created that way. Still, PostfixAdmin will not store the path in the maibox table home field. Perhaps it is enough to include the path in the Dovecot user_query? > Otherwise, postfixadmin has a post-create hook that is able to execute > arbitrary scripts, creating homedirs should be among the included > examples. If you are referring to maildir_name_hook in config.inc.php script, no, homedirs are not included (in Postfix Admin version 2.3.4). Thank you anyway, Tom! I must consider your advice. Olli From tss at iki.fi Thu Nov 24 21:25:47 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 21:25:47 +0200 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111241923.00215.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> Message-ID: <1322162747.24607.129.camel@hurina> On Thu, 2011-11-24 at 19:23 +0100, Miguel Tormo wrote: > The fd limit is the maximum number of concurrently open file descriptors per process. It can be configured on a per-user basis. > You can check the actual value running 'ulimit -n'. It can be increased up to the value set in /proc/sys/fs/file-max. You can set it editing /etc/security/limits.conf. I'm not sure if changing /etc/security/limits.conf helps. It's probably only used by PAM when user logs in, so if Dovecot is started in system bootup it's unlikely to have been even read yet. Also I think some OSes override the limits in /etc/init.d/ scripts. Of course, I could be completely wrong in all of the above, I haven't really tested any of it :) > What the above comment means is that under max load the auth service could need up to 4096 file descriptors open, that's more than the default in most linux systems (1024). I wonder if there's a good reason for why auth default should be that high. Probably in earlier times imap/pop3 processes just weren't disconnecting early enough from auth. So I guess I'll just drop it back to default and this warning would go away in most systems. I had started to get a little bit annoyed at that warning myself also. From tss at iki.fi Thu Nov 24 21:29:46 2011 From: tss at iki.fi (Timo Sirainen) Date: Thu, 24 Nov 2011 21:29:46 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <201111242123.34156.olli.raisanen@telemail.fi> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> <201111242123.34156.olli.raisanen@telemail.fi> Message-ID: <1322162990.24607.131.camel@hurina> On Thu, 2011-11-24 at 21:23 +0200, Olli R?is?nen wrote: > Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: > > > > How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail > > as mailstore? > > Well, yes - at least the home directory will be created that way. Still, > PostfixAdmin will not store the path in the maibox table home field. Perhaps > it is enough to include the path in the Dovecot user_query? It would be better if you used PostfixAdmin's "maildir" field as the home directory, and then mail_location=~/Maildir. Like the sql queries being: password_query = \ SELECT username AS user, password, \ CONCAT('/home/vmail/', maildir) AS userdb_home, \ CONCAT('*:bytes=', quota) AS userdb_quota_rule \ FROM mailbox \ WHERE username = '%u' and ACTIVE = '1' user_query = \ SELECT CONCAT('/home/vmail/', maildir) AS home, \ CONCAT('*:bytes=', quota) AS quota_rule \ FROM mailbox \ WHERE username = '%u' AND active = '1' I used these in another PostfixAdmin server's install. From tom at whyscream.net Thu Nov 24 21:50:39 2011 From: tom at whyscream.net (Tom Hendrikx) Date: Thu, 24 Nov 2011 20:50:39 +0100 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <1322162990.24607.131.camel@hurina> References: <201111241122.21131.olli.raisanen@telemail.fi> <4ECE6D3B.6020208@whyscream.net> <201111242123.34156.olli.raisanen@telemail.fi> <1322162990.24607.131.camel@hurina> Message-ID: <4ECEA00F.9040306@whyscream.net> On 24-11-11 20:29, Timo Sirainen wrote: > On Thu, 2011-11-24 at 21:23 +0200, Olli R?is?nen wrote: >> Tom Hendrikx kirjoitti 2011-11-24 kello 18:13:47: >>> >>> How about using /srv/vmail/%d/%n for homedir, and /srv/vmail/%d/%n/mail >>> as mailstore? >> >> Well, yes - at least the home directory will be created that way. Still, >> PostfixAdmin will not store the path in the maibox table home field. Perhaps >> it is enough to include the path in the Dovecot user_query? > > It would be better if you used PostfixAdmin's "maildir" field as the > home directory, and then mail_location=~/Maildir. Like the sql queries > being: > > password_query = \ > SELECT username AS user, password, \ > CONCAT('/home/vmail/', maildir) AS userdb_home, \ > CONCAT('*:bytes=', quota) AS userdb_quota_rule \ > FROM mailbox \ > WHERE username = '%u' and ACTIVE = '1' > > user_query = \ > SELECT CONCAT('/home/vmail/', maildir) AS home, \ > CONCAT('*:bytes=', quota) AS quota_rule \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' > > I used these in another PostfixAdmin server's install. > I even simplified this up to: # taken from stock dovecot2 config userdb { driver = static args = uid=vmail gid=mail home=/var/spool/vmail/%u } mail_location = maildir:~/Maildir password_query = SELECT username AS user, password FROM mailbox WHERE username = '%u' AND active = '1' (no quota's, no special cases for home/mail directory configuration) -- Tom From mlists at edicom.eu Thu Nov 24 22:35:09 2011 From: mlists at edicom.eu (Miguel Tormo) Date: Thu, 24 Nov 2011 21:35:09 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322162747.24607.129.camel@hurina> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> Message-ID: <201111242135.09681.mlists@edicom.eu> El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: > I'm not sure if changing /etc/security/limits.conf helps. It's probably > only used by PAM when user logs in, so if Dovecot is started in system > bootup it's unlikely to have been even read yet. Also I think some OSes > override the limits in /etc/init.d/ scripts. Of course, I could be > completely wrong in all of the above, I haven't really tested any of > it Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). Another permanent way between reboots that I just googled is changing this line of includes/linux/fs.h: #define INR_OPEN 1024 /* Initial setting for nfile rlimits */ and recompile the kernel :P. From noel.butler at ausics.net Fri Nov 25 00:30:18 2011 From: noel.butler at ausics.net (Noel Butler) Date: Fri, 25 Nov 2011 08:30:18 +1000 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111242135.09681.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> <201111242135.09681.mlists@edicom.eu> Message-ID: <1322173818.6600.5.camel@tardis> On Thu, 2011-11-24 at 21:35 +0100, Miguel Tormo wrote: > El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: > > I'm not sure if changing /etc/security/limits.conf helps. It's probably > > only used by PAM when user logs in, so if Dovecot is started in system > > bootup it's unlikely to have been even read yet. Also I think some OSes > > override the limits in /etc/init.d/ scripts. Of course, I could be > > completely wrong in all of the above, I haven't really tested any of > > it > > Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). > much simpler way without rebuilding is as Timo suggested, just pick your value #!/bin/sh # Start/stop dovecot. ulimit -n 16384 ... Be careful, building in kernel based higher limits means /every/ service and can lead to DoS, this way limits dovecot to 16384, but keeps other stuff like mysql at 4096, and everything else at 1024, bit safer. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From olli.raisanen at telemail.fi Fri Nov 25 00:43:51 2011 From: olli.raisanen at telemail.fi (Olli =?iso-8859-15?q?R=E4is=E4nen?=) Date: Fri, 25 Nov 2011 00:43:51 +0200 Subject: [Dovecot] How to create home directories for virtual users? In-Reply-To: <1322162990.24607.131.camel@hurina> References: <201111241122.21131.olli.raisanen@telemail.fi> <201111242123.34156.olli.raisanen@telemail.fi> <1322162990.24607.131.camel@hurina> Message-ID: <201111250043.51861.olli.raisanen@telemail.fi> Timo Sirainen kirjoitti 2011-11-24 kello 21:29:46: > > It would be better if you used PostfixAdmin's "maildir" field as the > home directory, and then mail_location=~/Maildir. Thank you, Timo! (And thank you, Tom, for the simplification example!) I never thought ~/Maildir could refer to anything but UNIX system users! :) Timo's queries (beneath) work indeed. However, it took quite a while of testing, and I wonder if it really would cause some trouble if homedir and maildir were kept the same... Regards, Olli > Like the sql queries being: > > password_query = \ > SELECT username AS user, password, \ > CONCAT('/home/vmail/', maildir) AS userdb_home, \ > CONCAT('*:bytes=', quota) AS userdb_quota_rule \ > FROM mailbox \ > WHERE username = '%u' and ACTIVE = '1' > > user_query = \ > SELECT CONCAT('/home/vmail/', maildir) AS home, \ > CONCAT('*:bytes=', quota) AS quota_rule \ > FROM mailbox \ > WHERE username = '%u' AND active = '1' > > I used these in another PostfixAdmin server's install. From pw at wk-serv.de Fri Nov 25 00:58:01 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 24 Nov 2011 23:58:01 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111124143321.GB28600@dibs.tanso.net> References: <1321467345.21919.580.camel@hurina> <4EC403B2.60005@wk-serv.de> <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> Message-ID: <4ECECBF9.5070507@wk-serv.de> Jan-Frode Myklebust schrieb: > We use a clusterfs (so it's not very important which backend server the > user is directed to), and have configured the dovecot director with: > > director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 > director_servers = 192.168.42.15 192.168.42.17 > passdb { > args = proxy=y nopassword=y > driver = static > } > > which gives us loadbalancing of new logins, and also failover should a > backend server fail. That looks nice. What is your frontend configuration? How many users do you have that you need that many backend servers? Are your index stored together with the mails? Patrick From janfrode at tanso.net Fri Nov 25 10:17:15 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 25 Nov 2011 09:17:15 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <4ECECBF9.5070507@wk-serv.de> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> Message-ID: <20111125081715.GA24352@dibs.tanso.net> On Thu, Nov 24, 2011 at 11:58:01PM +0100, Patrick Westenberg wrote: > > What is your frontend configuration? The frontend is two servers running dovecot director, and ucarp for managing failover ip-addresses. ===================================================================== # 2.0.14: /etc/dovecot/dovecot.conf # OS: x86_64 Red Hat Enterprise Linux Server release 5.x director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * lmtp_proxy = yes passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 2051 } service auth { client_limit = 4196 unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 1024 } service lmtp { inet_listener lmtp { address = * port = 24 } } service pop3-login { executable = pop3-login director process_limit = 1024 } ssl_cert = How many users do you have that you need that many backend servers? I don't think I'm allowed to share that externally, sorry. This is the mail-system of an ISP, so we have quite a few users.. :-) > Are your index stored together with the mails? They're not stored together, because we made the error of not creating a home directory for our users, but both maildirs and indexes are stored on similar cluster fs' (GPFS). -jf From pw at wk-serv.de Fri Nov 25 11:02:19 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Fri, 25 Nov 2011 10:02:19 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111125081715.GA24352@dibs.tanso.net> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> <20111125081715.GA24352@dibs.tanso.net> Message-ID: <19dc5fa7b626f394a4e54fb0105ac5e8@wk-serv.de> On Fri, 25 Nov 2011 09:17:15 +0100, Jan-Frode Myklebust wrote: > On Thu, Nov 24, 2011 at 11:58:01PM +0100, Patrick Westenberg wrote: >> >> What is your frontend configuration? > > The frontend is two servers running dovecot director, and ucarp for > managing failover ip-addresses. Two additional directors or the two directors from your posted config (.15/.17)? Will I have to configure external IPs for the backend servers or just internal for the connections to and from the directors? I'm not sure if the director "redirects" the connections to the backend servers or if they pass through on the internal network. >> How many users do you have that you need that many backend servers? > > I don't think I'm allowed to share that externally, sorry. This is > the > mail-system of an ISP, so we have quite a few users.. :-) Ok :-) From joseba.torre at ehu.es Fri Nov 25 12:48:14 2011 From: joseba.torre at ehu.es (Joseba Torre) Date: Fri, 25 Nov 2011 11:48:14 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <201111242135.09681.mlists@edicom.eu> References: <1322095616.24607.108.camel@hurina> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> <201111242135.09681.mlists@edicom.eu> Message-ID: <4ECF726E.6020109@ehu.es> El 24/11/11 21:35, Miguel Tormo escribi?: > El Jueves, 24 de Noviembre de 2011 20:25:47 Timo Sirainen escribi?: >> I'm not sure if changing /etc/security/limits.conf helps. It's probably >> only used by PAM when user logs in, so if Dovecot is started in system >> bootup it's unlikely to have been even read yet. Also I think some OSes >> override the limits in /etc/init.d/ scripts. Of course, I could be >> completely wrong in all of the above, I haven't really tested any of >> it > > Well actually this is tricky, I'm not sure if it would work after a system boot, but I'm sure it works after a reboot of the service. It is also true what you say that some OSes override the limits in the /etc/init.d/ scripts, for example issuing an 'ulimit -n 4096' (if using bash) before dropping privileges (that should be inherited, but if privileges are dropped using "su" or something that uses PAM, then the /etc/security/limits.conf file comes into play again). I tested this "intensively" some time ago, and Timo is right -as usual :) -: modifications in limits.conf are only relevant after a login. This means that they are not enforced at system boot, but they are if you reboot the service by hand (because you have already logged in). The permanent solution is, as Noel Butler has pointed, modifying the startup script and using ulimit before actually starting dovecot. HTH From janfrode at tanso.net Fri Nov 25 13:38:13 2011 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 25 Nov 2011 12:38:13 +0100 Subject: [Dovecot] v2.1.rc1 released In-Reply-To: <1322162747.24607.129.camel@hurina> References: <1322095616.24607.108.camel@hurina> <4ECE871C.3000804@luigirosa.com> <201111241923.00215.mlists@edicom.eu> <1322162747.24607.129.camel@hurina> Message-ID: <20111125113813.GA27970@dibs.tanso.net> On Thu, Nov 24, 2011 at 09:25:47PM +0200, Timo Sirainen wrote: > I'm not sure if changing /etc/security/limits.conf helps. It's probably > only used by PAM when user logs in, so if Dovecot is started in system > bootup it's unlikely to have been even read yet. Also I think some OSes > override the limits in /etc/init.d/ scripts. Of course, I could be > completely wrong in all of the above, I haven't really tested any of > it :) The OP was running Centos, and the Red Hat way of increasing nofile for services is to add the ulimit setting to /etc/sysconfig/dovecot: ulimit -n 32768 This is what we use, and it survives dovecot upgrades (since this is tagged as a configfile in RPM, not as script that will be overwritten on upgrades). -jf From ef at math.uni-bonn.de Fri Nov 25 14:49:03 2011 From: ef at math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) Date: Fri, 25 Nov 2011 13:49:03 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <1320438508.21919.177.camel@hurina> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> Message-ID: <20111125124902.GB27737@gumme.math.uni-bonn.de> > That shouldn't happen because of Dovecot's indexes. Hm, also if the indexes are local? Fine. > Then the 'S' flag is added to the current Maildir filename without > losing any other changes. And this is supposed to work even over NFS? Great. So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? From sf.rique at gmail.com Fri Nov 25 17:29:35 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Fri, 25 Nov 2011 13:29:35 -0200 Subject: [Dovecot] Which FTS to use! Message-ID: Hello, i am currently using squat dovecot, but some users that have lots os emails ( 1GB ) are complaining about speed. Iam using dovecot 2.0.13 on debian 6 my squat conf is plugin { fts = squat fts_squat = partial=4 full=5 } I am thinking in maybe start to use Solr, so any one can help me with an decision? I am probably gonna update dovecot to 2.1 next year. and them maybe change the FTS plugin. This year i will get 10k disk only to store the dovecot index, rigth now they are saved on the same storage that the mails are and also change the "full" parameter to something like 10 or 15 I know i will use more disk space for it, but it will make a lot of differences right? it will match the message without the need of opening it for any search smaller than 10 character i guess. But this will be enough? or Solr or Lucene still better? Anyway, i am not current updating the indexes automatic, but i am planing on it, just not sure what is better, the command doveadm index or doveadm search... with one is better to do the task? Thanks! []'sf.rique From tss at iki.fi Fri Nov 25 18:47:51 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 25 Nov 2011 18:47:51 +0200 Subject: [Dovecot] Which FTS to use! In-Reply-To: References: Message-ID: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> On 25.11.2011, at 17.29, Henrique Santos Fernandes wrote: > Hello, i am currently using squat dovecot, but some users that have lots os > emails ( 1GB ) are complaining about speed. > Iam using dovecot 2.0.13 on debian 6 > > my squat conf is > plugin { > fts = squat > fts_squat = partial=4 full=5 > } > > I am thinking in maybe start to use Solr, so any one can help me with an > decision? With v2.0 Solr is the only other choice. > I am probably gonna update dovecot to 2.1 next year. and them maybe change > the FTS plugin. With v2.1 Lucene is another possibility, and it's somewhat easier to install than Solr. Also v2.1's Solr backend is improved, so I wouldn't really recommend spending a lot of time and disk I/O on v2.0's Solr since you'll need to rebuild the index anyway in v2.1 (or keep using "solr_old" backend with some missing featues). > This year i will get 10k disk only to store the dovecot index, rigth now > they are saved on the same storage that the mails are and also change the > "full" parameter to something like 10 or 15 I know i will use more disk > space for it, but it will make a lot of differences right? it will match > the message without the need of opening it for any search smaller than 10 > character i guess. > > > But this will be enough? or Solr or Lucene still better? The main problem with Squat is its index updating performance, which sucks. > Anyway, i am not current updating the indexes automatic, but i am planing > on it, just not sure what is better, the command doveadm index or doveadm > search... with one is better to do the task? doveadm index From lists at wildgooses.com Fri Nov 25 18:59:22 2011 From: lists at wildgooses.com (Ed W) Date: Fri, 25 Nov 2011 16:59:22 +0000 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <4ECFC96A.9020204@wildgooses.com> On 25/11/2011 12:49, Edgar Fu? wrote: >> That shouldn't happen because of Dovecot's indexes. > Hm, also if the indexes are local? Fine. > >> Then the 'S' flag is added to the current Maildir filename without >> losing any other changes. > And this is supposed to work even over NFS? Great. > > So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? You don't have any Thunderbird clients accessing this box do you? I have some wierd issue where our TB with the option "don't mark message read", still triggers messages to be marked read... Wierdly it only does it on some messages and all those from specific senders - can't correlate it with anything obvious in the message itself though... Regards Ed W From ef at math.uni-bonn.de Fri Nov 25 19:20:55 2011 From: ef at math.uni-bonn.de (=?iso-8859-1?Q?Edgar_Fu=DF?=) Date: Fri, 25 Nov 2011 18:20:55 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <4ECFC96A.9020204@wildgooses.com> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> <4ECFC96A.9020204@wildgooses.com> Message-ID: <476867D4-69AE-4854-96F8-43C4559DC51F@math.uni-bonn.de> > You don't have any Thunderbird clients accessing this box do you? Yes, I have. But I also experienced the problem myself only using mutt and Apple Mail. > I have some w[ei]rd issue where our TB with the option > "don't mark message read", still triggers messages to be marked read It's the other way round. People read their messages, they are flagged as read and all of a sudden revert to unread. From e-frog at gmx.de Fri Nov 25 19:23:59 2011 From: e-frog at gmx.de (e-frog) Date: Fri, 25 Nov 2011 18:23:59 +0100 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <4ECFC96A.9020204@wildgooses.com> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> <4ECFC96A.9020204@wildgooses.com> Message-ID: <4ECFCF2F.50307@gmx.de> On 25.11.2011 17:59, wrote Ed W: > On 25/11/2011 12:49, Edgar Fu? wrote: > You don't have any Thunderbird clients accessing this box do you? I > have some wierd issue where our TB with the option "don't mark message > read", still triggers messages to be marked read... Wierdly it only does > it on some messages and all those from specific senders - can't > correlate it with anything obvious in the message itself though... I had the same issue a while ago with Thunderbird. On one installation it worked on another it didn't. After comparing their config I found the culprit. Changing mail.server.default.mime_parts_on_demand to **false** fixed this issue for me. This setting is discussed in... http://kb.mozillazine.org/Entire_message_fetched_when_opening_a_IMAP_message ... but not mentioning the "don't mark message as read" issue. Regards, e-frog * * Englisch * Deutsch * Englisch * Deutsch From tss at iki.fi Fri Nov 25 19:47:22 2011 From: tss at iki.fi (Timo Sirainen) Date: Fri, 25 Nov 2011 19:47:22 +0200 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <5539B894-4826-480F-95F6-A7D8CBF9B147@iki.fi> On 25.11.2011, at 14.49, Edgar Fu? wrote: >> That shouldn't happen because of Dovecot's indexes. > Hm, also if the indexes are local? Fine. Yep. It just means that indexes then aren't fully up to date. Hmm. I guess possibly one potential problem is if 1. dovecot.index.log says that "set flags to ()" but then it doesn't actually finish that 2. Next time indexes are accessed from second server, and flags are set to (\Seen) 3. And then finally user logs in back to original server, which sees that "aha, flags should be set to ()" and does that. But the problem is the first step. It shouldn't normally do it only halfway through, unless it crashes or something.. Also typically flags aren't replaced, they're added/removed, but this completely depends on the client and I don't remember what Apple Mail does. >> Then the 'S' flag is added to the current Maildir filename without >> losing any other changes. > And this is supposed to work even over NFS? Great. > > So, what can I do to track down the problem as, according to what I read from your answer, it's probably a Dovecot bug? You could try enabling mail_log plugin with flag_change event to see when it actually changes. http://wiki.dovecot.org/Plugins/MailLog From noel.butler at ausics.net Sat Nov 26 02:19:13 2011 From: noel.butler at ausics.net (Noel Butler) Date: Sat, 26 Nov 2011 10:19:13 +1000 Subject: [Dovecot] Seen flag getting lost In-Reply-To: <20111125124902.GB27737@gumme.math.uni-bonn.de> References: <20111025110238.GB8900@gumme.math.uni-bonn.de> <1320438508.21919.177.camel@hurina> <20111125124902.GB27737@gumme.math.uni-bonn.de> Message-ID: <1322266753.6288.6.camel@tardis> On Fri, 2011-11-25 at 13:49 +0100, Edgar Fu? wrote: > > Then the 'S' flag is added to the current Maildir filename without > > losing any other changes. > And this is supposed to work even over NFS? Great. Absolutely, always has worked over NFS -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From rajeev at teamsg.in Sat Nov 26 08:32:48 2011 From: rajeev at teamsg.in (Rajeev Sharma) Date: Sat, 26 Nov 2011 12:02:48 +0530 Subject: [Dovecot] Dovecot 1.2 ACL problem. Message-ID: Sir, I have migrate from courier-imap to dovecot without any problem but on courier in have shared folder which access on the basis on rights. I have manage these folders in dovecot but i have one problem all the shared folder is accessible by all user whether they have rights or not i have place dovecot-acl in shared folder and assign user my dovecot-acl is user=rajeev at teamsg.in tr. But all other user which is not listing on dovecot-acl also able to access these folder. Here is the output of dovecot -n. # 1.2.17: /usr/local/etc/dovecot.conf # OS: Linux 2.6.18-194.3.1.el5 i686 CentOS release 5.5 (Final) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log ssl_listen: 192.168.0.251:993 ssl_cert_file: /var/qmail/control/servercert.pem ssl_key_file: /var/qmail/control/servercert.pem login_dir: /var/run/dovecot//login login_executable: /usr/local/libexec/dovecot/imap-login first_valid_uid: 508 last_valid_uid: 508 mail_location: maildir:~/Maildir namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes namespace: type: shared separator: . prefix: INBOX.Shared. location: maildir:/home/vpopmail/domains/sgcricket.com/Maildir/Out:INDEX=~/Maildir/Shared list: children auth default: user: vpopmail passdb: driver: vpopmail userdb: driver: vpopmail plugin: acl: vfile Please help whats wrong i am doing. Rajeev Sharma. From dan.colascione at gmail.com Sat Nov 26 08:36:39 2011 From: dan.colascione at gmail.com (Daniel Colascione) Date: Fri, 25 Nov 2011 22:36:39 -0800 Subject: [Dovecot] Assertion failure in 2.0.16 - SEARCH () Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nov 25 22:25:39 dancol dovecot: master: Error: service(imap): child 25115 killed with signal 6 (core dumps disabled) Nov 25 22:25:59 dancol dovecot: imap(dancol): Panic: file mail-search-build.c: line 59 (mail_search_build_key_int): assertion failed: (sarg->value.subargs != NULL) Nov 25 22:25:59 dancol dovecot: imap(dancol): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3d291) [0xb7636291] -> /usr/lib/dovecot/libdovecot.so.0(+0x3d2ff) [0xb76362ff] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0xb760bb7a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x24109) [0xb7692109] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x24177) [0xb7692177] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_build+0x82) [0xb7692242] - -> dovecot/imap [dancol 127.0.0.1 SEARCH](imap_search_args_build+0x4a) [0x805db5a] -> dovecot/imap [dancol 127.0.0.1 SEARCH](cmd_search+0xbf) [0x80537ef] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x80568cc] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x80569a2] -> dovecot/imap [dancol 127.0.0.1 SEARCH](client_handle_input+0x175) [0x8056c25] -> dovecot/imap [dancol 127.0.0.1 SEARCH](client_input+0x5f) [0x80574cf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x51) [0xb76430d1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0xb7644193] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0xb7643050] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2a) [0xb762ec1a] -> dovecot/imap [dancol 127.0.0.1 SEARCH](main+0x315) [0x805fef5] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb74b6c76] -> dovecot/imap [dancol 127.0.0.1 SEARCH]() [0x804eb81] $ telnet localhost imap Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. . LOGIN dancol XXXXXXXX . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in . SELECT rss * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8 EXISTS * 0 RECENT * OK [UIDVALIDITY 1322264663] UIDs valid * OK [UIDNEXT 11] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. . SEARCH () Connection closed by foreign host. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk7QiNcACgkQ17c2LVA10VvhHACgihmbmTjVNl/PbKcreIDRTKpC 5wkAoJd9FHXM4GhPfnVUjwrpdsqnVQ6E =RlpQ -----END PGP SIGNATURE----- From mark.zealey at webfusion.com Sat Nov 26 18:33:30 2011 From: mark.zealey at webfusion.com (Mark Zealey) Date: Sat, 26 Nov 2011 18:33:30 +0200 Subject: [Dovecot] using dsync to convert mailboxes looses caching options Message-ID: <4ED114DA.7030707@webfusion.com> Hi there, We're trying to convert users from Maildir to sdbox at present; I'm using dsync to achieve this (2.0.16) however when the user's have been converted we only get minimal information in the caching files. Is there some way to preserve all the caching decisions that were previously made so that when the user logs in to the new mailbox we don't have to cause an io storm rebuilding the cache that we know was good? Dovecot seems to be partially doing this - if i remove the logs/cache from the source mailbox no cache files are built in the conversion; if i put them back then we get a cache file built but it only contains a few bits of information (guid, date.save). Looking into this a bit further i find that when the caches are present at source the fields are preserved but the 'last used' date and caching decisions are not which I suspect means dsync doesn't bother caching on import - only fields with a yes decision in the source are copied (but their decision is only copied as a tmp with the date of import). For example: Source idxview cache: -- Cache fields -- # Name Type Size Dec Last used 0: flags bit 4 tmp 2011-11-25 16:09 1: date.received fix 4 yes 2011-11-26 16:17 2: size.virtual fix 8 tmp 2011-11-25 16:09 3: imap.bodystructure str - tmp 2011-11-25 16:09 4: mime.parts var - tmp 2011-11-25 16:09 5: hdr.IMPORTANCE hdr - tmp 2011-11-25 16:09 6: hdr.X-PRIORITY hdr - tmp 2011-11-25 16:09 7: hdr.CONTENT-TYPE hdr - tmp 2011-11-25 16:09 ... 18: date.sent fix 8 no 1970-01-01 01:00 19: date.save fix 4 yes 2011-11-26 16:17 20: size.physical fix 8 no 1970-01-01 01:00 21: imap.body str - no 1970-01-01 01:00 ... (24 total entries. stripped ones are just hdr. with a tmp decision) dst cache: # Name Type Size Dec Last used 0: flags bit 4 tmp 1970-01-01 01:00 1: date.sent fix 8 no 1970-01-01 01:00 2: date.received fix 4 tmp 1970-01-01 01:00 3: date.save fix 4 tmp 2011-11-26 16:19 4: size.virtual fix 8 tmp 1970-01-01 01:00 5: size.physical fix 8 no 1970-01-01 01:00 6: imap.body str - no 1970-01-01 01:00 7: imap.bodystructure str - tmp 1970-01-01 01:00 8: imap.envelope str - no! 1970-01-01 01:00 9: pop3.uidl str - no 1970-01-01 01:00 10: guid str - tmp 2011-11-26 16:19 11: mime.parts var - tmp 1970-01-01 01:00 12: hdr.IMPORTANCE hdr - tmp 1970-01-01 01:00 13: hdr.X-PRIORITY hdr - tmp 1970-01-01 01:00 ... and dst cached message for example: RECORD: seq=57, uid=12207, flags=0x09 (Seen Answered) - ext 1 modseq : 1 (0100000000000000) - ext 3 cache : 5516 (8c150000) - cache offset=5516 size=92, prev_offset = 0 - guid: 1321895282.XXXt,S=7399760,W=7496225 - date.save: 1322324286 (3e11d14e) src idx record for this message contains pretty much all fields. Thanks, Mark From sf.rique at gmail.com Sun Nov 27 15:17:16 2011 From: sf.rique at gmail.com (Henrique Santos Fernandes) Date: Sun, 27 Nov 2011 11:17:16 -0200 Subject: [Dovecot] Which FTS to use! In-Reply-To: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> References: <0E0758C1-349F-49C1-8E4D-74A737BAEF96@iki.fi> Message-ID: So i guess i will just stay with squat until i upgrade to 2.1 and then move to solr.. And make the cron update daily for active users! Thanks a lot! []'sf.rique On Fri, Nov 25, 2011 at 2:47 PM, Timo Sirainen wrote: > On 25.11.2011, at 17.29, Henrique Santos Fernandes wrote: > > > Hello, i am currently using squat dovecot, but some users that have lots > os > > emails ( 1GB ) are complaining about speed. > > Iam using dovecot 2.0.13 on debian 6 > > > > my squat conf is > > plugin { > > fts = squat > > fts_squat = partial=4 full=5 > > } > > > > I am thinking in maybe start to use Solr, so any one can help me with an > > decision? > > With v2.0 Solr is the only other choice. > > > I am probably gonna update dovecot to 2.1 next year. and them maybe > change > > the FTS plugin. > > With v2.1 Lucene is another possibility, and it's somewhat easier to > install than Solr. Also v2.1's Solr backend is improved, so I wouldn't > really recommend spending a lot of time and disk I/O on v2.0's Solr since > you'll need to rebuild the index anyway in v2.1 (or keep using "solr_old" > backend with some missing featues). > > > This year i will get 10k disk only to store the dovecot index, rigth now > > they are saved on the same storage that the mails are and also change > the > > "full" parameter to something like 10 or 15 I know i will use more disk > > space for it, but it will make a lot of differences right? it will match > > the message without the need of opening it for any search smaller than > 10 > > character i guess. > > > > > > But this will be enough? or Solr or Lucene still better? > > The main problem with Squat is its index updating performance, which sucks. > > > Anyway, i am not current updating the indexes automatic, but i am planing > > on it, just not sure what is better, the command doveadm index or doveadm > > search... with one is better to do the task? > > doveadm index From Patrick.Wood at tups.net.au Mon Nov 28 05:24:54 2011 From: Patrick.Wood at tups.net.au (Patrick Wood) Date: Mon, 28 Nov 2011 03:24:54 +0000 Subject: [Dovecot] POP3/IMAP crash signal 10 Message-ID: <0D810DB78132A8488DE17E655BF29B685C5EAA5C@TUPSDC01.tups.net.au> Hi, I'm building a Postfix/Dovecot mail server and while I am able to send/receive emails using telnet, after establishing a connection to Dovecot via a client (Mail Live, Thunderbird etc) the following appears in the logs: Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.info] pop3-login: Login: user=, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, mpid=282 78 Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.error] master: Error: service(pop3-login): child 28277 killed with signal 10 (core not dumped - set service pop3-log in { drop_priv_before_exec=yes }) Nov 28 14:11:02 mailserver dovecot: [ID 583609 mail.info] pop3(user at domain.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/5, size=3250 I also experience very poor connectivity to the server, with Microsoft Mail and Thunderbird regularly failing to connect (connection errors in the client and no login attempt in the dovecot logs). I assume this is related to the issue above. I am running dovecot 2.0.16 on Solaris 10 SPARC using a ZFS filesystem for mailbox storage. Here is my configuration: # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4u zfs auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:/storage0/home/vmail/%d/%n/Maildir mail_plugins = quota passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=1G } postmaster_address = root at domain.com sendmail_path = /usr/lib/sendmail ssl_cert = Hello, I tried to upgrade from 2.0.15 to 2.1.rc1 over the weekend but ran into some problems with squat. My two largest mailboxes were corrupted, producing this error: Nov 27 17:38:12 kerio1 dovecot[42860]: imap(bigmailbox): Error: Corrupted squat uidlist file /var/mail/metro-email.com/bigmailbox/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.search.uids: wrong indexid There is a good chance that I had shot my own foot, by forgetting to update postfix's dovecot-lda reference to the new instance. I had also enabled an inet based LMTP listener, although it is not being used yet. Even using non-corrupted indexes, it seemed that FTS was not being used. A basic search such as this one took many minutes to complete: doveadm search -u jsample mailbox INBOX body eps I found messages like this in mail.log after performing the search: Nov 27 18:21:39 kerio1 dovecot[42860]: indexer-worker(jsample): Indexed 0 messages in Drafts Nov 27 18:21:39 kerio1 dovecot[42860]: indexer-worker(jsample): Indexed 0 messages in INBOX When I rolled back to 2.0.15, the same search returned almost immediately. The larger mailboxes went quickly again, but only after they had a chance to rebuild. A few other notes about our configuration: We are on PPC, with the stats service disabled (because of kqueue issue discussed earlier). I made some minor changes in my config since the previous installed version (diff below). mail_location override is in place for all users via password file, e.g.: userdb_mail=mdbox:/var/mail/metro-email.com/jsample/mdbox Thanks, Tony kerio1:~ metroadmin$ /opt/dovecot-2.1.rc1/bin/doveconf -n -c /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf # 2.1.rc1: /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_mechanisms = plain login lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_proxy = yes mail_location = maildir:%h mail_plugins = fts fts_squat mail_log notify mail_privileged_group = mail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader mdbox_rotate_size = 10 M namespace { inbox = yes location = prefix = separator = / } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users driver = passwd-file } plugin { fts = squat mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from subject sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at metro-email.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = metroadmin } } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 192.168.20.10 127.0.0.1 port = 24 } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_cert = # 2.1.rc1: /opt/dovecot-2.1.rc1/etc/dovecot/dovecot.conf 9a10 > lmtp_proxy = yes 15c16 < managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave --- > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader 24c25 < args = scheme=CRYPT username_format=%u /opt/dovecot-2.0.15/etc/dovecot/users --- > args = scheme=CRYPT username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users 50a52,57 > service lmtp { > inet_listener lmtp { > address = 192.168.20.10 127.0.0.1 > port = 24 > } > } 53a61,65 > service stats { > fifo_listener stats-mail { > mode = 00 > } > } 57c69 < args = username_format=%u /opt/dovecot-2.0.15/etc/dovecot/users --- > args = username_format=%u /opt/dovecot-2.1.rc1/etc/dovecot/users kerio1:~ metroadmin$ From msbrogli-dovecot at vialink.com.br Tue Nov 29 00:14:19 2011 From: msbrogli-dovecot at vialink.com.br (Marcelo Salhab Brogliato) Date: Mon, 28 Nov 2011 20:14:19 -0200 Subject: [Dovecot] Fault tolerant architecture Message-ID: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Hi, I'm new to this list and want your help. I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). I'm using dovecot 1.2.9. I guess you already had some of these problems or maybe in another architecture these neither exists. Thanks for any help, Marcelo From list at airstreamcomm.net Tue Nov 29 01:16:37 2011 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 28 Nov 2011 17:16:37 -0600 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <8c0fae947f9c32073f798299f027684c@mail.airstreamcomm.net> On Mon, 28 Nov 2011 20:14:19 -0200, Marcelo Salhab Brogliato wrote: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we > have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. > Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS > using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is > using two IP addresses in our DNS records. > > About NFS using local indexes, I'm configuring a test server. But how to > configure local indexes when my mail_location comes from sql (userdb_home > actually). > I'm using dovecot 1.2.9. > > I guess you already had some of these problems or maybe in another > architecture these neither exists. > > Thanks for any help, > > Marcelo Marcelo, There are a number of ways to bring HA to a cluster of mail servers, one that we have experimented with lately is a bit exotic, but might work for you. >From the base layer we are experimenting with GlusterFS, distributed and replicated file system that offers very simple management and high availablity. It does run in userspace, which according to some will suffer from performance bottlenecks, but so far we have not seen any serious problems while running on 15k disks in raid 10. Assuming you have two virtual machines you could create a distributed file system between them, and have a mirrored copy of the data on both. Next is dovecot/postfix/webmail which would be setup to use the local GlusterFS mount on the system containing the mail storage and indexes. To provide HA on the connectivity side we used ucarp, which creates a virtual IP address between two servers and fails that virtual IP over to another server in the event of a server going down. I personally would never use DNS load balancing (two IPs for one DNS name) as it would round robin to each server regardless of whether it is running or not. This is somewhat exotic, but it works and provides a very high level of availability. However with HA comes more complexity and management. Good luck and let me know if you would like more specifics. From seandarcy2 at gmail.com Tue Nov 29 01:50:18 2011 From: seandarcy2 at gmail.com (sean darcy) Date: Mon, 28 Nov 2011 18:50:18 -0500 Subject: [Dovecot] can not get t'bird to create folder Message-ID: dovecot-2.0.16 on Fedora 15. thunderbird-8.0 I can access dovecot from thunderbird, imap maildir. But when I try to create a new folder, nothing happens. I'm not using LAYOUT=fs, so I have added, tb-extra-mailbox-sep, which didn't seem relevant anyhow. Any help appreciated. sean From micah at riseup.net Tue Nov 29 02:04:41 2011 From: micah at riseup.net (Micah Anderson) Date: Mon, 28 Nov 2011 19:04:41 -0500 Subject: [Dovecot] Indexes to MLC-SSD References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> Message-ID: <87sjl7lrom.fsf@algae.riseup.net> Dovecot-GDH writes: > If I/O performance is a concern, you may be interested in ZFS and Flashcache. > > Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) > ZFS does run on Linux http://zfs-fuse.net > > Flashcache: https://github.com/facebook/flashcache/ That site has no information about what flashcache is. From seandarcy2 at gmail.com Tue Nov 29 04:28:29 2011 From: seandarcy2 at gmail.com (sean darcy) Date: Mon, 28 Nov 2011 21:28:29 -0500 Subject: [Dovecot] can't get sieve to sort virus into spam Message-ID: I can't get sieve to put virus files in the SPAM folder. dovecot -n # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38.8-32.fc15.i686.PAE i686 Fedora release 15 (Lovelock) auth_debug_passwords = yes info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_access_groups = mail mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { antispam_backend = dspam-exec antispam_dspam_args = --deliver;--user;%u antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam = Spam antispam_trash = trash;Trash;Deleted Items; Deleted Messages sieve = /home/vmail/%d/%n/dovecot.sieve.script sieve_dir = /home/vmail/%d/%n/dovecot.sieve/ sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } service auth { unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } user = vmail } ssl_cert = References: Message-ID: <1266621424.20111129030956@gmail.com> On Tuesday, November 29, 2011 at 02:28:29 UTC, seandarcy2 at gmail.com confabulated: > I can't get sieve to put virus files in the SPAM folder. > dovecot -n > # 2.0.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.38.8-32.fc15.i686.PAE i686 Fedora release 15 (Lovelock) > auth_debug_passwords = yes > info_log_path = /var/log/dovecot-info.log > log_path = /var/log/dovecot.log > mail_access_groups = mail > mail_home = /home/vmail/%d/%n > mail_location = maildir:~/mail > mail_privileged_group = mail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date > mbox_write_locks = fcntl > passdb { > args = scheme=CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file > } > plugin { > antispam_backend = dspam-exec > antispam_dspam_args = --deliver;--user;%u > antispam_dspam_binary = /usr/bin/dspam > antispam_signature = X-DSPAM-Signature > antispam_signature_missing = error > antispam_spam = Spam > antispam_trash = trash;Trash;Deleted Items; Deleted Messages > sieve = /home/vmail/%d/%n/dovecot.sieve.script > sieve_dir = /home/vmail/%d/%n/dovecot.sieve/ > sieve_global_dir = /var/lib/dovecot/sieve/global/ > sieve_global_path = /var/lib/dovecot/sieve/default.sieve > } > service auth { > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service lmtp { > inet_listener lmtp { > address = 127.0.0.1 > port = 24 > } > user = vmail > } > ssl_cert = ssl_key = userdb { > driver = passwd > } > userdb { > args = username_format=%u /etc/dovecot/users > driver = passwd-file > } > protocol lmtp { > mail_plugins = quota sieve > } > protocol imap { > mail_plugins = " antispam" > } > cat /var/lib/dovecot/sieve/default.sieve > require "fileinto"; > if header :contains "X-DSPAM-Result" "Virus" { > fileinto "SPAM"; > } > if header :contains "X-DSPAM-Result" "Spam" { > fileinto "SPAM"; > } > and there is a folder SPAM: > cat mail/subscriptions > Sent > Trash > Drafts > SPAM > yet : > Content-Type: multipart/alternative; > boundary=14dae9399d3b9b67cc04b2d60790 > X-DSPAM-Result: Virus > X-DSPAM-Processed: Mon Nov 28 21:03:47 2011 > X-DSPAM-Confidence: 1.0000 > X-DSPAM-Probability: 1.0000 > X-DSPAM-Signature: 4ed43d8341479178914901 > --14dae9399d3b9b67cc04b2d60790 > Content-Type: text/plain; charset=ISO-8859-1 > X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H* > does not end up into SPAM, or anyplace else, including inbox. > How do I make this work? What gets logged? If sieve has an issue storing something, it is logged. If you don't have the autocreate plugin loaded, the directories have to be created some other way. Does SPAM exist as a directory? Have you compiled the global sieve file using sievec? Does the particular mail account have sieve rules? Global rules are not executed if there are account rules. If that is the case, check out the sieve_before and sieve_after parameters. -- There are 10 kinds of people in the world... Those who understand binary, and those who don't. From rainer.frey at inxmail.de Tue Nov 29 09:19:21 2011 From: rainer.frey at inxmail.de (Rainer Frey) Date: Tue, 29 Nov 2011 08:19:21 +0100 Subject: [Dovecot] Authentication listener configuration Message-ID: <22BC7966-5C73-4085-AB63-3B5E20FCE335@inxmail.de> Hi, I'm wondering about the authentication listeners that dovecot uses. In many configurations I see: service auth { unix_listener auth-userdb { Does any component of dovecot use this listener internally (lmtp, lda ...) and does that read the config and search for the listener, or is the exact name "auth-userdb" expected? Is this maybe dependent on the userdb? Which ones need it? And if I want to use postfix SMTP auth, should I define a second listener, or just re-configure that default one to postfix's needs? Also, in which cases do I need to define an auth-worker service, and a unix_listener for it? I don't find this in the documentation, if that's caused by my lacking search skills, please just send apointer. Thanks Rainer From robert at schetterer.org Tue Nov 29 09:49:52 2011 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 29 Nov 2011 08:49:52 +0100 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <4ED48EA0.4060706@schetterer.org> Am 28.11.2011 23:14, schrieb Marcelo Salhab Brogliato: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. > > About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). > I'm using dovecot 1.2.9. > > I guess you already had some of these problems or maybe in another architecture these neither exists. > > Thanks for any help, > > Marcelo > i have run , store drbd with ocfs behind keepalived loadbalancers this should run with vm too, but it might not optimal, depening i.e how much mailboxes etc search this list archive, there were posts about equal issues -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From stephan at rename-it.nl Tue Nov 29 10:00:39 2011 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 29 Nov 2011 09:00:39 +0100 Subject: [Dovecot] can't get sieve to sort virus into spam In-Reply-To: <1266621424.20111129030956@gmail.com> References: <1266621424.20111129030956@gmail.com> Message-ID: <4ED49127.6020000@rename-it.nl> On 11/29/2011 4:09 AM, Duane Hill wrote: > On Tuesday, November 29, 2011 at 02:28:29 UTC, seandarcy2 at gmail.com confabulated: > >> I can't get sieve to put virus files in the SPAM folder. >> does not end up into SPAM, or anyplace else, including inbox. >> How do I make this work? > What gets logged? If sieve has an issue storing something, it is > logged. The message must be stored somewhere, even if Sieve fails entirely. If your Dovecot logs provide no indication that a message was handled, are you even sure that LTMP is getting a message at all? In this regard your MTA's logs are of importance as well. Regards, Stephan. From pw at wk-serv.de Tue Nov 29 16:24:22 2011 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 29 Nov 2011 15:24:22 +0100 Subject: [Dovecot] Questions about Proxy/Director In-Reply-To: <20111125081715.GA24352@dibs.tanso.net> References: <1321472837.21919.598.camel@hurina> <4ECAC68D.3060007@wk-serv.de> <20111122071624.GA6110@dibs.tanso.net> <4ECB76A8.80208@wk-serv.de> <20111122104547.GA10680@dibs.tanso.net> <2094e82e55c67f5591540ca7c800b92c@wk-serv.de> <20111124104709.GB24297@dibs.tanso.net> <4ECE483B.5090208@wk-serv.de> <20111124143321.GB28600@dibs.tanso.net> <4ECECBF9.5070507@wk-serv.de> <20111125081715.GA24352@dibs.tanso.net> Message-ID: <4ED4EB16.1030704@wk-serv.de> Hi again, I have a working test environment not but I don't feel very comfortable with a non encrypted connection between the proxy/director and the backend servers. If I set ssl = required on my backend server I get this error: imap-login: Disconnected (tried to use disabled plaintext auth) Is there a chance to encrypt this connection? # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 auth_debug = yes auth_verbose = yes director_mail_servers = 172.17.0.31 director_servers = 172.17.213.213 disable_plaintext_auth = no lmtp_proxy = yes mail_debug = yes passdb { args = proxy=y nopassword=y driver = static } service auth { unix_listener auth-userdb { mode = 0600 user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } ssl = required ssl_cert = Hi, after upgrading from 2.0.15 to 2.0.16 authentication with vpopmail won?t work anymore. 2.0.x and 1.2.x works (this output is from 2.0.15): Nov 28 18:29:21 mail dovecot: auth: Debug: auth client connected (pid=3855) Nov 28 18:29:21 mail dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=10.114.57.11 rip=10.114.57.11 lport=143 rport=37693 resp=d3NjaG9lbGxlckBodGwtcGVyZy5hYy5hdAB3c2Nob2VsbGVyQGh0bC1wZXJnLmFjLmF0AGRpYWJsbw== Nov 28 18:29:21 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: client out: OK 1 user=wschoeller at htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: master in: REQUEST 4001103873 3855 1 7a21a4d39cc6b187b3b14dc4cfbf077a Nov 28 18:29:21 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:29:21 mail dovecot: auth: Debug: master out: USER 4001103873 wschoeller at htl-perg.ac.at uid=89 gid=89 home=/var/vpopmail/domains/htl-perg.ac.at/wschoeller Nov 28 18:29:21 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=10.114.57.11, lip=10.114.57.11, mpid=3856, secured Nov 28 18:29:21 mail dovecot: imap(wschoeller at htl-perg.ac.at): Disconnected: Logged out bytes=29/405 Nov 28 18:29:21 mail dovecot: auth: Debug: auth client connected (pid=3858) 2.0.16 doesn?t work: Nov 28 18:43:15 mail dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/auth Nov 28 18:43:15 mail dovecot: auth: Debug: auth client connected (pid=27392) Nov 28 18:43:15 mail dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=10.114.57.11 rip=10.114.57.11 lport=143 rport=42768 resp=d3NjaG9lbGxlckBodGwtcGVyZy5hYy5hdAB3c2Nob2VsbGVyQGh0bC1wZXJnLmFjLmF0AGRpYWJsbw== Nov 28 18:43:15 mail dovecot: auth-worker: Debug: Loading modules from directory: /usr/lib/dovecot/auth Nov 28 18:43:15 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): lookup user=wschoeller domain=htl-perg.ac.at Nov 28 18:43:15 mail dovecot: auth-worker: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): Password mismatch Nov 28 18:43:15 mail dovecot: auth-worker: Error: md5_verify(wschoeller at htl-perg.ac.at): Not a valid MD5-CRYPT or PLAIN-MD5 password Nov 28 18:43:15 mail dovecot: auth-worker: Error: smd5_verify(wschoeller at htl-perg.ac.at): SMD5 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha_verify(wschoeller at htl-perg.ac.at): SSHA password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha256_verify(wschoeller at htl-perg.ac.at): SSHA256 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Error: ssha512_verify(wschoeller at htl-perg.ac.at): SSHA512 password too short Nov 28 18:43:15 mail dovecot: auth-worker: Warning: Invalid OTP data in passdb Nov 28 18:43:15 mail dovecot: auth-worker: Debug: vpopmail(wschoeller at htl-perg.ac.at,10.114.57.11): CLEARTEXT(mypassword) != '', try OTP scheme instead Nov 28 18:43:17 mail dovecot: auth: Debug: client out: FAIL 1 user=wschoeller at htl-perg.ac.at Nov 28 18:43:17 mail dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=10.114.57.11, lip=10.114.57.11, secured Dovecot -n: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.36-gentoo-r8 i686 Gentoo Base System release 1.12.14 auth_debug = yes auth_debug_passwords = yes disable_plaintext_auth = no first_valid_uid = 89 listen = * mail_location = maildir:~/.maildir namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . type = private } passdb { args = webmail=127.0.0.1 driver = vpopmail } service auth { user = root } ssl_cert = References: <4EA8780E.5030808@wk-serv.de> <4EA8C396.3040105@hardwarefreak.com> <4EA916D1.30409@wildgooses.com> <87sjl7lrom.fsf@algae.riseup.net> Message-ID: <0DD532C1-7E22-49BA-A457-250194CF4828@rocketfish.com> https://github.com/facebook/flashcache/blob/master/doc/flashcache-doc.txt On Nov 28, 2011, at 4:04 PM, Micah Anderson wrote: > Dovecot-GDH writes: > >> If I/O performance is a concern, you may be interested in ZFS and Flashcache. >> >> Specifically, ZFS' ZIL (ZFS Intent Log) and its L2ARC (Layer 2 Adaptive Read Cache) >> ZFS does run on Linux http://zfs-fuse.net >> >> Flashcache: https://github.com/facebook/flashcache/ > > That site has no information about what flashcache is. > > From dbrooks at mdah.state.ms.us Tue Nov 29 22:46:35 2011 From: dbrooks at mdah.state.ms.us (Donny Brooks) Date: Tue, 29 Nov 2011 14:46:35 -0600 Subject: [Dovecot] Best setup for performance and fault tolerance Message-ID: <6f4f-4ed54480-5-fba0340@98207411> Hello all. I am in need of some guidance. First a little background. Currently our mail server is on physical hardware (Dell server with 2x 2.8GHz Xeon w/ 4GB ram, raid5 array, single gigabit nic) running on Fedora 11 and postfix-2.5.6-3.fc11.x86_64 with dovecot-1.2.11-3.fc11.x86_64. Mailstore is via Maildir format that was converted from mbox about a year ago. This same machine is also our PDC with samba, Master LDAP, slave MySQL replication, primary DNS, and home server for about 20 users. We have approximately 200 end users that have mailboxes on the server ranging from 1KB to 20GB in size. Total mail store is currently at 300GB. About 75 of the users are currently POP access and their mail will be moved to the server soon and setup as IMAP. This is calculated to add roughly another 150GB of mail for a total of 450GB mail store. Being a state agency we have to keep the mail indefinitely for public record reasons. We use a mixture of Thunderbird as an IMAP client and SOGo for web access. Now to the problem: Recently we have been having super slow access to the mail server. Turns out the load was insanely high partially due to the samba home server portion, which is being moved off as we speak, and the other part is due to people searching their mail. Just yesterday one of our users nearly brought the entire agency to its knees by performing a search on her 8GB of mail via IMAP. Since the server is old in both hardware and software I have been tasked with moving it to newer hardware and a newer OS. We currently have 3 virtual servers running Xen and a SAN. The new setup will be placed in the virtual environment. I will probably run Fedora 16 as the OS but am open to Centos, Fedora, or Ubuntu. Now to the question: What is the best way to setup Dovecot so that it is tuned for performance and high available? We have been running with this single point of failure for years so as long as we are moving the mail server we might as well build in some redundancy. To solve the searching problem I thought of maybe setting up some type of indexing. I do kind of want to break the various services out on to separate virtual machines for a little more fault tolerance, but that is not totally necessary. What do you think of things like iRedmail? I see it's usefulness but the not being able to separate services kind of defeats the purpose, plus I want to setup a high available MySQL cluster and possibly OpenLDAP or 389 cluster so iRedMail may not be the best solution. Sorry for the long email but I am trying to get all the information out there at once so it will help get more directed responses in the shortest amount of time. I look forward to any and all input on this matter -- Donny B MDAH From Ralf.Hildebrandt at charite.de Tue Nov 29 23:42:56 2011 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 29 Nov 2011 22:42:56 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders Message-ID: <20111129214256.GL14970@charite.de> While our webmail installation creates "Sent" as a Sent-Items Folder, Apple Mail creates "Sent Messages". Is there a way of "mapping" foldernames e.g. map "Sent Messages" to "Sent" map "Deletes Messages" to "Trash" and so on? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From micah at riseup.net Tue Nov 29 23:50:59 2011 From: micah at riseup.net (micah anderson) Date: Tue, 29 Nov 2011 16:50:59 -0500 Subject: [Dovecot] dovecot-lda quota rule In-Reply-To: <87ehx8gic7.fsf@algae.riseup.net> References: <874nybztuz.fsf@algae.riseup.net> <1321388840.21919.520.camel@hurina> <87ehx8gic7.fsf@algae.riseup.net> Message-ID: <87zkfek37g.fsf@algae.riseup.net> On Wed, 16 Nov 2011 11:09:12 -0500, micah anderson wrote: > On Tue, 15 Nov 2011 22:27:20 +0200, Timo Sirainen wrote: > > On Thu, 2011-11-10 at 13:59 -0500, Micah Anderson wrote: > > > I really like the feature where you can define quota rules with percents > > > which trigger off of the default values[0] (so you can set the Trash to > > > allow for 10% more of the user's quota for example). > > > > > > What I would really love in dovecot would be for the ability to > > > configure a quota rule for dovecot-lda. I would like to configure things > > > so we don't bounce emails for users until they are well over quota, the > > > IMAP quota plugin is a really great way to notify people that they are > > > over quota because it fails to write to other folders.... that should be > > > enough to get people's attention that they need to deal with things, but > > > bouncing is harsh. > > > > You can give LDA-specific quota settings. Like maybe: > > > > protocol lda { > > plugin { > > quota_rule2 = INBOX:storage=200% > > } > > } I just tried doing this, and I started bouncing mail for everyone because they were over quota (even when they arent), so I am clearly missing something in how I have it configured. This is what I have: namespace { separator = . prefix = inbox = yes } namespace { separator = . prefix = INBOX. inbox = no hidden = yes list = no } plugin { quota = dict:Your mail quota::proxy::quota } dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } plugin { quota_exceeded_message = You are over quota. To avoid losing mail, immediately empty your Trash and Sent folders and delete emails with large attachments. quota_rule = *:bytes=100663296 # in case someone doesn't have sql quota quota_rule2 = Trash:bytes=+20%% # let trash be larger by 20% quota_rule3 = Spam:bytes=+10%% # let spam be larger by 10% quota_rule4 = INBOX.Trash:bytes=+20%% # repeat the above for namespaces quota_rule5 = INBOX.Spam:bytes=+10%% # same quota_rule6 = INBOX.restored:bytes=+500%% # allow for temporary folder } ... protocol lda { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins plugin { quota_rule7 = INBOX:bytes=+10%% } } Maybe I should have that quota_rule7 be *:bytes=+10%% ? thanks for any suggestions for what I am missing here! micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From p at state-of-mind.de Tue Nov 29 23:51:49 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 29 Nov 2011 22:51:49 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <20111129215149.GA29651@state-of-mind.de> * Ralf Hildebrandt : > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? There's an RFC, but it hasn't been adopted yet. Neither by IMAP servers nor by IMAP clients. ;) p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From tss at iki.fi Wed Nov 30 00:17:05 2011 From: tss at iki.fi (Timo Sirainen) Date: Wed, 30 Nov 2011 00:17:05 +0200 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> On 29.11.2011, at 23.42, Ralf Hildebrandt wrote: > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? With v2.1 it would be somewhat easy to create a plugin that supports aliases, but each client would still see each others' Sent boxes and possibly synchronize everything twice.. From slusarz at curecanti.org Wed Nov 30 00:17:23 2011 From: slusarz at curecanti.org (Michael M Slusarz) Date: Tue, 29 Nov 2011 15:17:23 -0700 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129215149.GA29651@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> Message-ID: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Quoting Patrick Ben Koetter

: > * Ralf Hildebrandt : >> While our webmail installation creates "Sent" as a Sent-Items Folder, >> Apple Mail creates "Sent Messages". >> >> Is there a way of "mapping" foldernames e.g. >> >> map "Sent Messages" to "Sent" >> map "Deletes Messages" to "Trash" >> and so on? > > There's an RFC, but it hasn't been adopted yet. Neither by IMAP > servers nor by > IMAP clients. ;) Not true. We've (IMP) had special-use (RFC 6154) support since August 2010. Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. michael From p at state-of-mind.de Wed Nov 30 00:26:51 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 29 Nov 2011 23:26:51 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Message-ID: <20111129222651.GG10344@state-of-mind.de> * Michael M Slusarz : > Quoting Patrick Ben Koetter

: > > >* Ralf Hildebrandt : > >>While our webmail installation creates "Sent" as a Sent-Items Folder, > >>Apple Mail creates "Sent Messages". > >> > >>Is there a way of "mapping" foldernames e.g. > >> > >>map "Sent Messages" to "Sent" > >>map "Deletes Messages" to "Trash" > >>and so on? > > > >There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >servers nor by > >IMAP clients. ;) > > Not true. We've (IMP) had special-use (RFC 6154) support since August 2010. > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. Very nice. Thanks for the update. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From stan at hardwarefreak.com Wed Nov 30 07:59:33 2011 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 29 Nov 2011 23:59:33 -0600 Subject: [Dovecot] Best setup for performance and fault tolerance In-Reply-To: <6f4f-4ed54480-5-fba0340@98207411> References: <6f4f-4ed54480-5-fba0340@98207411> Message-ID: <4ED5C645.1000605@hardwarefreak.com> On 11/29/2011 2:46 PM, Donny Brooks wrote: > Hello all. I am in need of some guidance. First a little background. Currently our mail server is on physical hardware (Dell server with 2x 2.8GHz Xeon w/ 4GB ram, raid5 array, single gigabit nic) running on Fedora 11 and postfix-2.5.6-3.fc11.x86_64 with dovecot-1.2.11-3.fc11.x86_64. Mailstore is via Maildir format that was converted from mbox about a year ago. This same machine is also our PDC with samba, Master LDAP, slave MySQL replication, primary DNS, and home server for about 20 users. > > We have approximately 200 end users that have mailboxes on the server ranging from 1KB to 20GB in size. Total mail store is currently at 300GB. About 75 of the users are currently POP access and their mail will be moved to the server soon and setup as IMAP. This is calculated to add roughly another 150GB of mail for a total of 450GB mail store. Being a state agency we have to keep the mail indefinitely for public record reasons. We use a mixture of Thunderbird as an IMAP client and SOGo for web access. > > Now to the problem: Recently we have been having super slow access to the mail server. Turns out the load was insanely high partially due to the samba home server portion, which is being moved off as we speak, and the other part is due to people searching their mail. Just yesterday one of our users nearly brought the entire agency to its knees by performing a search on her 8GB of mail via IMAP. > > Since the server is old in both hardware and software I have been tasked with moving it to newer hardware and a newer OS. We currently have 3 virtual servers running Xen and a SAN. The new setup will be placed in the virtual environment. I will probably run Fedora 16 as the OS but am open to Centos, Fedora, or Ubuntu. > > Now to the question: What is the best way to setup Dovecot so that it is tuned for performance and high available? We have been running with this single point of failure for years so as long as we are moving the mail server we might as well build in some redundancy. To solve the searching problem I thought of maybe setting up some type of indexing. I do kind of want to break the various services out on to separate virtual machines for a little more fault tolerance, but that is not totally necessary. > > What do you think of things like iRedmail? I see it's usefulness but the not being able to separate services kind of defeats the purpose, plus I want to setup a high available MySQL cluster and possibly OpenLDAP or 389 cluster so iRedMail may not be the best solution. > > Sorry for the long email but I am trying to get all the information out there at once so it will help get more directed responses in the shortest amount of time. I look forward to any and all input on this matter > -- > Donny B > MDAH Build an Enkive server: http://www.enkive.org/ and configure your SMTP MTAs to transparently copy all email to it (recipient_bcc for example). This fulfills your retention requirements. Since all emails are now archived by Enkive as they arrive, cron a nightly script on the Dovecot server to delete any emails over a week/month/etc old (depending on your short term access needs) from your active Dovecot mailboxes. This drastically reduces your Dovecot storage requirements. Searches will be performed by the Enkive server, removing that load from your Dovecot host. Search interface demo here: http://www.enkive.org/demo In addition your total mail storage (active+archive) requirement will be a fraction of what it is now because Enkive performs deduplication of email content and attachments so you save even more disk space. Enkive can run fine as a VM if you give it the required resources. Carve an appropriately sized LUN off the SAN array for the Enkive server storage. Format it with XFS for best performance. Enkive should be a good fit for your needs. Bear in mind installing/configuring it is not for the faint of heart: http://wiki.enkive.org/index.php/Installation_Instructions http://wiki.enkive.org/index.php/GettingMailIntoEnkive But thankfully administration is relatively easy: http://wiki.enkive.org/index.php/Administrator_Manual -- Stan From robert at schetterer.org Wed Nov 30 08:48:36 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:48:36 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129214256.GL14970@charite.de> References: <20111129214256.GL14970@charite.de> Message-ID: <4ED5D1C4.8010607@schetterer.org> Am 29.11.2011 22:42, schrieb Ralf Hildebrandt: > While our webmail installation creates "Sent" as a Sent-Items Folder, > Apple Mail creates "Sent Messages". > > Is there a way of "mapping" foldernames e.g. > > map "Sent Messages" to "Sent" > map "Deletes Messages" to "Trash" > and so on? > wish i would know a solution i know newest outlook can configured equal to tb folders default schema , except spam folder -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 08:50:39 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:50:39 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> References: <20111129214256.GL14970@charite.de> <18C5BE49-8647-4F52-9F45-0BC59542D40B@iki.fi> Message-ID: <4ED5D23F.2030609@schetterer.org> Am 29.11.2011 23:17, schrieb Timo Sirainen: > On 29.11.2011, at 23.42, Ralf Hildebrandt wrote: > >> While our webmail installation creates "Sent" as a Sent-Items Folder, >> Apple Mail creates "Sent Messages". >> >> Is there a way of "mapping" foldernames e.g. >> >> map "Sent Messages" to "Sent" >> map "Deletes Messages" to "Trash" >> and so on? > > With v2.1 it would be somewhat easy to create a plugin that supports aliases, but each client would still see each others' Sent boxes and possibly synchronize everything twice.. that would be no good solution....in...my eyes > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 08:53:08 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 07:53:08 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> Message-ID: <4ED5D2D4.2090603@schetterer.org> Am 29.11.2011 23:17, schrieb Michael M Slusarz: > Quoting Patrick Ben Koetter

: > >> * Ralf Hildebrandt : >>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>> Apple Mail creates "Sent Messages". >>> >>> Is there a way of "mapping" foldernames e.g. >>> >>> map "Sent Messages" to "Sent" >>> map "Deletes Messages" to "Trash" >>> and so on? >> >> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >> servers nor by >> IMAP clients. ;) > > Not true. We've (IMP) had special-use (RFC 6154) support since August > 2010. > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > > michael > what helps this, tb, outlook, apple mail, winmail and some mobile clients must follow , this is what people use, and it will take years after some of them might upgrade -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From akam at zakamye.ru Wed Nov 30 08:57:57 2011 From: akam at zakamye.ru (=?UTF-8?B?0JDQudC00LDRgCDQmtCw0LzQsNC70L7Qsg==?=) Date: Wed, 30 Nov 2011 10:57:57 +0400 Subject: [Dovecot] second active directory Message-ID: <4ED5D3F5.1020102@zakamye.ru> Hello! There is dovecot authenticating through AD. All is working well, if authentication is just one of AD. But there is another AD, and working with 2 AD's dovecot don't recognizes last AD user's home directory. For example, if user name in the second AD - 1developer and Mail - it-2c at zakamye.ru then dovecot creates folder /var/mail/zakamye.ru/1cdeveloper but it should be /var/mail/zakamye.ru/it-2c. What I do wrong?( And, if I change the order of viewing AD, does not work precisely the second AD: passdb ldap-DUSLYK { driver = ldap args = /etc/dovecot/dovecot-DUSLYK.conf } userdb ldap-DUSLYK { driver = ldap args = /etc/dovecot/dovecot-DUSLYK-users.conf } passdb ldap-DUSL-OFFICE { driver = ldap args = /etc/dovecot/dovecot-DUSL-OFFICE.conf } userdb ldap-DUSL-BASE { driver = ldap args = /etc/dovecot/dovecot-DUSL-OFFICE-users.conf } Sorry for my bad english, but maybe you can understand logs and configs: http://pastebin.com/mtk4XsNE -- ????? ?. ???????, ????????? ????????????? ??? "?? ??????" ?. ???. ?????, ???. ????????? ???.: +7 (8552) 47-01-99 ???. 3025 ???.: +7 (8552) 70-65-15 ???. 280 ???.: +7 (919) 621-28-52 E-mail: akam at zakamye.ru From p at state-of-mind.de Wed Nov 30 10:21:42 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 30 Nov 2011 09:21:42 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5D2D4.2090603@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> Message-ID: <20111130082141.GB1594@state-of-mind.de> * Robert Schetterer : > Am 29.11.2011 23:17, schrieb Michael M Slusarz: > > Quoting Patrick Ben Koetter

: > > > >> * Ralf Hildebrandt : > >>> While our webmail installation creates "Sent" as a Sent-Items Folder, > >>> Apple Mail creates "Sent Messages". > >>> > >>> Is there a way of "mapping" foldernames e.g. > >>> > >>> map "Sent Messages" to "Sent" > >>> map "Deletes Messages" to "Trash" > >>> and so on? > >> > >> There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >> servers nor by > >> IMAP clients. ;) > > > > Not true. We've (IMP) had special-use (RFC 6154) support since August > > 2010. > > > > Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > > > > michael > > > > what helps this, tb, outlook, apple mail, winmail and some mobile > clients must follow , this is what people use, and it will take years > after some of them might upgrade What helps this == Was hilft es? TB has it on the CR list. Apple has their own X-LIST feature together with google. p at rick > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Wed Nov 30 10:25:46 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 09:25:46 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111130082141.GB1594@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> Message-ID: <4ED5E88A.20205@schetterer.org> Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: > * Robert Schetterer : >> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>> Quoting Patrick Ben Koetter

: >>> >>>> * Ralf Hildebrandt : >>>>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>>>> Apple Mail creates "Sent Messages". >>>>> >>>>> Is there a way of "mapping" foldernames e.g. >>>>> >>>>> map "Sent Messages" to "Sent" >>>>> map "Deletes Messages" to "Trash" >>>>> and so on? >>>> >>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>> servers nor by >>>> IMAP clients. ;) >>> >>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>> 2010. >>> >>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>> >>> michael >>> >> >> what helps this, tb, outlook, apple mail, winmail and some mobile >> clients must follow , this is what people use, and it will take years >> after some of them might upgrade > > What helps this == Was hilft es? > > TB has it on the CR list. > Apple has their own X-LIST feature together with google. cool, *g left another hundred clients however having this widly solved would be a dream why i did take so long for it.........., its a Problem since years > > p at rick > > > > > >> >> -- >> Best Regards >> >> MfG Robert Schetterer >> >> Germany/Munich/Bavaria > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Nov 30 10:33:24 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 09:33:24 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5E88A.20205@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> Message-ID: <4ED5EA54.7000007@schetterer.org> Am 30.11.2011 09:25, schrieb Robert Schetterer: > Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: >> * Robert Schetterer : >>> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>>> Quoting Patrick Ben Koetter

: >>>> >>>>> * Ralf Hildebrandt : >>>>>> While our webmail installation creates "Sent" as a Sent-Items Folder, >>>>>> Apple Mail creates "Sent Messages". >>>>>> >>>>>> Is there a way of "mapping" foldernames e.g. >>>>>> >>>>>> map "Sent Messages" to "Sent" >>>>>> map "Deletes Messages" to "Trash" >>>>>> and so on? >>>>> >>>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>>> servers nor by >>>>> IMAP clients. ;) >>>> >>>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>>> 2010. >>>> >>>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>>> >>>> michael >>>> >>> >>> what helps this, tb, outlook, apple mail, winmail and some mobile >>> clients must follow , this is what people use, and it will take years >>> after some of them might upgrade >> >> What helps this == Was hilft es? >> >> TB has it on the CR list. >> Apple has their own X-LIST feature together with google. > > cool, *g left another hundred clients > > however having this widly solved would be a dream > why i did take so long for it.........., its a Problem since years hoping big spreaded clients will solve it soon perhaps a chance to block non fixed clients would be nice any chance ?, i see clients anounce their name and versions in debug log > >> >> p at rick >> >> >> >> >> >>> >>> -- >>> Best Regards >>> >>> MfG Robert Schetterer >>> >>> Germany/Munich/Bavaria >> > > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From p at state-of-mind.de Wed Nov 30 10:56:27 2011 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Wed, 30 Nov 2011 09:56:27 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <4ED5E88A.20205@schetterer.org> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> Message-ID: <20111130085626.GD1594@state-of-mind.de> * Robert Schetterer : > Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: > > * Robert Schetterer : > >> Am 29.11.2011 23:17, schrieb Michael M Slusarz: > >>> Quoting Patrick Ben Koetter

: > >>>> > >>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP > >>>> servers nor by IMAP clients. ;) > >>> > >>> Not true. We've (IMP) had special-use (RFC 6154) support since August > >>> 2010. > >>> > >>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. > >>> > >>> michael > >>> > >> > >> what helps this, tb, outlook, apple mail, winmail and some mobile > >> clients must follow , this is what people use, and it will take years > >> after some of them might upgrade > > > > What helps this == Was hilft es? > > > > TB has it on the CR list. > > Apple has their own X-LIST feature together with google. > > cool, *g left another hundred clients > > however having this widly solved would be a dream > why i did take so long for it.........., its a Problem since years Yes, it is a problem, but having a standard doesn't solve it and open source software is not a guarant to have it implemented either, unless you contribute the code yourself or get to sponsor someone to do it. BTW: We'd contribute ? 500 if Timo implemented RFC 6154 in Dovecot 2.1. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From robert at schetterer.org Wed Nov 30 11:05:26 2011 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 30 Nov 2011 10:05:26 +0100 Subject: [Dovecot] MUAs creating different "Sent" folders In-Reply-To: <20111130085626.GD1594@state-of-mind.de> References: <20111129214256.GL14970@charite.de> <20111129215149.GA29651@state-of-mind.de> <20111129151723.Horde.uMreJ4F5lbhO1VnzP2tmQ4A@bigworm.curecanti.org> <4ED5D2D4.2090603@schetterer.org> <20111130082141.GB1594@state-of-mind.de> <4ED5E88A.20205@schetterer.org> <20111130085626.GD1594@state-of-mind.de> Message-ID: <4ED5F1D6.7000501@schetterer.org> Am 30.11.2011 09:56, schrieb Patrick Ben Koetter: > * Robert Schetterer : >> Am 30.11.2011 09:21, schrieb Patrick Ben Koetter: >>> * Robert Schetterer : >>>> Am 29.11.2011 23:17, schrieb Michael M Slusarz: >>>>> Quoting Patrick Ben Koetter

: >>>>>> >>>>>> There's an RFC, but it hasn't been adopted yet. Neither by IMAP >>>>>> servers nor by IMAP clients. ;) >>>>> >>>>> Not true. We've (IMP) had special-use (RFC 6154) support since August >>>>> 2010. >>>>> >>>>> Offhand, I know that serverwise Cyrus 2.5 supports RFC 6154. >>>>> >>>>> michael >>>>> >>>> >>>> what helps this, tb, outlook, apple mail, winmail and some mobile >>>> clients must follow , this is what people use, and it will take years >>>> after some of them might upgrade >>> >>> What helps this == Was hilft es? >>> >>> TB has it on the CR list. >>> Apple has their own X-LIST feature together with google. >> >> cool, *g left another hundred clients >> >> however having this widly solved would be a dream >> why i did take so long for it.........., its a Problem since years > > Yes, it is a problem, but having a standard doesn't solve it and open source > software is not a guarant to have it implemented either, unless you contribute > the code yourself or get to sponsor someone to do it. > > BTW: We'd contribute ? 500 if Timo implemented RFC 6154 in Dovecot 2.1. very cool !!!!, as an add on idea, if it would be possible to identificate unfixed clients on the fly and doing some action on it, it might be a solution to allow imap at fixed clients only , others have to use pop3 then, not a solution everywhere but it could be implemented in some places ( i.e mine *g ) > > p at rick > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From info at simonecaruso.com Wed Nov 30 12:33:29 2011 From: info at simonecaruso.com (Simone Caruso) Date: Wed, 30 Nov 2011 11:33:29 +0100 Subject: [Dovecot] Fault tolerant architecture In-Reply-To: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> References: <803FB563-4CF5-4A47-B9EE-7A424D23D7D5@vialink.com.br> Message-ID: <4ED60679.1010903@simonecaruso.com> On 28/11/2011 23:14, Marcelo Salhab Brogliato wrote: > Hi, > I'm new to this list and want your help. > I'm the mail admin for some domains in Rio de Janeiro - Brazil. Today we have only one machine running dovecot (imap+pop3) with local mail. > We are migrating to two virtual machines in kvm running in separate hosts. Then we have two main problems: > - How to share mail files to both dovecots? We've been thinking about NFS using local indexes. Is this a good approach? > - How do we have a fault tolerant mail servers? Our first solutions is using two IP addresses in our DNS records. > For imap i need only 1 VIP both for loadbalancing or HA, for SMTP u can use dns failover with multiple mx records. > About NFS using local indexes, I'm configuring a test server. But how to configure local indexes when my mail_location comes from sql (userdb_home actually). > I'm using dovecot 1.2.9. I suggest to not use local indexing (for high i/o after failover) , anyway NFS for HA setup works well! If u need a cluster (master/master) consider having an HA loadbancer (dovecot director): -- Simone Caruso IT Consultant +39 349 65 90 805 From akam at zakamye.ru Wed Nov 30 12:39:31 2011 From: akam at zakamye.ru (=?UTF-8?B?0JDQudC00LDRgCDQmtCw0LzQsNC70L7Qsg==?=) Date: Wed, 30 Nov 2011 14:39:31 +0400 Subject: [Dovecot] second active directory In-Reply-To: <4ED5D3F5.1020102@zakamye.ru> References: <4ED5D3F5.1020102@zakamye.ru> Message-ID: <4ED607E3.40805@zakamye.ru> SOLVED. Problem solved. There was double users in both AD domains. 30.11.2011 10:57, ????? ??????? ?????: > Hello! > There is dovecot authenticating through AD. All is working well, if > authentication is just one of AD. > But there is another AD, and working with 2 AD's dovecot don't > recognizes last AD user's home directory. > For example, if user name in the second AD - 1developer and Mail - > it-2c at zakamye.ru then dovecot creates folder > /var/mail/zakamye.ru/1cdeveloper but it should be > /var/mail/zakamye.ru/it-2c. > > What I do wrong?( > > And, if I change the order of viewing AD, does not work precisely the > second AD: > passdb ldap-DUSLYK { > driver = ldap > args = /etc/dovecot/dovecot-DUSLYK.conf > } > userdb ldap-DUSLYK { > driver = ldap > args = /etc/dovecot/dovecot-DUSLYK-users.conf > } > passdb ldap-DUSL-OFFICE { > driver = ldap > args = /etc/dovecot/dovecot-DUSL-OFFICE.conf > } > userdb ldap-DUSL-BASE { > driver = ldap > args = /etc/dovecot/dovecot-DUSL-OFFICE-users.conf > } > > Sorry for my bad english, but maybe you can understand logs and configs: > http://pastebin.com/mtk4XsNE > From heptalium at gmx.de Wed Nov 30 20:04:16 2011 From: heptalium at gmx.de (Jens =?UTF-8?B?TWVpw59uZXI=?=) Date: Wed, 30 Nov 2011 19:04:16 +0100 Subject: [Dovecot] lda/sieve ignores quota from ldap Message-ID: <20111130190416.641ede1f@meissner.pentament.dyndns.org> Hello, I'm using Dovecot 2.0 with Sieve as MDA and Postfix as MTA. The Postfix delivery command is /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT". I have set a default quota with quota_rule = *:storage=1G. Some users should have a larger quota. This is defined in a ldap directory and I have set user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid,mailQuota=quota_rule=*:storage=%$. The imap_quota plugin shows the right quota for all users but the lda ignores the quota rule from ldap. If a user is over default quota but not over his ldap defined quota I get the following error message: From: Mail Delivery Subsystem Your message to <[?]> was automatically rejected: Quota exceeded (mailbox for user is full) The log file shows the following: Nov 30 18:21:17 [?] dovecot: lda([?]): Error: sieve: msgid=<20111130172116.7F065A07D@[?]>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) Nov 30 18:21:17 [?] dovecot: lda([?]): Error: sieve: script /var/mail/[?]/dovecot.sieve failed with unsuccessful implicit keep (user logfile /var/mail/[?]/dovecot.sieve.log may reveal additional details) Nov 30 18:21:17 [?] dovecot: lda([?]): msgid=<20111130172116.7F065A07D@[?]>: rejected: Quota exceeded (mailbox for user is full) dovecot.sieve.log: error: msgid=<20111130172116.7F065A07D@[?]>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full). What is wrong? What must I do to getting this to work? Thanks in advance. Regards Jens doveconf -n: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.3 nfs mail_location = maildir:/var/mail/%u mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = maildir quota_rule = *:storage=1G sieve = /var/mail/%u/dovecot.sieve sieve_dir = /var/mail/%u/sieve } protocols = " imap sieve pop3" ssl_cert = From terry at cnysupport.com Wed Nov 30 22:36:46 2011 From: terry at cnysupport.com (Terry Carmen) Date: Wed, 30 Nov 2011 15:36:46 -0500 Subject: [Dovecot] MS Exchange IMAP Proxy Message-ID: <20111130153646.Horde.fehRaFeGiNBO1pPe5xWA0vA@www.cnysupport.com> I need to make the IMAP interface of an Exchange 2000 server available on the net, however I would like to give it a little protection, and believe Dovecot's IMAP proxy might be appropriate. Does anybody have a *really simple* config that would allow IMAP pass-through to a single Exchange server? The examples I've seen all assume some sort of load balancing, which isn't an issue here, so I'd like to eliminate the need for an unnecessary database server. Does Dovecot do any cleanup of the IMAP commands that would be useful in protecting Exchange (from this, http://www.cvedetails.com/cve/CVE-2007-0221/ for example), or am I barking up the wrong tree? If Dovecot isn't helpful for this, can anybody point me to a better resource? Thanks! Terry