February 2009 - dovecot

[Dovecot] "dovecot-uidlist: Duplicate file entry at line" error
by Dovecot List 24 Feb '09

24 Feb '09

This is with respect to an error that I am facing in dovecot. The error is that is seen in logs is "Feb 23 00:04:46 mailblade1 dovecot: IMAP(USERNAME): /indexes/USERNAME/.INBOX/dovecot-uidlist: Duplicate file entry at line 7: 1234776125.M559298P3988.s2o.qlc.co.in,S=13111,W=13470:2, (uid 94277 -> 97805)" This error is seen for multiple users. Once this error occurs for a user, the user downloads mails all over again in POP3. Please help! And let us know what can be done. Details of our setup ---------------------- The setup has deployed 2 servers with Dovecot using the same storage over NFS with the following options: * Dovecot Version: v1.1.8 on both servers * Server1 for retrieving mails using POP and imap Access * Server2 for Mail Delivery via postfix and dovecot as LDA * Number of users: 20000 * Number of mails Handled: 500K per day --Note: Based on recommendation, kernel was upgraded from 2.6.18 (- see below for details) and yet the issue persists. Specifications of the server ------------------------------- 1) Server spec used for mail Access OS: Linux 2.6.28 i686 CentOS release 5.2 (Final) kernel : 2.6.28 /proc/sys/fs/inotify/max_user_instances = 1024 /proc/sys/fs/epoll/max_user_instances = 1024 Memory : 2GB 2) Server used for Dovecot Delivery OS: Linux 2.6.21-1.3194.fc7 i686 Fedora release 7 (Moonshine) kernel : 2.6.21 /proc/sys/fs/inotify/max_user_instances = 128 /proc/sys/fs/epoll/max_user_instances = NA Memory : 2 GB Conf on both the system as follows = -------------------------------------- # 1.1.8: /usr/local/etc/dovecot.conf #NFS Settings mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes #Other special settings pop3_no_flag_updates(pop3): yes pop3_lock_session(pop3): yes # all the settings base_dir: /usr/local/var/run/dovecot log_path: info_log_path: log_timestamp: %b %d %H:%M:%S syslog_facility: mail protocols: imap pop3 listen(default): *:143 listen(imap): *:143 listen(pop3): *:110 ssl_listen: ssl_disable: yes ssl_ca_file: ssl_cert_file: /etc/ssl/certs/dovecot.pem ssl_key_file: /etc/ssl/private/dovecot.pem ssl_key_password: ssl_parameters_regenerate: 168 ssl_cipher_list: ssl_cert_username_field: commonName ssl_verify_client_cert: no disable_plaintext_auth: no verbose_ssl: no shutdown_clients: yes nfs_check: yes version_ignore: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_user: dovecot login_greeting: Welcome to MailServe Popserver. login_log_format_elements: user=<%u> method=%m rip=%r lip=%l %c login_log_format: %$: %s login_process_per_connection: no login_chroot: yes login_greeting_capability: no login_process_size: 64 login_processes_count: 3 login_max_processes_count: 128 login_max_connections: 256 valid_chroot_dirs: mail_chroot: max_mail_processes: 600 mail_max_userip_connections(default): 50 mail_max_userip_connections(imap): 50 mail_max_userip_connections(pop3): 8 verbose_proctitle: yes first_valid_uid: 99 last_valid_uid: 0 first_valid_gid: 99 last_valid_gid: 0 mail_extra_groups: mail_access_groups: mail_privileged_group: mail_uid: mail_gid: mail_location: maildir:~/Maildir:INDEX=/indexes/%u:CONTROL=/indexes/%u mail_cache_fields: mail_never_cache_fields: imap.envelope mail_cache_min_mail_count: 0 mailbox_idle_check_interval: 30 mail_debug: no mail_full_filesystem_access: no mail_max_keyword_length: 50 mail_save_crlf: no mmap_disable: yes dotlock_use_excl: yes fsync_disable: no mail_nfs_storage: yes mail_nfs_index: yes mailbox_list_index_disable: yes lock_method: fcntl maildir_stat_dirs: no maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: no mbox_read_locks: fcntl mbox_write_locks: dotlock fcntl mbox_lock_timeout: 300 mbox_dotlock_change_timeout: 120 mbox_min_index_size: 0 mbox_dirty_syncs: yes mbox_very_dirty_syncs: no mbox_lazy_writes: yes dbox_rotate_size: 2048 dbox_rotate_min_size: 16 dbox_rotate_days: 1 umask: 63 mail_drop_priv_before_exec: no mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_process_size: 256 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_log_prefix: %Us(%u): mail_log_max_lines_per_sec: 10 imap_max_line_length: 65536 imap_capability: imap_client_workarounds: imap_logout_format: bytes=%i/%o pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes pop3_enable_last: no pop3_reuse_xuidl: no pop3_lock_session(default): no pop3_lock_session(imap): no pop3_lock_session(pop3): yes pop3_uidl_format: %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls pop3_logout_format: top=%t/%p, retr=%r/%b, del=%d/%m, size=%s dict_db_config: auth default: mechanisms: plain realms: default_realm: cache_size: 0 cache_ttl: 3600 cache_negative_ttl: 3600 executable: /usr/local/libexec/dovecot/dovecot-auth user: root chroot: username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ username_translation: username_format: master_user_separator: anonymous_username: anonymous krb5_keytab: gssapi_hostname: winbind_helper_path: /usr/bin/ntlm_auth failure_delay: 2 verbose: no debug: no debug_passwords: no ssl_require_client_cert: no ssl_username_from_cert: no ntlm_use_winbind: no count: 1 worker_max_count: 5 worker_max_request_count: 0 process_size: 256 passdb: driver: sql args: /usr/local/etc/dovecot-mysql.conf deny: no pass: no master: no userdb: driver: sql args: /usr/local/etc/dovecot-mysql.conf userdb: driver: prefetch args: socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 user: nobody group: nobody master: path: /var/run/dovecot/auth-master mode: 384 user: nobody group: nobody plugin: quota: maildir:storage=1024 -------------------------------------

4 3

[Dovecot] Postfix LDA vs the Dovecot LDA + other quetions
by Linux Advocate 24 Feb '09

24 Feb '09

Guys , 1. What is the benefit of using Dovecot' LDA as compared to postfix's LDA? Will there be better performance? And along the same vein, 2. What is the benefit of using Dovecot's SASL as compared to Cyrus SASL? Is there better performance? 3. Quota Management ( Creating, enforcing, notifying ) - is it done thru dovecot or postfix or are both needed? Regards, mgomez

5 14

[Dovecot] pkgsrc-ing sieve and managesieve
by Edgar Fuß 24 Feb '09

24 Feb '09

I'm trying to integrate both the new sieve implementation and managesieve into pkgsrc. Managesieve seems to need some dovecot libraries that dovecot doesn't install (lib-storage/libstorage.a, lib-auth/libauth.a, lib-imap/libimap.a, lib-index/libindex.a, lib-mail/libmail.a, lib-charset/libcharset.a, lib-dict/libdict.a and lib-charset/libcharset.a). There seems to be no provision in pkgsrc for the build of a package needing the work directory of the build of another package. Would it make sense for dovecot to install those libraries?

2 1

[Dovecot] sieve-test again
by Ian P. Christian 23 Feb '09

23 Feb '09

I'm not sure what it is I'm doing wrong here, perhaps my 'test' mail is not in the format expected by sieve-test. The 'test' file is a file taken directly from a dovecot maildir - so is in the normal format of having headers, a new line, then the body. There is nothing exceptional about this email. Notice how the header below is 'innocent', however it appears that the test is ignoring this header. # grep DSPAM-Res test X-DSPAM-Result: Innocent # cat sievetest require ["regex", "fileinto","envelope","reject","vacation","relational","comparator-i;ascii-numeric"]; if header :contains "X-DSPAM-Result" "Innocent" { fileinto "moo"; stop; } # sieve-test -c ./sievetest ./test Performed actions: (none) Implicit keep: * store message in folder: INBOX Info: final result: success I don't think it's this header, but rather *all* headers, as the following check also appears to fail. if header :contains "To" "o" { fileinto "Spam"; stop; } Am I mis-using the tool? Thanks, Ian

2 3

[Dovecot] from postfix deliver to dovecot deliver
by Johan Hendriks 23 Feb '09

23 Feb '09

Hello all. I use dovecot in combination with postfix, postfixadmin for virtual users. Deliver to the mailbox now is done by postfix Now i want to enable quota's and therefor i need dovecot deliver. Can i just switch to dovecot deliver? Or must i first do some prrperations in the mailboxes. regards -- _______________________ *Johan Hendriks* *Schavemaker Transport* Tel: +31 (0)251 229098 Fax: +31 (0)251 212016 email: j.hendriks(a)schavemaker.com <mailto:j.hendriks@schavemaker.com> web: http://www.schavemaker.com _______________________ /Confidentiality Notice: The information in this document may be confidential. It is intended only for the use of the named recipient. If you are not the intended recipient, please notify me immediately and then delete this document. Do not disclose the contents of this document to any other person, nor take any copies. Violation of this notice may be unlawful. / _______________________

2 2

[Dovecot] lda catchall
by Stéphane Cottin 23 Feb '09

23 Feb '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'm using dovecot lda in a qmail / vpopmail testing environnement. I would like to replace vpopmail lda with dovecot's one , because of quotas , sieve filers, indexing while delivering ... Everything is working great, except the catchall. If I want to bounce, a first line like "# bounce-no-mailbox" in the .qmail file works, without this line, emails to unknown addresses are deferred and stay in the queue, because they are deliverable for qmail, but not for dovecot lda. How can I set a catchall email address for dovecot lda ? thx Stéphane Cottin -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmixCcACgkQNgQUYqHIqqvttQCfcNBLd9Qo06g5nMcrfZAwVlpv WegAnj7jIA/XFJWmt/2jkg1sxRYKRaI7 =JTTV -----END PGP SIGNATURE-----

1 0

[Dovecot] sieve-test issue
by Ian P. Christian 23 Feb '09

23 Feb '09

Hi all, I'm using a build of dovecot 1.2, and I'm seeing this: # ls -al 1235038385.M765212P13483.tch-mailstore1\,S\=3927\,W\=4017\:2\,S -rw------- 1 mail mail 3927 2009-02-19 10:13 1235038385.M765212P13483.tch-mailstore1,S=3927,W=4017:2,S (root@tch-mailstore1:/mailstore/mail/domain.net/i/ian/mail/.postini/cur) (2009-02-23 13:46:04) # sieve-test -c ../../../sieve/default.sieve 1235038385.M765212P13483.tch-mailstore1\,S\=3927\,W\=4017\:2\,S Fatal: Can't open mail stream as raw Can anyone suggest what this error means? Thanks, Ian

1 1

[Dovecot] v1.2.beta1 released
by Timo Sirainen 23 Feb '09

23 Feb '09

http://dovecot.org/releases/1.2/beta/dovecot-1.2.beta1.tar.gz http://dovecot.org/releases/1.2/beta/dovecot-1.2.beta1.tar.gz.sig Largest changes since v1.1 can be found from http://dovecot.org/doc/NEWS-1.2 Changes since alpha5: - Added support for ESORT extension (also SORT .. RETURN (PARTIAL) from CONTEXT=SORT that allows windowing SORT results) - pop3: If client idles for 10 seconds, commit transaction (allowing mbox to become unlocked). - crashfixes to virtual mailboxes - all kinds of bug fixes, error message improvements, etc. - http://dovecot.org/patches/1.2/stats-plugin.c may give some interesting statistics about commands There isn't really much left to do for v1.2.0 except some small fixing to shared mailbox code. And writing documentation for it..

3 6

[Dovecot] IMAP ACLs and global ACLs in v1.2
by Timo Sirainen 23 Feb '09

23 Feb '09

I just committed code for IMAP ACL support based on the code from Kolab people. I did quite large changes though. I also changed how global ACLs are handled. Previously local ACLs could override global ACLs, but now that users are able to modify the ACLs I think it should be the other way around. From what I've heard global ACLs are typically added for things like 1) disallowing user from doing something in specific mailboxes such as expunge messages, 2) giving a specific master user access to some mailboxes ONLY, such as voicemail. For 1) case a user really shouldn't be able to grant himself privileges back but still should be able to share the mailbox to others (so the global ACL should have 'a' privilege). For 2) case a user shouldn't be able to remove privileges from the voicemail. So now the user can freely update the local ACLs in whatever way he wants. But in the end if there are any global ACLs, all the local ACLs for those identifiers are ignored. Also local negative identifiers are ignored, so that e.g. global "user=foo lr" can't be disabled by a local "-user=foo lr" ACL. I guess I should update the Wiki ACL page about how exactly the ACLs are applied now that they're well in my memory. :) One thing I'm not really sure about is the "owner" handling. IMAP ACL specifications have no such concept. I think many/most other servers simply add a default ACL for the user name directly. It's a useful concept though, especially with the global ACLs. So currently Dovecot replies: x getacl inbox * ACL "inbox" "owner" lrwstiekxacd x OK Getacl completed. But should it just internally convert "owner" to "username" when replying? But then again if there's a separate rule directly for the "username" it breaks. Do any of the existing ACL-capable IMAP clients want to see the username in the ACL reply or can they handle "owner" just fine? Also what about global ACLs? The current code just lists them twice with local ACLs, which is definitely wrong: x getacl foo * ACL "foo" "xx" w "xx" lr "owner" lrwstiekxacd x OK Getacl completed. Perhaps the global ACLs could have a prefix character? Maybe '#'? Which reminds me, the current dovecot-acl file is mapped to IMAP ACLs like this: dovecot-acl : IMAP ACL - anyone : anyone - authenticated : authenticated - owner : owner - user=name : name - group=name : $name - group-override=name : !$name So all global ACLs would get the # prefix, like #anyone, #name, #!$name. Any thoughts? BTW. Listing shared mailboxes still doesn't work. I guess we'll see tomorrow if I still have energy to get that done.

6 14

[Dovecot] dovecot auth
by ben 23 Feb '09

23 Feb '09

Hi, I'm using dovecot with a passdb-file . When I login using : test(a)testdomain.com I notice dovecot strips off the domainname and looks up everything in front of the @ in his passdb-file... So I went on to configure the following in my dovecot.conf : auth_username_format = %u@%d restarted dovecot, but he still strips of the @ and the domainname ... LOG (sorry for f* up format) : dovecot: Feb 21 12:15:08 Info: auth(default): new auth connection: pid=24483 dovecot: Feb 21 12:15:12 Info: auth(default): client in: AUTH 1 PLAIN service=pop3 lip=195.207.136.102 rip=195.207.136.108 lport=110 rport=36132 resp=AGJlbkBpdHV4LmJlAHRlc3QxMjMu dovecot: Feb 21 12:15:12 Info: auth(default): passwd-file(test@testdomain.com@testdomain.com,195.207.136.108): lookup: user=test file=/opt/dovecot/etc/dovecot.passwd dovecot: Feb 21 12:15:12 Info: auth(default): passwd-file(test@testdomain.com@testdomain.com,195.207.136.108): unknown user dovecot: Feb 21 12:15:14 Info: auth(default): client out: FAIL 1 user=test(a)testdomain.com Can someone enlighten me on how to config it so he looks up "test(a)testdomain.com" fully ? thx

3 5