- dovecot - dovecot.org

[Dovecot] When should dictionary entries for the expire plugin be added/updated?
by Ralph Seichter 28 Aug '10

28 Aug '10

I've set up Dovecot 2.0 with the expire plugin using an SQLite DB, and when mail is delivered via dovecto-lda, the DB gets updated as expected. However, when I use Thunderbird 3.1.2 for Mac OS X to move a message to any of the folders known to the expire plugin (e.g. the 'tmp' folder), the SQLite DB is not updated. Here is an excerpt of my current configuration: # /etc/dovecot/dovecot.conf dict { expire = sqlite:/etc/dovecot/dict-expire.conf } plugin { expire = Junk expire2 = Trash expire3 = tmp expire4 = *.News expire5 = *.Reports expire_dict = proxy::expire sieve = /etc/dovecot/sieve/%u.sieve } service dict { unix_listener dict { # Ensure that there is no permission-related problem mode = 0666 } } protocol lda { mail_plugins = expire sieve postmaster_address = postmaster(a)domain.tld sendmail_path = /usr/sbin/sendmail } # /etc/dovecot/dict-expire.conf connect = /var/lib/dovecot/expires.sqlite map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire fields { username = $user mailbox = $mailbox } } # Schema dump of /var/lib/dovecot/expires.sqlite PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE expires ( username VARCHAR(75) NOT NULL, mailbox VARCHAR(255) NOT NULL, expire integer NOT NULL, PRIMARY KEY (username, mailbox) ); CREATE TRIGGER mergeexpire BEFORE INSERT ON expires FOR EACH ROW BEGIN UPDATE expires SET expire=NEW.expire WHERE username=NEW.username AND mailbox=NEW.mailbox; SELECT RAISE(IGNORE) WHERE (SELECT 1 FROM expires WHERE username=NEW.username AND mailbox=NEW.mailbox) IS NOT NULL; END; COMMIT; With these settings, rows will be inserted to (or updated within) expires.sqlite when dovecot-lda stores messages to any of my "Reports" folders using sieve rules. When I manually move messages to "tmp" or "Junk", the database remains unchanged. Also, when I delete messages in Thunderbird, I'd expect database additions for the folder "Trash", but this does not happen either. What am I doing wrong here? -Ralph

5 27

[Dovecot] Error: service(auth): child 2212 killed with signal 11
by Toorop 28 Aug '10

28 Aug '10

Hi, Auth always crash (segfault) using vpopmail backend. Any idea ? vpopmail 5.5.0 os: ubuntu 10.04 on rackspace cloud # dovecot -n dovecot -n # 2.0.1: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.35.1-rscloud x86_64 Ubuntu 10.04.1 LTS auth_debug = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no first_valid_uid = 89 log_path = /var/log/dovecot/dovecot.log mail_debug = yes passdb { driver = vpopmail } protocols = imap pop3 service auth { unix_listener auth-userdb { group = vchkpw mode = 0600 user = vpopmail } user = root } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { driver = vpopmail } Log: Aug 28 10:59:48 auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Aug 28 10:59:48 auth: Debug: auth client connected (pid=2211) Aug 28 10:59:49 auth: Debug: client in: AUTH 1 CRAM-MD5 service=imap secured lip=184.106.217.69 rip=178.32.33.247 lport=143 rport=49099 Aug 28 10:59:49 auth: Debug: client out: CONT 1 PDM3MjI0MzQ4NjQzNTI3NjUuMTI4Mjk5MzE4OUByYWNrc3BhY2UuZGVkaW1haWwuZXU+ Aug 28 10:59:49 auth: Debug: client in: CONT<hidden> Aug 28 10:59:49 auth: Debug: vpopmail(toorop(a)respublica.fr,178.32.33.247): lookup user=toorop domain=respublica.fr Aug 28 10:59:49 auth: Debug: client out: OK 1 user=toorop(a)respublica.fr Aug 28 10:59:49 auth: Debug: master in: REQUEST 1 2211 1 e4920e51f96ae453663098df19c5b374 Aug 28 10:59:49 auth: Debug: vpopmail(toorop(a)respublica.fr,178.32.33.247): lookup user=toorop domain=respublica.fr Aug 28 10:59:49 imap-login: Info: Internal login failure (auth failed, 1 attempts): user=<toorop(a)respublica.fr>, method=CRAM-MD5, rip=178.32.33.247, lip=184.106.217.69, mpid=2214, TLS Aug 28 10:59:49 master: Error: service(auth): child 2212 killed with signal 11 (core dumps disabled) strace : http://pastebin.com/kYsWHP5J Thanks, -- Toorop «Ils ne savaient pas que c'était impossible, alors ils l'ont fait.» Mark Twain

2 3

[Dovecot] Roff typo in sievec.1.in
by Mike Abbott 27 Aug '10

27 Aug '10

There is a roff typo in pigeonhole's sievec.1.in. Roff treats the leading apostrophe on line 54 as an invalid command and produces bad output: dump to be written to stdout. The out-file argument may also be omitted, which has the same effect as for a com- piled Sieve binary file. Note that this option is not The output should be: dump to be written to stdout. The out-file argument may also be omitted, which has the same effect as '-'. The output is identical to what the sieve-dump(1) command pro- duces for a compiled Sieve binary file. Note that this Here is a patch: --- a/doc/man/sievec.1.in +++ b/doc/man/sievec.1.in @@ -50,8 +50,9 @@ Don\(aqt write the binary to \fIout\-file\fP, but write a textual dump of the binary instead. In this context, the \fIout\-file\fP value '\-' has special meaning: it causes the the textual dump to be written to \fBstdout\fP. -The \fIout\-file\fP argument may also be omitted, which has the same effect as -'\-'. The output is identical to what the \fBsieve\-dump\fP(1) command produces +The \fIout\-file\fP argument may also be omitted, which has the same effect +as '\-'. +The output is identical to what the \fBsieve\-dump\fP(1) command produces for a compiled Sieve binary file. Note that this option is not allowed when the \fIout\-file\fP argument is a directory. .TP

2 1

[Dovecot] pigeonhole needs to ship doc/man/{reporting-bugs.inc, sed.sh}
by Paul Howarth 27 Aug '10

27 Aug '10

A tarball created from current pigeonhole hg using "make dist" doesn't include the files doc/man/{reporting-bugs.inc,sed.sh} and fails to build as a result. Attached patch works for me. Paul

2 1

[Dovecot] dovecot - mac firewall problem
by Patrick Fay 27 Aug '10

27 Aug '10

Hi, I am running dovecot 1.2.11 on mac osx 1.5.8. Everything works perfectly with the application-level firewall off, but enabling the application firewall prevents dovecot connections. I have tried explicitly authorizing dovecot in the firewall, but it does not work. I have searched everywhere I can think of to look, and haven't found a solution, but have seen a couple other reports of what seems to be the same problem. The firewall logs the activity with what looks like a corrupt process name: a typical appfirewall.log entry looks like: Aug 26 20:43:45 hostname Firewall[55]: Deny ^L connecting from XX.XX.XX.XX:37310 uid = 0 proto=6 Aug 26 20:43:53 hostname Firewall[55]: Deny ^H�^U��^Z connecting from XX.XX.XX.XX:37310 uid = 0 proto=6 Aug 26 20:44:09 hostname Firewall[55]: Deny ^L connecting from XX.XX.XX.XX:37310 uid = 0 proto=6 Aug 26 20:44:34 hostname Firewall[55]: Deny ^L connecting from XX.XX.XX.XX:37312 uid = 0 proto=6 Aug 26 20:44:45: --- last message repeated 6 times --- where "hostname" is my server name and the XX's are my client's IP address. For all of the other services I've used, the process name (e.g. dovecot) should appear after "Deny" when blocking traffic, instead of the funny characters. Any advice on how I could resolve this issue would be greatly appreciated. Thanks!

2 1

[Dovecot] LDAP & static userdb
by spamvoll＠googlemail.com 27 Aug '10

27 Aug '10

hi.. im just testing 2.0 before upgrading to v1.2 :) in auth-ldap.conf.ext i found # If you don't have any user-specific settings, you can avoid the userdb LDAP # lookup by using userdb static instead of userdb ldap, for example: # <doc/wiki/UserDatabase.Static.txt> #userdb { #driver = static #args = uid=vmail gid=vmail home=/var/vmail/%u #} On 1.2 im using LDAP lookup and everthing works like a charm. user_attrs = homeDirectory=home,uid=mail=maildir:/home/MAILBOXES/%$/mail On 2.0 is now tried to use this option cause all my mail are stored in /home/MAILBOXES/%uid/mail userdb { driver = static args = uid=vmail gid=vmail home=/home/MAILBOXES/%u/mail } and that doesnt work log say: dovecot: imap(hpeter): Error: user hpeter: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/home/MAILBOXES/hpeter/mail Hans

1 1

Re: [Dovecot] (Single instance) attachment storage
by Damon Atkins 27 Aug '10

27 Aug '10

Some interesting reading on SHA256 checksum http://blogs.sun.com/bonwick/entry/zfs_dedup http://blogs.sun.com/darren/entry/improving_zfs_dedup_performance_via

1 0

[Dovecot] Single instance storage - testing please
by Timo Sirainen 27 Aug '10

27 Aug '10

http://hg.dovecot.org/dovecot-2.0-sis contains the code for it. Otherwise it's the latest (as of writing this) dovecot-2.0 hg tree. Please test if you're interested in SIS. :) Once there's at least some testing, I'll probably add this to v2.0.x since very little of this new code is used when SIS is disabled (which is the default of course). SIS works pretty much like explained in http://dovecot.org/list/dovecot/2010-July/050832.html and http://dovecot.org/list/dovecot/2010-July/050992.html Two things I'm not yet entirely sure about: 1. What hash algorithm to use? Currently it's hard coded to SHA1. Besides more CPU usage, the other potential problem with larger hashes is that they also generate larger filenames. The filenames are currently hex-encoded, but to save space they could be changed to some kind of modified-base64 (base64 uses '/' chars, so it can't be regular base64). Example filename lengths: hex modified-base64 SHA1 73 50 SHA256 97 66 SHA512 161 109 Yet another possibility would be to use SHA256/SHA512 and just truncate the hash length to less number of bits. 2. Should I add support for trusting hash uniqueness and to avoid disk I/O generated by the byte-by-byte comparison? It could still first check that the file sizes match. Usage ----- You can enable SIS for sdbox and mdbox: mail_attachment_dir = /var/attachments Just setting the above enables "instant SIS", where byte-by-byte comparison is done immediately during saving mails. Alternative is to leave the comparing later by setting: mail_attachment_fs = sis-queue /var/attachments/queue:posix This does no deduplication itself yet. To do that you'll need a nighty (or whatever) run, which calls: doveadm sis deduplicate /var/attachments /var/attachments/queue There's also a feature to easily find all attachments based on a hash. For example: % sha1sum foo 351641b73feb7cf7e87e5a8c3ca9a37d7b21e525 foo % doveadm sis find /var/attachments 351641b73feb7cf7e87e5a8c3ca9a37d7b21e525 /var/attachments/35/16/351641b73feb7cf7e87e5a8c3ca9a37d7b21e525-e13a841f28ba764c123b00008c4a11c1 /var/attachments/35/16/351641b73feb7cf7e87e5a8c3ca9a37d7b21e525-1d3b940628ba764c0b3b00008c4a11c1 If you want to save attachments to a separate files without SIS (e.g. you want to use your filesystems deduplication), set: mail_attachment_fs = posix By default only attachments larger than 128 kB are written to attachment storage. You can change it from: mail_attachment_min_size = 128k It's also possible to create a plugin that adds further restrictions to when the attachment is saved separately. This might be useful to reduce disk seeks for attachments that are typically shown inline by clients/webmail. You can do this by overriding mailbox.save_is_attachment() method. If you want to distribute attachments to multiple filesystems, just create /var/attachments/[0-9a-f][0-9a-f] as symlinks pointing to whatever mount paths you want.

3 8

[Dovecot] Panic, rawtrace, log, etc
by Brandon Lamb 27 Aug '10

27 Aug '10

While running some doveadm expunge -A mailbox Spam savedbefore 30d and doveadm purge -A after 2010-08-26 16:20:51 auth: Error: net_accept() failed: Too many open files 2010-08-26 16:20:51 master: Warning: service(auth): process_limit reached, client connections are being dropped 2010-08-26 16:21:21 auth: Panic: file auth-request-handler.c: line 85 (auth_request_handler_unref): assertion failed: (hash_table_count(handler->requests) == 0) 2010-08-26 16:21:21 auth: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x39c1a) [0x7fe41f5f5c1a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x39c6a) [0x7fe41f5f5c6a] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fe41f5d3026] -> dovecot/auth [8 wait, 0 passdb, 0 userdb]() [0x412153] -> dovecot/auth [8 wait, 0 passdb, 0 userdb]() [0x40c3ad] -> dovecot/auth [8 wait, 0 passdb, 0 userdb](auth_client_connections_deinit+0x41) [0x40c461] -> dovecot/auth [8 wait, 0 passdb, 0 userdb](main+0x2ea) [0x4168ca] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fe41e60ac4d] -> dovecot/auth [8 wait, 0 passdb, 0 userdb]() [0x40b1f9] 2010-08-26 16:21:21 master: Error: service(auth): child 14880 killed with signal 6 (core dumps disabled) 2010-08-26 16:21:21 auth: Error: Master requested auth for nonexistent client 14875 2010-08-26 16:21:21 auth: Error: Master requested auth for nonexistent client 14873 2010-08-26 16:21:21 auth: Error: Master requested auth for nonexistent client 14873 2010-08-26 16:21:21 auth: Error: Master requested auth for client 14874 with invalid cookie 2010-08-26 16:21:21 auth: Error: Master requested auth for client 14874 with invalid cookie 2010-08-26 16:21:21 auth: Error: Master requested auth for client 14875 with invalid cookie 2010-08-26 16:21:21 auth: Error: Master requested auth for client 14876 with invalid cookie 2010-08-26 16:21:21 auth: Error: Master requested auth for client 14875 with invalid cookie

2 3

[Dovecot] Configuration Update From v1.x To v2.0.1-2
by Carlos Mennens 26 Aug '10

26 Aug '10

I just upgraded my Dovecot IMAP package on my mail server and when I restarted the service, I got the following warnings... doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:12: add auth_ prefix to all settings inside auth {} and remove the auth {} section completely doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:14: passdb pam {} has been replaced by passdb { driver=pam } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:16: userdb passwd {} has been replaced by userdb { driver=passwd } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:18: auth_user has been replaced by service auth { user } The list was much longer but I was able to self resolve them. I was wondering if someone here can assist me in converting my 'dovecot.conf' file? Below is how my dovecot.conf file appears: protocols = imap disable_plaintext_auth = yes log_timestamp = "%b %d %H:%M:%S " mail_location = maildir:~/mail auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep } auth default { mechanisms = plain login passdb pam { } userdb passwd { } user = root socket listen { client { path = /var/run/dovecot/auth-client user = postfix group = postfix mode = 0660 } } } ***doveconf -n*** [root@mail /]# doveconf -n # 2.0.1: /etc/dovecot/dovecot.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:12: add auth_ prefix to all settings inside auth {} and remove the auth {} section completely doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:14: passdb pam {} has been replaced by passdb { driver=pam } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:16: userdb passwd {} has been replaced by userdb { driver=passwd } doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:18: auth_user has been replaced by service auth { user } # OS: Linux 2.6.35 x86_64 auth_mechanisms = plain login mail_location = maildir:~/mail passdb { driver = pam } protocols = imap service auth { unix_listener auth-client { group = postfix mode = 0660 user = postfix } user = root } userdb { driver = passwd } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep }

4 7