March 2015 - dovecot - dovecot.org

Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1
by Stephan Bosch 12 Mar '15

12 Mar '15

Hello Dovecot users, Last time I had a few stupid problems in the releases, so I'll follow Timo's example and I release an RC first. The highlights include the implementation of the index and metadata extensions. Quite a few bugs are fixed as well. Changelog v0.4.7: * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output to binary up-to-date checking. - The Sieve interpreter now flushes duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - file storage: Restructured storage initialization to address backwards compatibility issues. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1… http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

5 12

Expunge (?)
by Muzaffer Tolga Ozses 12 Mar '15

12 Mar '15

Hi, I don't know if this is the correct term. Let me elaborate. From time to time, I want to expunge (?) the mail folder so that users' Trash'd mails don't occupy space. So, here are my questions: 1. Is this the correct term? 2. Is it a Good Idea™ to do it? Thanks, -- mto

2 2

Sieve security: Any way to protect credentials used in extprograms?
by E.B. 11 Mar '15

11 Mar '15

I need to connect to a database in a script called using Sieve extprograms plugin. When delivering mail, Sieve is running as the mail recipient user, which means any files, either the sieve script or the extprograms it invokes, are run under that user's permissions. What would be a way to hide the database credentials in a more restricted file? I can think of... * Store the credentials in the database itself, return them from a DICT lookup that is fed to the sieve script. The credentials would have to be duplicated for every user record in the database. * Somehow put the value in the Sieve environment, but I'm not sure how to get a value into the environment (just return it in a userdb lookup?) and then how to get at that value once I'm executing the sieve script (can I retrieve it with the "environment" extension?) Are there better ways? [Of course, if I could do more elaborate things like call a stored procedure from Dovecot DICT then I could avoid this situation]

2 1

Sieve reject with ORIG-TO vs TO
by E.B. 11 Mar '15

11 Mar '15

Hi, The bounce message generated by the reject extension has what looks like a hard coded message prefix that comes before the configurable reason text: "Your message to <username> was automatically rejected:" In some cases, the <username> is NOT the original-to address, which can cause confusion to the sender or expose private aliasing data that some people might want to hide. Is there a way to make the reject extension use the original-to address in that hard-coded message prefix? Is there a way to completely customize the FULL message? Is there a way to customize the headers for such messages?

2 1

FTS search with CLucene giving odd results/exposing config issue
by David Gessel 11 Mar '15

11 Mar '15

I recently got FTS search with Clucene working on FreeBSD 10.1 (I had some trouble with 9.3, postponed solving it, then restarted with 10.1 by manually installing libtextcat, a listed dependency on http://wiki2.dovecot.org/Plugins/FTS/Lucene, but not part of the FreeBSD makefile: maybe that helped, maybe it was irrelevant). However, I now get some anomalous results. I'm putting them together as they may be related. Or, maybe not. 1) Searching from Thunderbird returns 35x as many results for the same term as searching from the command line. I'm not sure if the indexes are created or enumerable. 2) There seems to be a glitch in the maildir naming structure that results in a double slash (//) which doesn't seem to impact IMAP serving or Sieve, but maybe is causing the problem above with Clucene. Any hints? If I run a search from Thunderbird and "run search on server" for a somewhat infrequently used term like "titanium," I get results in more or less every archive folder one by one over about 20 minutes of searching and get about 516 results. It seems to run a little faster without the "run search on server" checkbox selected, but Thunderbird on Windows is abandonware and the only useful datapoint is 516 results. Maybe it is time for Claws. Anyway, if I run: # doveadm search -u user(a)domain.com mailbox '*' body titanium doveadm(user(a)domain.com): Error: stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory doveadm(user(a)domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal error occurred. Refer to server log for more information. [2015-03-11 09:11:16] bcadc922ba26b953a25701002812f29a 812478 bcadc922ba26b953a25701002812f29a 9338037 bcadc922ba26b953a25701002812f29a 9342918 bcadc922ba26b953a25701002812f29a 9343205 bcadc922ba26b953a25701002812f29a 9346096 bcadc922ba26b953a25701002812f29a 9350977 bcadc922ba26b953a25701002812f29a 9351379 bcadc922ba26b953a25701002812f29a 9352318 bcadc922ba26b953a25701002812f29a 9352441 bcadc922ba26b953a25701002812f29a 9352501 3d5d982e3a89b853c12c01002812f29a 1901940 3d5d982e3a89b853c12c01002812f29a 1908073 3d5d982e3a89b853c12c01002812f29a 1941453 3d5d982e3a89b853c12c01002812f29a 2036962 3d5d982e3a89b853c12c01002812f29a 2038360 I get 15 results. # doveadm fetch -u user(a)domain.com "mailbox" mailbox-guid 3d5d982e3a89b853c12c01002812f29a uid 1908073 mailbox: INBOX All are from the inbox. This takes roughly 30 seconds (much faster, that's good, though still not super fast) and in the logs I see promising messages like: Mar 11 09:27:52 host dovecot: indexer-worker(user(a)domain.com): Indexed 3708 messages in Commerce but also slightly less promising messages like: Mar 11 09:27:52 host dovecot: indexer-worker(user(a)domain.com): Warning: Maildir /mail/domain.com/user//.Bulkmail: Expunged message reappeared, giving a new UID (old uid=9335155, file=1426089637.M350915P91305.host,S=23845,W=24282:2,) If I execute # doveadm index -A '*' I get doveadm(user(a)domain.com): Error: stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory doveadm(user(a)domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal error occurred. Refer to server log for more information. [2015-03-11 09:51:26] (there's nothing in the server log) But note in both previous results a double slash (//) in the directory structure. I don't think that's right. So: Does # doveadm search search subfolders? The internets say either '*' or INBOX should search the hierarchy and the search command does set off indexing operations on every subfolder as indicated by the logs, but the results are only from the top level folder.... perhaps that is related to... Why do some errors report a double slash (//) in directory names? I don't seem to be having any other symptoms (if, indeed, this is one) of incompatibilities. My sieve filters work just fine and are accessible from the managesieve plugin in Thunderbird. My config file is below: # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.1-RELEASE amd64 auth_default_realm = domain.com auth_mechanisms = plain login auth_realms = domain.com domain2.org disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 hostname = domain.com imap_idle_notify_interval = 29 mins last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes listen = * mail_gid = 5000 mail_location = maildir:~:CONTROL=/var/no-quota/%u mail_plugins = quota mail_log notify fts fts_lucene mail_temp_dir = /var/tmp mail_uid = 5000 maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate spamtest spamtestplus editheader mbox_lazy_writes = no namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. mime_parts mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=50G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/ sieve_dir = ~/sieve sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric +editheader sieve_global_dir = /usr/local/etc/dovecot/sieve } postmaster_address = user(a)domain.com protocols = imap lmtp sieve sieve sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 vsz_limit = 128 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = </usr/local/etc/ca/host-cert.pem ssl_key = </usr/local/etc/ca/host-unencrypted-key.pem ssl_prefer_server_ciphers = yes userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota mail_log notify fts fts_lucene sieve } protocol imap { mail_max_userip_connections = 20 mail_plugins = quota mail_log notify fts fts_lucene imap_quota imap_zlib } protocol lmtp { mail_plugins = quota mail_log notify fts fts_lucene sieve } (END)

1 0

libdriver_msql.so
by kaniggl 11 Mar '15

11 Mar '15

Hello, i installed dovecot 2.11.1 on ubuntu Linux dadd3041 3.16.0-30-generic #40-Ubuntu SMP Mon Jan 12 22:07:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux To make it clear, architecture is PowerPC 64bit Then i installed dovecot via apt-get. But the file /usr/lib/dovecot/modules/auth/libdriver_mysql.so is missing. So i compiled dovecot myself on this system and got a file /usr/lib/dovecot/modules/auth/libdriver_mysql.so Then i restartetd the server and try to do a login via a webmailer (roundcube), but i get no connection to the database server. This is found in /var/log/mail.log Mar 11 15:19:04 dadd3041 dovecot: master: Dovecot v2.2.9 starting up (core dumps disabled) Mar 11 15:19:04 dadd3041 postfix/master[3367]: reload -- version 2.11.1, configuration /etc/postfix Mar 11 15:19:04 dadd3041 postfix/master[3367]: terminating on signal 15 Mar 11 15:19:04 dadd3041 postfix/master[3597]: daemon started -- version 2.11.1, configuration /etc/postfix Mar 11 15:19:25 dadd3041 dovecot: auth: Error: dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: ELFCLASS32 Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver 'mysql' Mar 11 15:19:25 dadd3041 dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs Mar 11 15:19:25 dadd3041 dovecot: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=::1, lip=::1, TLS handshaking, session=<rJpS9AMR9wAAAAAAAAAAAAAAAAAAAAAB> I know this means it is not a 64bit object but how could i achieve to get right one? best regards, kaniggl

3 4

dec2str ...
by Hardy Flor 11 Mar '15

11 Mar '15

a very stupid question: What reason is there for an output with printf until dec2str to convert the numeric value to a string and not to use the format identifier "%d" or "u%"?

2 1

full text index "per user"?
by Ralf Hildebrandt 11 Mar '15

11 Mar '15

Is there any way of disabling the creation of a full text index on a per user basis? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

1 1

Deduplicate Public Folder
by Bobber 11 Mar '15

11 Mar '15

How do you perform the doveadm deduplicate command on public folders? If I specify a user, nothing is found. Using -A also doesn't seem to work. -- *Bob Wooldridge* Blog: http://kc0dxf.net/blog/

2 1

Sieve extprograms socket vs. direct execution
by E.B. 11 Mar '15

11 Mar '15

Hi, I'm hoping to get some clarification of the differences between calling a script using the Sieve extprograms plugin execute method via direct execution or using the socket feature. Being naive, I see the socket option and think that way you tell Dovecot to spawn a daemon and I think that's going to be far superior in performance. But if that really was the difference, why would direct execution even be an option? (And doesn't the daemon still have to spawn a new shell every time if the target is a shell script? Does that defeat the purpose of having a daemon?) So I don't really know what the difference is, and under what circumstances you'd want to use one or the other. Can someone please help clarify?

1 0