September 2015 - dovecot

Cyrus mailbox (plain files) to Dovecot
by Wolfgang Rosenauer 25 Sep '15

25 Sep '15

Hi, I'm migrating from a Cyrus to a Dovecot installation right now. As part of it I've got plain Cyrus mailboxes (w/o real metadata; so to say I've got the /var/spool/imap/user part but not the /var/lib/imap/user one)). Those former mailboxes I want to provide under a public namespace via ACLs. The question I cannot answer right now is: How can I convert these plain mailboxes on a FS level to maildir++ so I can provide them as public mailboxes on the new system? The tools I have found require a valid/complete cyrus mailbox. (I don't really care about message flags etc). Thanks, Wolfgang

2 1

v2.2.19 release candidate released
by Timo Sirainen 25 Sep '15

25 Sep '15

http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements. * "doveadm director flush" command has a changed meaning now: It safely moves users to their wanted backends, instead of simply forgetting the mapping entirely and leaving the existing connections untouched. Use -F parameter to get the original unsafe behavior. + Added support for imap-hiberanate processes. + Optimized tracking mailboxes' vsizes (= sum of all messages' size). If mailbox_list_index=yes, it's also stored in there. This makes it very efficient to look up vsizes for all mailboxes. + Added a quota "count" backend, which uses the mailbox vsizes to get the current quota usage. It requires using the new quota_vsizes=yes setting, which tracks the messages' "virtual sizes" rather than "physical sizes". Their distiction is minor and mostly irrelevant nowadays (if mail sizes should be counted with LF or CRLF newlines). + "doveadm director up/down" commands added. The monitoring script should be using these commands instead of changing the vhost count. This allows admin to manually disable a server by changing the vhost count to 0 without the monitoring script changing it back. + Added support for HAProxy protocol: http://wiki2.dovecot.org/HAProxy + Added push-notification plugin framework, which can be used to easily implement push notifications to various backends. Implemented "ox" backend for notifying Open-Xchange via HTTP/json. + imap_logout_format supports more variables now, e.g. number of deleted messages. + pop3: Added pop3_delete_type setting (related to pop3_deleted_flag). + plugin { fts_enforced=yes } setting now fails body searches unless it can be done via the full text search engine. + Added %{passdb:*} and %{userdb:*} variables to various places + auth: Added ":protected" suffix for passdb and userdb fields. If used, the field doesn't overwrite an existing field. + IMAP/POP3 proxy: If a backend server dies, avoid client reconnection spikes by slowly disconnecting clients over time. This is enabled by setting login_proxy_max_disconnect_delay=secs passdb extra field. + imap: Added new read-only METADATA entries: /private/specialuse, /shared/comment, /shared/admin + imap: If client disconnects in the middle of a command, log how long the command had been running. - mdbox: Rebuilding could have caused message's reference count to overflow the 16bit number in some situations, causing problems when trying to expunge the duplicates. - Various search fixes (fts, solr, tika, lib-charset, indexer) - Various virtual plugin fixes - Various fixes and optimizations to dsync, imapc and pop3-migration - imap: Various RFC compliancy and crash fixes to NOTIFY

5 13

dovecot --hostdomain issue
by dovecot＠outputservices.com 25 Sep '15

25 Sep '15

I am on a server I called "email. But dovecot says i am on sunfire.outputservices.com ------------------------------------ dovecot --hostdomain sunfire.outputservices.com ------------------------------------ I would like to at least get it to say email.outputservices.com. But what I really want it to say is posta.outputservices.com. First where does --hostdomain get it's information so I can clear that up? Then how can I get it to identify as posta.outputservices.com? Dave Ryan dovecot(a)outputservices.com Senior System Administrator Output Services, Inc. Thursday, September 24, 2015 1124 hrs MDT ----------------------------------------------------------- doveconf -n # 2.2.15: /usr/local/tools/dovecot/etc/dovecot/dovecot.conf # OS: SunOS 5.10 i86pc base_dir = /dovecot/var/run/dovecot log_path = /dovecot/log/log login_trusted_networks = 137.106.0.0/16 mail_location = mbox:~/posta:INBOX=/var/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /dovecot/etc/dovecot/conf.d/ldap.settings driver = ldap } protocols = imap service auth { unix_listener /postfix/mqueue/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } } ssl_cert = </dovecot/certs/dovecot.pem ssl_key = </dovecot/certs/dovecot.key ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 userdb { args = /dovecot/etc/dovecot/conf.d/ldap.settings driver = ldap } -----------------------------------------------------------

2 1

Released Pigeonhole v0.4.9.rc1 for Dovecot v2.2.19.rc1
by Stephan Bosch 25 Sep '15

25 Sep '15

Hello Dovecot users, Here's the Pigeonhole RC that goes with the Dovecot RC. Changelog v0.4.9.rc1: * Properly implemented checking of ABI version for Sieve interpreter plugins, much like Dovecot itself does for plugins. This will prevent plugin ABI mismatches. + Implemented a vnd.dovecot.environment extension. This builds upon the standard environment extension and adds a few more environment items, such as username and default mailbox. It also creates a variables namespace so that environment items can be accessed directly. I am still thinking about more environment items that can be added. + Sieve extprograms plugin: Made line endings of the input passed to the external programs configurable. This can be configured separately for each of the three extensions. + ManageSieve: Implemented proxy XCLIENT support. This allows the proxy to pass client information to the back-end. - ManageSieve: Fixed an assert failure occurring when a client disconnects during the GETSCRIPT command. - doveadm sieve plugin: Fixed incorrect initialization of mail user. This caused a few memory leaks. - sieve-filter command line tool: Fixed handling of failure-related implicit keep when there is an explicit default destination folder. This caused message duplication. - lib-sieve: Fixed bug in RFC5322 header folding. Words longer than the optimal line length caused empty lines in the output, which would break the resulting message header. This surfaced in References: headers with very long message IDs. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.9.rc1… http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.9.rc1… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

FTS not indexing new folders (mailboxes)
by Francisco Wagner C. Freire 24 Sep '15

24 Sep '15

Hello, Im getting problem on FTS on new created folders (mailboxes). How to simulate: - Enable FTS (solr on our case) - Create a new folder (mailbox) - Move some message to that mailbox - Try to search for the message contents *doveadm search -u user@domain text sometext* # Found nothing, but if i move the message to another mailbox, the search catch the message. # I tried doveadm index command but no response as well # This works only after a full fts rescan *doveadm fts rescan -u user@domain* # And now, works *doveadm search -u user@domain text sometext3c7b60241443ba55af680000cc4d0d4b 1* # My dovecot version = 2.2.18 (EE) # My fts config: fts = solr fts_autoindex = yes fts_autoindex_max_recent_msgs = 50 fts_solr = url=http://127.0.0.1:8080/solr/ Anyone got this problem?

3 2

Can't receive email
by lists＠lazygranch.com 24 Sep '15

24 Sep '15

No problem sending email, but I can't receive email. Diagnostics follow: a login user(a)domain.com password a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE] Logged in b select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 9 EXISTS * 1 RECENT * OK [UNSEEN 9] First unseen. * OK [UIDVALIDITY 1439944213] UIDs valid * OK [UIDNEXT 10] Predicted next UID * OK [HIGHESTMODSEQ 2] Highest b OK [READ-WRITE] Select completed (0.017 secs). c list "" * * LIST (\HasNoChildren \Trash) "." Trash * LIST (\HasNoChildren) "." Queue * LIST (\HasNoChildren \Sent) "." Sent * LIST (\HasNoChildren \Drafts) "." Drafts * LIST (\HasNoChildren) "." INBOX c OK List completed (0.001 secs). d lsub "" * * LSUB (\Trash) "." Trash * LSUB () "." Queue * LSUB (\Sent) "." Sent * LSUB (\Drafts) "." Drafts d OK Lsub completed (0.003 secs). e logout * BYE Logging out e OK Logout completed. closed --------------------------------------- from dovecot.log Sep 19 23:35:13 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [xxx.xxx.xxx.xxx] Sep 19 23:35:13 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [xxx.xxx.xxx.xxx] Sep 19 23:35:13 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A [xxx.xxx.xxx.xxx] Sep 19 23:35:13 auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Sep 19 23:35:13 auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Sep 19 23:35:13 auth: Debug: passwd-file /usr/local/etc/dovecot/users: Read 2 users in 0 secs Sep 19 23:35:13 auth: Debug: auth client connected (pid=1698) Sep 19 23:38:13 imap-login: Info: Disconnected: Inactivity (no auth attempts in 180 secs): user=<>, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, TLS handshaking, session=<MghMJCIgiACsOCYK> -------------------------------- # dovecot -n # 2.2.18: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: FreeBSD 10.1-RELEASE-p19 amd64 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes log_path = /var/log/dovecot.log mail_debug = yes mail_gid = 1003 mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~ mail_privileged_group = vpostfix mail_uid = 1003 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = postfix mode = 0600 user = postfix } } ssl_cert = </usr/local/etc/ipsec.d/certs/domain.com.crt ssl_key = </usr/local/etc/ipsec.d/private/domain.com.key userdb { args = username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } verbose_ssl = yes # -------------------------- # dovecot --version 2.2.18‎ ‎

2 2

dict lookup from quota plugin
by Bjørnar Ness 23 Sep '15

23 Sep '15

I am having trouble with using a custom daemon listening on a socket together with the quota plugin. My configuration is: quota = dict:User quota::dict:/tmp/dovecot-proxy-dict:: The problem is that dovecot sends keys without usernames, for example: priv/quota/storage Is it possible to get around this limitation without code modification? Also, is it planned to support the "new-style" userdb/passdb lookups from other plugins? -- Bj(/)rnar

2 1

bug - over system quota and dovecot-uidlist
by krzf83＠gmail.com 23 Sep '15

23 Sep '15

Ever since I've migraded from courier pop3 to dovecot pop3 I'm constantly getting complains from users that went over system quota for short while and then their pop3 client downloads all messages again. I never happened with courier pop3. I think that dovecot does not handle dovecot-uidlist file safely so it can withstand such situations. rsync and other similar programs always writes to a copy of the file like dovecot-uidlis.xcb453 and on success unlinks old file and renames new one. Dovecot should not assume that write operation on dovecot-uidlist will succeed. Yes, I know I can put metadata like dovecot-uidlist somewhere else, on another partition but that really splits mailbox into multiple locations making for example backups way harder to make, restore, share etc.

2 1

Dovecot proxy ignores trusted root certificate store
by Alex Bulan 22 Sep '15

22 Sep '15

Dovecot v2.2.18 OS: FreeBSD 10.1/amd64 Dovecot in proxy mode ignores the root certificate store and can't verify the backend's SSL certificate. I've pointed ssl_client_ca_file to my root certificate store, but I suspect ssl_client_ca_file is only used in imapc context. It seems to be ignored in proxy context. doveconf -n ssl_client_ca_file: ssl_client_ca_file = /usr/local/share/certs/ca-root-nss.crt In my password_query I return host set to the backend's IP address, starttls='yes', proxy='y'. The backend's certificate chain is correct and it verifies successfully with "openssl s_client -connect x.x.x.x:110 -starttls pop3 -CAfile /usr/local/share/certs/ca-root-nss.crt". But the Dovecot proxy fails to verify the intermediate certificate it receives from the backend. The inode atime of ca-root-nss.crt is never updated, either at Dovecot start or when it connects to the backend, so Dovecot (via the openssl library) never reads the file. Sep 20 19:59:48 dovecot: pop3-login: Invalid certificate: unable to get local issuer certificate: /C=US/O=GeoTrust Inc./OU=Domain Validated SSL/CN=GeoTrust DV SSL CA - G4 Sep 20 19:59:48 dovecot: pop3-login: Invalid certificate: certificate not trusted: /C=US/O=GeoTrust Inc./OU=Domain Validated SSL/CN=GeoTrust DV SSL CA - G4 Sep 20 19:59:48 dovecot: pop3-login: Error: proxy: Received invalid SSL certificate from x.x.x.x:110: unable to get local issuer certificate: /C=US/O=GeoTrust Inc./OU=Domain Validated SSL/CN=GeoTrust DV SSL CA - G4: user=<xxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, session=<lz9YjzYgIADYyWAp>

6 13

2.2.18 Regression: Incorrect STATUS response for virtual mailboxes
by Guilhem Moulin 22 Sep '15

22 Sep '15

As shown below, adding a message to TRASH doesn't increase virtual/test's MESSAGES count. However SELECTing virtual/test triggers the increase; so does running `doveadm mailbox status vsize virtual/test` in another shell. $ mkdir -m0700 ~/mail/virtual/test $ echo -e "TRASH\n\tall" > ~/mail/virtual/test/dovecot-virtual $ /usr/lib/dovecot/imap S: * PREAUTH [CAPABILITY IMAP4rev1 … ] Logged in as guilhem C: a STATUS TRASH (MESSAGES UIDNEXT UNSEEN) C: b STATUS virtual/test (MESSAGES UIDNEXT UNSEEN) S: * STATUS TRASH (MESSAGES 5599 UIDNEXT 5619 UNSEEN 0) S: a OK Status completed (0.001 secs). S: * STATUS virtual/test (MESSAGES 5599 UIDNEXT 5600 UNSEEN 0) S: b OK Status completed (0.032 secs). C: c APPEND TRASH {1+} S: x S: c OK [APPENDUID 1442277584 5619] Append completed (0.029 secs). C: d STATUS TRASH (MESSAGES UIDNEXT UNSEEN) C: e STATUS virtual/test (MESSAGES UIDNEXT UNSEEN) S: * STATUS TRASH (MESSAGES 5600 UIDNEXT 5620 UNSEEN 1) S: d OK Status completed (0.000 secs). S: * STATUS virtual/test (MESSAGES 5599 UIDNEXT 5600 UNSEEN 0) S: e OK Status completed (0.000 secs). # it doesn't help to LOGOUT and issue the STATUS command again. However, # it does to execute `doveadm mailbox status vsize virtual/test` in another # shell, or to SELECT virtual/test in the IMAP session. C: f STATUS TRASH (MESSAGES UIDNEXT UNSEEN) C: g STATUS virtual/test (MESSAGES UIDNEXT UNSEEN) S: * STATUS TRASH (MESSAGES 5600 UIDNEXT 5620 UNSEEN 1) S: f OK Status completed (0.000 secs). S: * STATUS virtual/test (MESSAGES 5600 UIDNEXT 5601 UNSEEN 1) S: g OK Status completed (0.000 secs). Cheers, -- Guilhem.

2 1