v2.2.16 release candidate released
http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig
Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16.
Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements.
The largest changes since v2.2.15:
* dbox: Resyncing (e.g. doveadm force-resync) no longer deletes
dovecot.index.cache file. The cache file was rarely the problem
so this just caused unnecessary slowness.
* Mailbox name limits changed during mailbox creation: Each part of
a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255
chars long (instead of 200). This also reduces the max number of
hierarchical levels to 16 (instead of 20) to keep the maximum name
length 4096 (a common PATH_MAX limit). The 255 char limit is
hopefully large enough for migrations from all existing systems.
It's also the limit on many filesystems.
+ director: Added director_consistent_hashing setting to enable
consistent hashing (instead of the mostly-random MD5 hashing).
This causes fewer user moves between backends when backend counts
are changed, which may improve performance (mainly due to caching).
+ director: Added support for "tags", which allows one director ring
to serve multiple backend clusters with different sets of users.
+ LMTP server: Added lmtp_user_concurrency_limit setting to limit how
many LMTP deliveries can be done concurrently for a single user.
+ LMTP server: Added support for STARTTLS command.
+ If logging data is generated faster than it can be written, log a
warning about it and show information about it in log process's
process title in ps output. Also don't allow a single service to
flood too long at the cost of delaying other services' logging.
+ stats: Added support for getting global statistics.
+ stats: Use the same session IDs as the rest of Dovecot.
+ stats: Plugins can now create their own statistics fields
+ doveadm server: Non-mail related commands can now also be used
via doveadm server (TCP socket).
+ doveadm proxying: passdb lookup can now override doveadm_port and
change the username.
+ doveadm: Search query supports now "oldestonly" parameter to stop
immediately on the first non-match. This can be used to optimize:
doveadm expunge mailbox Trash savedbefore 30d oldestonly
+ doveadm: Added "save" command to directly save mails to specified
mailbox (bypassing Sieve).
+ doveadm fetch: Added body.snippet field, which returns the first
100 chars of a message without whitespace or HTML tags. The result
is stored into dovecot.index.cache, so it can be fetched efficiently.
+ dsync: Added -t <timestamp> parameter to sync only mails newer than
the given received-timestamp.
+ dsync: Added -F [-]<flag> parameter to sync only mails with[out] the
given flag/keyword.
+ dsync: Added -a <mailbox> parameter to specify the virtual mailbox
containing user's all mails. If this mailbox is already found to
contain the wanted mail (by its GUID), the message is copied from
there instead of being re-saved. (This isn't efficient enough yet
for incremental replication.)
+ dsync: -m parameter can now specify \Special-use names for mailboxes.
+ imapc: Added imapc_features=gmail-migration to help migrations from
GMail. See http://wiki2.dovecot.org/Migration/Gmail
+ imapc: Added imapc_features=search to support IMAP SEARCH command.
(Currently requires ESEARCH support from remote server.)
+ expire plugin: Added expire_cache=yes setting to cache most of the
database lookups in dovecot index files.
+ quota: If overquota-flag in userdb doesn't match the current quota
usage, execute a configured script.
+ redis dict: Added support for expiring keys (:expire_secs=n) and
specifying the database number (:db=n)
- auth: Don't crash if master user login is attempted without
any configured master=yes passdbs
- Parsing UTF-8 text for mails could have caused broken results
sometimes if buffering was split in the middle of a UTF-8 character.
This affected at least searching messages.
- String sanitization for some logged output wasn't done properly:
UTF-8 text could have been truncated wrongly or the truncation may
not have happened at all.
- fts-lucene: Lookups from virtual mailbox consisting of over 32
physical mailboxes could have caused crashes.- Timo Sirainen <dovecot@dovecot.org>:
- LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user.
That's broken! 0 doesn't remove the limit :(
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
On 06 Mar 2015, at 18:47, Ralf Hildebrandt <r@sys4.de> wrote:
- Timo Sirainen <dovecot@dovecot.org>:
- LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user.
That's broken! 0 doesn't remove the limit :(
Should have been fixed in the rc1 already: http://hg.dovecot.org/dovecot-2.2/rev/9c69c197b527
On 3/6/2015 7:53 AM, Timo Sirainen wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig
Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16.
Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements.
Great to hear both Dovecot and you are doing well. I do need to ask you to check the list for two threads:
mdbox attachment errors Rebuilding SIS attachment links from log
A few of us have been having SIS problems.
-- Daniel
On 06 Mar 2015, at 21:58, Daniel Miller <dmiller@amfes.com> wrote:
On 3/6/2015 7:53 AM, Timo Sirainen wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig
Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16.
Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements.
Great to hear both Dovecot and you are doing well. I do need to ask you to check the list for two threads:
mdbox attachment errors Rebuilding SIS attachment links from log
A few of us have been having SIS problems.
Unless there's a way to reproduce a bug I don't think I can do anything about it (I could spend hours looking at the code or trying to reproduce it and come up with nothing). But a while ago I did think about a SIS redesign that would make it much less likely to break - just need to get it actually implemented:
Currently single instance storage works by having one global directory that contains all the attachments. They are hashed by the attachment content, so for example /var/attachments/ac/7d/ac7d1274891248912489124 would be the attachment. Then each instance would have its own hard link to it, e.g. /var/attachments/ac/7d/hashes/ac7d1274891248912489124-1234567890. sdbox and mdbox can use these by containing the "ac7d1274891248912489124-1234567890" in the header metadata. When mail is deleted, the hard link is deleted. If the link count had been 2, the original attachment file was deleted also. (There's of course some race conditions here, but in those rare situations the attachment would just be duplicated, which isn't too bad.)
The main problem with the old design is that all the users' attachments are dumped into a single global directory. It's difficult to take backups and in general it seems too difficult to manage correctly so I haven't really recommended using it in any bigger installations.
So here's the new idea, which is nearly the same as the old, but with a small change that makes it much nicer I think:
Instead of storing the attachment hard links to a global dir, store the hard links under the user's mail dir. This way taking backups doesn't require anything complicated, just tar the user's mail dir. You can rm -rf the user without forever leaving the user's attachments lying around in the global dir (assuming there's a job that periodically cleans out attachments with link count=1). In general there's no easy way to accidentally break things.
The only new complication here is that if users are split to multiple filesystems, hard linking across them isn't going to work. So this would then require not only having a per-user mail directory but also per-user attachment directory (which would actually be the per-filesystem attachment dir).
The SIS is implemented as lib-fs backend wrapper, so a new one could be implemented easily without breaking the old one.
Instead of deleting it, it should be marked "deleted" by renaming only. If it is then required, so it can be recovered. This directory can be deleted by another control process that searches the mbox files for references.
What has become of the question that attachments are not only saved as separate files? (mdbox-files over 2 MB from 17.03.2014)
Am 07.03.2015 um 17:33 schrieb Timo Sirainen:
... When mail is deleted, the hard link is deleted. If the link count had been 2, the original attachment file was deleted also.
This easy to correct errors is unfortunately still exists. I once hung my solution, but I can only test on a system.
# dovecot --version 2.2.16.rc1 # doveadm -f table mailbox status "messages guid" -u flor_hardy "*" mailbox messages guid Trash 0 60a619171f66fd54035800004b126f5b Entwürfe 0 62a619171f66fd54035800004b126f5b INBOX 0 61a619171f66fd54035800004b126f5b
On 09 Mar 2015, at 13:07, Hardy Flor <HFlor@gmx.de> wrote:
This easy to correct errors is unfortunately still exists. I once hung my solution, but I can only test on a system.
# dovecot --version 2.2.16.rc1 # doveadm -f table mailbox status "messages guid" -u flor_hardy "*" mailbox messages guid Trash 0 60a619171f66fd54035800004b126f5b Entwürfe 0 62a619171f66fd54035800004b126f5b INBOX 0 61a619171f66fd54035800004b126f5b
<dovecot-2.2.16rc1-table.diff><Nachricht als Anhang.eml>
Thanks, committed with some coding style changes: http://hg.dovecot.org/dovecot-2.2/rev/8780ca0fbf22 <http://hg.dovecot.org/dovecot-2.2/rev/8780ca0fbf22>
participants (4)
-
Daniel Miller
-
Hardy Flor
-
Ralf Hildebrandt
-
Timo Sirainen