http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig
A few fixes and some imapc improvements since the release candidate.
* dbox: Resyncing (e.g. doveadm force-resync) no longer deletes
dovecot.index.cache file. The cache file was rarely the problem
so this just caused unnecessary slowness.
* Mailbox name limits changed during mailbox creation: Each part of
a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255
chars long (instead of 200). This also reduces the max number of
hierarchical levels to 16 (instead of 20) to keep the maximum name
length 4096 (a common PATH_MAX limit). The 255 char limit is
hopefully large enough for migrations from all existing systems.
It's also the limit on many filesystems.
+ director: Added director_consistent_hashing setting to enable
consistent hashing (instead of the mostly-random MD5 hashing).
This causes fewer user moves between backends when backend counts
are changed, which may improve performance (mainly due to caching).
+ director: Added support for "tags", which allows one director ring
to serve multiple backend clusters with different sets of users.
+ LMTP server: Added lmtp_user_concurrency_limit setting to limit how
many LMTP deliveries can be done concurrently for a single user.
+ LMTP server: Added support for STARTTLS command.
+ If logging data is generated faster than it can be written, log a
warning about it and show information about it in log process's
process title in ps output. Also don't allow a single service to
flood too long at the cost of delaying other services' logging.
+ stats: Added support for getting global statistics.
+ stats: Use the same session IDs as the rest of Dovecot.
+ stats: Plugins can now create their own statistics fields
+ doveadm server: Non-mail related commands can now also be used
via doveadm server (TCP socket).
+ doveadm proxying: passdb lookup can now override doveadm_port and
change the username.
+ doveadm: Search query supports now "oldestonly" parameter to stop
immediately on the first non-match. This can be used to optimize:
doveadm expunge mailbox Trash savedbefore 30d oldestonly
+ doveadm: Added "save" command to directly save mails to specified
mailbox (bypassing Sieve).
+ doveadm fetch: Added body.snippet field, which returns the first
100 chars of a message without whitespace or HTML tags. The result
is stored into dovecot.index.cache, so it can be fetched efficiently.
+ dsync: Added -t <timestamp> parameter to sync only mails newer than
the given received-timestamp.
+ dsync: Added -F [-]<flag> parameter to sync only mails with[out] the
given flag/keyword.
+ dsync: Added -a <mailbox> parameter to specify the virtual mailbox
containing user's all mails. If this mailbox is already found to
contain the wanted mail (by its GUID), the message is copied from
there instead of being re-saved. (This isn't efficient enough yet
for incremental replication.)
+ dsync: -m parameter can now specify \Special-use names for mailboxes.
+ imapc: Added imapc_features=gmail-migration to help migrations from
GMail. See http://wiki2.dovecot.org/Migration/Gmail
+ imapc: Added imapc_features=search to support IMAP SEARCH command.
(Currently requires ESEARCH support from remote server.)
+ expire plugin: Added expire_cache=yes setting to cache most of the
database lookups in dovecot index files.
+ quota: If overquota-flag in userdb doesn't match the current quota
usage, execute a configured script.
+ redis dict: Added support for expiring keys (:expire_secs=n) and
specifying the database number (:db=n)
- auth: Don't crash if master user login is attempted without
any configured master=yes passdbs
- Parsing UTF-8 text for mails could have caused broken results
sometimes if buffering was split in the middle of a UTF-8 character.
This affected at least searching messages.
- String sanitization for some logged output wasn't done properly:
UTF-8 text could have been truncated wrongly or the truncation may
not have happened at all.
- fts-lucene: Lookups from virtual mailbox consisting of over 32
physical mailboxes could have caused crashes.
/usr/lib64/dovecot/stats/libstats_mail.so
why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files?
Am 12.03.2015 um 18:30 schrieb Timo Sirainen:
http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.16.tar.gz.sig
A few fixes and some imapc improvements since the release candidate.
- dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness.
- Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems.
- director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching).
- director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users.
- LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user.
- LMTP server: Added support for STARTTLS command.
- If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging.
- stats: Added support for getting global statistics.
- stats: Use the same session IDs as the rest of Dovecot.
- stats: Plugins can now create their own statistics fields
- doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket).
- doveadm proxying: passdb lookup can now override doveadm_port and change the username.
- doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly
- doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve).
- doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently.
- dsync: Added -t <timestamp> parameter to sync only mails newer than the given received-timestamp.
- dsync: Added -F [-]<flag> parameter to sync only mails with[out] the given flag/keyword.
- dsync: Added -a <mailbox> parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.)
- dsync: -m parameter can now specify \Special-use names for mailboxes.
- imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail
- imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.)
- expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files.
- quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script.
- redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n)
- auth: Don't crash if master user login is attempted without any configured master=yes passdbs
- Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages.
- String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all.
- fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes.
On 12 Mar 2015, at 21:09, Reindl Harald h.reindl@thelounge.net wrote:
/usr/lib64/dovecot/stats/libstats_mail.so
why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files?
So that external plugins can add more files in there and extend the available statistics.
Am 13.03.2015 um 11:23 schrieb Timo Sirainen:
On 12 Mar 2015, at 21:09, Reindl Harald h.reindl@thelounge.net wrote:
/usr/lib64/dovecot/stats/libstats_mail.so
why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files?
So that external plugins can add more files in there and extend the available statistics
but they can also go to /usr/lib64/dovecot/ http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath
hence
cat /etc/ld.so.conf.d/dovecot-x86_64.conf /usr/lib64/dovecot
i just "rm -rf" the folder and other stuff for private builds in environments where dovecot is running only as proxy
rm -rf %{buildroot}%{_sysconfdir}/%{name}/README
%{buildroot}%{_docdir}/%{name}-%{version}
%{buildroot}%{_includedir}/%{name}/
%{buildroot}%{_mandir}/man1/
%{buildroot}%{_mandir}/man7/
%{buildroot}%{_datarootdir}/aclocal/
%{buildroot}%{_bindir}/dsync
%{buildroot}%{_libdir}/%{name}/*_plugin.so
%{buildroot}%{_libdir}/%{name}/doveadm/*_plugin.so
%{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so
%{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so.0
%{buildroot}%{_libdir}/%{name}/lib%{name}-lda.so.0.0.0
%{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so
%{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so.0
%{buildroot}%{_libdir}/%{name}/lib%{name}-compression.so.0.0.0
%{buildroot}%{_libdir}/%{name}/stats
%{buildroot}%{_libexecdir}/%{name}/%{name}-lda
%{buildroot}%{_libexecdir}/%{name}/gdbhelper
%{buildroot}%{_libexecdir}/%{name}/quota-status
%{buildroot}%{_libexecdir}/%{name}/deliver
%{buildroot}%{_libexecdir}/%{name}/lmtp
On 13 Mar 2015, at 12:34, Reindl Harald h.reindl@thelounge.net wrote:
Am 13.03.2015 um 11:23 schrieb Timo Sirainen:
On 12 Mar 2015, at 21:09, Reindl Harald h.reindl@thelounge.net wrote:
/usr/lib64/dovecot/stats/libstats_mail.so
why in the world a new sub-directory containing just one so-file enforcing pakcage buildsers to change SPEC files?
So that external plugins can add more files in there and extend the available statistics
but they can also go to /usr/lib64/dovecot/ http://fedoraproject.org/wiki/Packaging:Guidelines#Beware_of_Rpath
These are plugins, not libraries. They don't need rpaths. The reason for the new directory's existence is exactly the same as for the already existing plugin directories:
$lib/dovecot/auth/ $lib/dovecot/doveadm/ $lib/dovecot/settings/ $lib/dovecot/sieve/
So auth, doveadm, settings and sieve all work by loading all the plugins from inside the directories. If I changed the stats to work differently, e.g. load $lib/dovecot/libstats_*.so it would work differently than everything else, which would be confusing. For consistency it would be better then to get rid of the above directories as well. But changing that would break backwards compatibility with external plugins (e.g. with Sieve).
This happened after upgrading to 2.2.16. Running on Debian wheezy amd64. We never had such a crash on this server before. However, this too was a single incident. All accounts use Maildir.
Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/----/----/Maildir/dovecot.index: Append with UID 6684, but next_uid = 6685 Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Error: /stor/mail/domains/----/----/Maildir/dovecot.index view syncing failed to apply changes Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda5ad2e] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda5b344] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda5b67f] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda52910] -> /usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f34cd28e80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f34cd28f88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f34cd28e918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f34cd2307d3] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda46997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f34cce94ead] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda46b09] Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Fatal: master: service(imap): child 30144 killed with signal 6 (core dumps disabled)
# doveconf -n # 2.2.16: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.7 # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_master_user_separator = * auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no login_greeting = Dovecot ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pip=%{real_rip} mpid=%e %c session=<%{session}> mail_gid = vmail mail_location = /nowhere mail_plugins = quota expire listescape mail_uid = vmail managesieve_sieve_capability = fileinto envelope encoded-character subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = pipe antispam_pipe_program = /usr/bin/ssh antispam_pipe_program_args = -l;spamd;-i;/etc/dovecot/sareport.key;mx1.xxxxx antispam_pipe_program_notspam_arg = revoke antispam_pipe_program_spam_arg = report antispam_pipe_tmpdir = /tmp antispam_spam = Junk antispam_trash_pattern_ignorecase = trash;Deleted * expire = Trash expire2 = Trash/* expire3 = Junk expire_dict = proxy::expire quota = dict:user::proxy::quota quota_rule = *:storage=2G quota_rule2 = Trash:storage=+250M quota_rule3 = Junk:ignore quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve-global/fileinto-spam.sieve sieve_dir = ~/sieve sieve_extensions = -vacation -body -reject -enotify -environment -virustest -spamtest sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 10K sieve_quota_max_scripts = 100 sieve_quota_max_storage = 4M } pop3_no_flag_updates = yes protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 2448 unix_listener auth-userdb { group = root mode = 0600 user = vmail } } service dict { unix_listener dict { group = root mode = 0600 user = vmail } } service imap-login { process_min_avail = 4 service_count = 0 vsz_limit = 192 M } service imap-postlogin { executable = script-login /usr/local/bin/postlogin unix_listener imap-postlogin { group = root mode = 0600 user = vmail } user = root } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 128 M } service lmtp { inet_listener lmtp { address = 10.44.23.1 port = 7025 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 vsz_limit = 64 M } service managesieve { process_limit = 20 } service pop3-login { process_min_avail = 4 service_count = 0 vsz_limit = 192 M } service pop3 { executable = pop3 imap-postlogin process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = dovecot } ssl_cert =
On 03/23/2015 12:10 AM, Gedalya wrote:
This happened after upgrading to 2.2.16. Running on Debian wheezy amd64. We never had such a crash on this server before. However, this too was a single incident. All accounts use Maildir.
Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/----/----/Maildir/dovecot.index: Append with UID 6684, but next_uid = 6685 Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Error: /stor/mail/domains/----/----/Maildir/dovecot.index view syncing failed to apply changes Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f34cd27c57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f34cd27c5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f34cd22b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f34cd5a6391] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda5ad2e] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda5b344] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda5b67f] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda52910] -> /usr/lib/dovecot/libdovecot.so.0(+0x98a85) [0x7f34cd299a85] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f34cd28e80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f34cd28f88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f34cd28e899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f34cd28e918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f34cd2307d3] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda46997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f34cce94ead] -> dovecot/imap ----@---.com xx.xx.xx.xx UID FETCH - 2373 bytes waiting [0x7f34cda46b09] Mar 21 09:32:02 imap1 dovecot: imap(----@---.com): Fatal: master: service(imap): child 30144 killed with signal 6 (core dumps disabled)
And now it just happened, again, same user!
The previous dovecot version before the upgrade was 2.2.12
Mar 28 07:51:17 imap1 dovecot: imap(----@----.com): Error: Log synchronization error at seq=0,offset=0 for /stor/mail/domains/----.com/----/Maildir/dovecot.index: Append with UID 12794, but next_uid = 12795 Mar 28 07:51:17 imap1 dovecot: imap(----@----.com): Error: /stor/mail/domains/----.com/----/Maildir/dovecot.index view syncing failed to apply changes Mar 28 07:51:17 imap1 dovecot: imap(----@----.com): Panic: file index-sync.c: line 265 (index_mailbox_sync_next_expunge): assertion failed: (range->seq2 <= ctx->messages_count) Mar 28 07:51:17 imap1 dovecot: imap(----@----.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b57f) [0x7f136acdc57f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b5de) [0x7f136acdc5de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f136ac8b0d5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb5391) [0x7f136b006391] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4bad2e] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4bb344] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4bb67f] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4b2725] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4b2a35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f136acee80b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7f136acef88b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f136acee899] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f136acee918] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f136ac907d3] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4a6997] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f136a8f4ead] -> dovecot/imap ----@----.com xx.xx.xx.xx UID FETCH [0x7f136b4a6b09] Mar 28 07:51:17 imap1 dovecot: imap(----@----.com): Fatal: master: service(imap): child 12444 killed with signal 6 (core dumps disabled)
participants (4)
-
Gedalya
-
Hardy Flor
-
Reindl Harald
-
Timo Sirainen