- dovecot - dovecot.org

MDBOX DSYNC error: Broken physical size in mailbox
by paul-dovecotlist＠trainor.nz 17 Aug '20

17 Aug '20

Hi All, I currently have a small Dovecot server (Server A) and I've installed a second dovecot mailbox server (server B) and am trying to sync mailboxes from Server A to Server B. Both servers sit behind two dovecot directors on separate hosts and have identical config files (doveconf -n output below). Mailboxes on Server A are on an NFS mount. Server A has been working fine with no issues for a couple of years. I'm running this command on Server A: doveadm sync -u #user# tcp:#newservername#:24245 Small mailboxes with few items are syncing OK.. but most mailboxes fail (sycn stops) with the following type of error: (Mailbox name, storage file, UID and sizes vary). dsync-local(#username#)<b2vTDKerOV/fAQAAZMK/0g>: Error: Mailbox INBOX: Cache /#usershomedirectory#/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.cache: Deleting corrupted cache record uid=9: UID 9: Broken physical size in mailbox INBOX: read(/#usershomedirectory#/mdbox/storage/m.1) failed: Cached message size larger than expected (60616 > 8688, box=INBOX, UID=9) If I run it again, I'll get the same error, but with a new UID. If I do a local migration instead of server to server, I still get the errors. e.g. doveadm sync -u #user# maildir:~/Maildir-test/ I tried running a force resync (doveadm force-resync -u #user# -f "*) but it didn't help. There's no issues from an imap client accessing any emails. Using imapsync to sync affected mailboxes works fine for example. But... if I then run a dsync on Server B (doveadm sync -u #user# maildir:~/Maildir-test/) on an account that was successfully migrated with imapsync, I get the issue again. How worried should I be about these corrupted cache records, and any ideas how to resolve? Thanks in advance for your help! Paul. --- output of doveconf -n --- # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.11 (6c69c917) # OS: Linux 5.3.0-1032-aws x86_64 Ubuntu 18.04.5 LTS hostname = ### lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_trusted_networks = ### mail_fsync = always mail_location = mdbox:~/mdbox mail_plugins = " acl notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } namespace shared { list = children location = mdbox:/mailhomes/%%d/%%n/mdbox:INDEXPVT=/mailhomes/%d/%n/shared prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/tmp/global-acls:cache_secs=5 mail_plugins = " acl notify replication" sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /mailhomes/sieve/global/SpamToJunk.sieve sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M sieve_quota_max_scripts = 20 sieve_quota_max_storage = 10M sieve_redirect_envelope_from = sender zlib_save = bz2 zlib_save_level = 6 } protocols = " imap lmtp sieve submission sieve" recipient_delimiter = - service doveadm { inet_listener { port = 24245 } } service imap-login { process_min_avail = 2 } service imap { process_limit = 4096 } service lmtp { inet_listener lmtp { port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 2 service_count = 1 } service managesieve { process_limit = 1024 } service submission-login { inet_listener submission { port = 587 } } ssl = required ssl_cert = </etc/dovecot/dovecot-backend.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it submission_client_workarounds = whitespace-before-path submission_host = ### submission_relay_host = ### submission_relay_ssl = starttls submission_relay_trusted = yes userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } protocol lda { mail_plugins = " acl notify replication zlib" } protocol imap { mail_max_userip_connections = 100 mail_plugins = " acl notify replication zlib imap_acl" } protocol lmtp { mail_plugins = " acl notify replication zlib sieve" } protocol sieve { mail_max_userip_connections = 10 managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 64 k } protocol submission { mail_plugins = " acl notify replication" } local ### { doveadm_password = # hidden, use -P to show it } local ### { doveadm_password = # hidden, use -P to show it }

2 1

Apple Mail Since upgrade to dovecot 2.3.x unable to connect
by Johannes Rohr 17 Aug '20

17 Aug '20

|Dear all,| |a couple of days ago I upgraded our server from Ubuntu 18.04 to 20.04, thereby upgrading dovecot from 2.2.x to 2.3.x. | |Since then, some older versions of apple's mail.app (bundled with el Capitano, released in 2016) no longer connect. When I turn on SSL debugging, I see:| |Debug: SSL error: SSL_accept() failed: error:14209102:SSL routines:tls_early_post_process_client_hello:unsupported protocol imap-login: Debug: SSL error: SSL_accept() syscall failed: Invalid argument| || |Unfortunately, it doesn't reveal the name of the unsupported protocol. Also, what about the failed syscall? Does dovecot try and fail to open some file?| |Here are the contents of /etc/dovecot/conf.d/10-ssl.conf:| | ssl = yes ssl_cert = </etc/ssl/letsencrypt/idaweb-mail.rooot.de/fullchain.pem ssl_key = </etc/ssl/letsencrypt/idaweb-mail.rooot.de/key.pem ssl_ca = </etc/ssl/letsencrypt/idaweb-mail.rooot.de/ca.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = </etc/dovecot/dh.pem | |I would greatly appreciate any hints! | |Cheers,| |Johannes | | | ||

2 1

generating secure ARGON2ID passwords
by Felix Zielcke 16 Aug '20

16 Aug '20

Hello, I'm using dovecot 2.3.4.1 on a Debian buster system. with Argon2ID password scheme, it's only possible to configure the parameters used for it via the generic -r option to "doveadm pw". A higher -r will increase time and memory comsumption. But on my dual core VM it always uses p=1. So it seems the needed threads can only be changed on compile time? Will there be in future a way to explicitly specify memory, time and threads needed for Argon2ID?

2 2

Bug: GSSAPI: Unexpected NUL in auth data
by Paul Banks 16 Aug '20

16 Aug '20

Hi, After upgrading server to v2.3.11.3 (Arch linux). I can no longer connect from Evolution client to the server via GSSAPI (Kerberos). I get the following message in the log: auth: Debug: gssapi(?,x.x.x.x,<XXX>): Unexpected NUL in auth data The bug appears to have been introduced by this patch: https://github.com/dovecot/core/commit/d196f4c83c7383c17440d39eb994b4cd9ff5… Should mech_gssapi also have the MECH_SEC_ALLOW_NULS flag set? Kind regards, Paul

2 2

2.3.11.3: dsync failure
by Alexander Tsoy 15 Aug '20

15 Aug '20

Hello! dovecot 2.3.11.3 fails to backup/sync my mailbox. The mailbox is currently in mdbox format and is (mostly) lz4-compressed. See the trace below (I've added newlines to the output): # doveadm -v backup -u username mdbox:/var/vmail/username_new/mdbox dsync(username): Panic: file istream.c: line 332 (i_stream_read_memarea): assertion failed: (!stream->blocking) dsync(username): Error: Raw backtrace: #0 default_fatal_finish[0x7f1179173b29] -> #1 fatal_handler_real[0x7f1179173be0] -> #2 default_fatal_handler[0x7f1179173c3d] -> #3 i_panic[0x7f1179173e27] -> #4 i_stream_read_memarea[0x7f1179187b29] -> #5 read_more[0x7f117919519b] -> #6 read_from_buffer[0x7f117919535a] -> #7 i_stream_seekable_read[0x7f117919582e] -> #8 i_stream_seekable_stat[0x7f1179195cdf] -> #9 i_stream_stat[0x7f1179188764] -> #10 i_stream_limit_stat[0x7f1179192380] -> #11 i_stream_stat[0x7f1179188764] -> #12 i_stream_default_stat[0x7f117918a0a3] -> #13 i_stream_default_get_size[0x7f117918a282] -> #14 i_stream_get_size[0x7f1179188811] -> #15 mdbox_save_begin[0x7f11792edb85] -> #16 zlib_mail_save_compress_begin[0x7f117891b2ad] -> #17 mailbox_save_begin[0x7f11792c8f22] -> #18 mail_storage_try_copy[0x7f11792b2443] -> #19 mail_storage_copy[0x7f11792b2604] -> #20 mdbox_copy[0x7f11792ee96f] -> #21 mailbox_copy_int[0x7f11792c9611] -> #22 mailbox_copy[0x7f11792c98c2] -> #23 dsync_mailbox_save_body[0x5629fd7936a5] -> #24 dsync_mailbox_save_newmails[0x5629fd793dc3] -> #25 dsync_mailbox_import_mail[0x5629fd793eb0] -> #26 dsync_brain_recv_mail[0x5629fd78b982] -> #27 dsync_brain_sync_mails[0x5629fd78bd79] -> #28 dsync_brain_run_real[0x5629fd78576c] -> #29 dsync_brain_run[0x5629fd7859a5] -> #30 cmd_dsync_run_local[0x5629fd75e8d3] -> #31 cmd_dsync_run[0x5629fd75f3a8] -> #32 doveadm_mail_next_user[0x5629fd762f81] -> #33 doveadm_mail_single_user[0x5629fd763311] -> #34 doveadm_mail_cmd_exec[0x5629fd763963] -> #35 doveadm_mail_cmd[0x5629fd763c22] -> #36 doveadm_mail_try_run[0x5629fd7641e1] -> #37 main[0x5629fd78051d] -> #38 __libc_start_main[0x7f1178d56c00] -> #39 _start[0x5629fd74cb20] Aborted Tried different output compression types: gz, xz, zstd. dovecot 2.3.10.1 doesn't seem to have such problem. But since I already have some messages compressed with zstd, I would like to do migration using 2.3.11.x. Please let me know if I can provide any further information. I'm not on the list, so please CC me on reply. $ sudo dovecot -n # 2.3.11.3 (502c39af9): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.11 (d71e0372) # OS: Linux 5.4.58-gentoo x86_64 Gentoo Base System release 2.6 # Hostname: xxxxxxxx.xx auth_mechanisms = plain login default_client_limit = 50 default_process_limit = 10 imap_idle_notify_interval = 29 mins mail_fsync = always mail_location = mdbox:~/mdbox mail_plugins = " zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mdbox_preallocate_space = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / subscriptions = yes } passdb { args = scheme=CRYPT username_format=%n /etc/dovecot/users driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = xz zlib_save_level = 9 } protocols = imap lmtp service auth-worker { user = $default_internal_user } service auth { unix_listener auth-client { mode = 0660 user = mail } user = $default_internal_user } service imap-login { inet_listener imaps { port = 0 } } ssl = required ssl_cert = </etc/ssl/dovecot/server.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = username_format=%n /etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = " zlib sieve" } protocol lda { mail_plugins = " zlib sieve" } local_name imap.xxxx.xx { ssl_cert = </etc/ssl/dovecot/imap.xxxx.xx.crt ssl_key = # hidden, use -P to show it } local_name imap.xxxxxxxx.xx { ssl_cert = </etc/ssl/dovecot/imap.xxxxxxxx.xx.crt ssl_key = # hidden, use -P to show it }

1 0

Doveadm error since 2.3.11.3 when run as unprivileged user
by Arjen de Korte 15 Aug '20

15 Aug '20

I allow users to run 'doveadm' for mailbox maintenance (to expunge mail for instance). Since the upgrade to 2.3.11.3, this no longer works and results in the following error message: doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-ssl.conf line 13: ssl_key: Can't open file /etc/ssl/private/de-korte.org.key: Permission denied This is no surprise, as non-privileged users are not allowed to read the private keys of the server. Question is, why is doveadm trying to read this key in the first place (it is not needed for mailbox maintenance) and why is it failing now? Regards, Arjen

4 6

User doesn't exist
by Bruno Kiste 15 Aug '20

15 Aug '20

I try to use dovecot as a MTA with postfix. But it does not accept incoming mails. Why does it not work? [root@localhost bruno]# dovecot --version 2.3.8 (9df20d2db) /var/log/maillog Aug 14 11:03:32 localhost dovecot[566097]: lmtp(566373): Connect from local Aug 14 11:03:32 localhost dovecot[566097]: lmtp(bruno(a)2c2axxx.online-server.cloud)<566373><0O+4HWRTNl9lpAgA0J78UA>: Debug: auth-master: userdb lookup(bruno(a)2c2axxx.online-server.cloud): Started userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: lmtp(bruno(a)2c2axxx.online-server.cloud)<566373><0O+4HWRTNl9lpAgA0J78UA>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Connecting Aug 14 11:03:32 localhost dovecot[566097]: lmtp(bruno(a)2c2axxx.online-server.cloud)<566373><0O+4HWRTNl9lpAgA0J78UA>: Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Client connected (fd=16) Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: master in: USER#0111#011bruno(a)2c2axxx.online-server.cloud#011service=lmtp Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: passwd(bruno(a)2c2axxx.online-server.cloud): Performing userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: Loading modules from directory: /usr/lib64/dovecot/auth Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: passwd-file /etc/dovecot/users: Read 4 users in 0 secs Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: conn unix:auth-worker (pid=566367,uid=97): Server accepted connection (fd=14) Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: conn unix:auth-worker (pid=566367,uid=97): Sending version handshake Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: conn unix:auth-worker (pid=566367,uid=97): auth-worker<1>: Handling USER request Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: passwd(bruno(a)2c2axxx.online-server.cloud): Performing userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: passwd(bruno(a)2c2axxx.online-server.cloud): lookup Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): passwd(bruno(a)2c2axxx.online-server.cloud): unknown user - trying the next userdb Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: passwd(bruno(a)2c2axxx.online-server.cloud): Finished userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: auth-worker(566374): Debug: conn unix:auth-worker (pid=566367,uid=97): auth-worker<1>: Finished Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: passwd(bruno(a)2c2axxx.online-server.cloud): Finished userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: passwd-file(bruno(a)2c2axxx.online-server.cloud): Performing userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: passwd-file(bruno(a)2c2axxx.online-server.cloud): lookup: user=bruno(a)2c2axxx.online-server.cloud file=/etc/dovecot/users Aug 14 11:03:32 localhost dovecot[566097]: auth: passwd-file(bruno(a)2c2axxx.online-server.cloud): unknown user Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: passwd-file(bruno(a)2c2axxx.online-server.cloud): Finished userdb lookup Aug 14 11:03:32 localhost dovecot[566097]: auth: Debug: userdb out: NOTFOUND#0111 Aug 14 11:03:32 localhost dovecot[566097]: lmtp(bruno(a)2c2axxx.online-server.cloud)<566373><0O+4HWRTNl9lpAgA0J78UA>: Debug: auth-master: userdb lookup(bruno(a)2c2axxx.online-server.cloud): auth USER input: Aug 14 11:03:32 localhost dovecot[566097]: lmtp(bruno(a)2c2axxx.online-server.cloud)<566373><0O+4HWRTNl9lpAgA0J78UA>: Debug: auth-master: userdb lookup(bruno(a)2c2axxx.online-server.cloud): Userdb lookup failed Aug 14 11:03:32 localhost postfix/lmtp[566372]: 4E2127802973: to=<bruno(a)2c2axxx.online-server.cloud>, relay=2c2axxx.online-server.cloud[private/dovecot-lmtp], delay=0.26, delays=0.19/0.02/0.02/0.02, dsn=5.1.1, status=bounced (host 2c2axxx.online-server.cloud[private/dovecot-lmtp] said: 550 5.1.1 <bruno(a)2c2axxx.online-server.cloud> User doesn't exist: bruno(a)2c2axxx.online-server.cloud (in reply to RCPT TO command)) [root@localhost bruno]# cat /etc/dovecot/users #user:password:uid:gid:(gecos):home:(shell):extra_fields bruno:{CRAM-MD5}88XXXee:5001:5001::/home/bruno:: mail:{CRAM-MD5}88XXXee:5001:5001::/home/mail:: nas:{CRAM-MD5}8eXXX40:5001:5001::/home/nas asys:{CRAM-MD5}99XXX09:5001:5001::/home/asys

3 3

CentOS 8 packages available
by Aki Tuomi 14 Aug '20

14 Aug '20

We are happy to announce that we have CentOS 8 packages available starting from v2.3.11.3. You can find these packages at https://repo.dovecot.org/ Regards, Aki Tuomi Open-Xchange oy

2 1

2.3.11.3 on 32bit platforms
by Christian Göttsche 14 Aug '20

14 Aug '20

See also the 32bit build failures on Debian: https://buildd.debian.org/status/package.php?p=dovecot There are some compiler warnings on 32bit architectures in test-mech.c test-mech.c: In function ‘test_mechs’: test-mech.c:326:61: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 4 has type ‘unsigned int’ [-Wformat=] 326 | const char *testname = t_strdup_printf("auth mech %s %d/%lu", | ~~^ | | | long unsigned int | %u test-mech.c:338:12: warning: passing argument 2 of ‘test_mech_construct_apop_challenge’ from incompatible pointer type [-Wincompatible-pointer-types] 338 | &test_case->len); | ^~~~~~~~~~~~~~~ | | | size_t * {aka unsigned int *} test-mech.c:195:77: note: expected ‘long unsigned int *’ but argument is of type ‘size_t *’ {aka ‘unsigned int *’} 195 | test_mech_construct_apop_challenge(unsigned int connect_uid, unsigned long *len_r) | ~~~~~~~~~~~~~~~^~~~~ But the unit test still fails after a commit like: diff --git a/src/auth/test-mech.c b/src/auth/test-mech.c index cf05370..90c2215 100644 --- a/src/auth/test-mech.c +++ b/src/auth/test-mech.c @@ -192,7 +192,7 @@ static void test_mech_handle_challenge(struct auth_request *request, } static inline const unsigned char * -test_mech_construct_apop_challenge(unsigned int connect_uid, unsigned long *len_r) +test_mech_construct_apop_challenge(unsigned int connect_uid, size_t *len_r) { string_t *apop_challenge = t_str_new(128); @@ -323,7 +323,7 @@ static void test_mechs(void) struct test_case *test_case = &tests[running_test]; const struct mech_module *mech = test_case->mech; struct auth_request *request; - const char *testname = t_strdup_printf("auth mech %s %d/%lu", + const char *testname = t_strdup_printf("auth mech %s %d/%zu", mech->mech_name, running_test+1, N_ELEMENTS(tests));

1 0

About the changing namespace handling from v2.0 to v2.1 or later
by mizuta.takeshi＠fujitsu.com 14 Aug '20

14 Aug '20

Hi, all, Please tell me about the patch and description below. https://github.com/dovecot/core/commit/5f46b863383fc94afcd2693ecae8788cb5fd… example-config: Use a default uncommented "namespace inbox". https://doc.dovecot.org/configuration_manual/namespace/ There's a default inbox namespace added in 10-mail.conf. If the configuration doesn't explicitly specify a namespace (as was in v2.0 and older) a default namespace is created automatically. Since v2.1, namespace is enabled by default in 10-mail.conf, and "inbox" is added as a sectionname of namespace. I checked the dovecot's mail archive, but I didn't understand the reason for the change. Please tell me about followings. - Why "inbox" was added as a sectionname? - Why was namespace changed to be enabled by default? Best regards, MIZUTA Takeshi

1 0