- dovecot - dovecot.org

doveadm backup|sync works for every folder but INBOX
by Ralf Becker 09 Nov '22

09 Nov '22

I'm trying to migrate an old Cyrus 2.5 server to Dovecot 2.3.19 using doveadm backup -R, which works for all folders but the INBOX itself, which always stays empty. The Cyrus side uses altnamespace:no and unixhierarchysep:no, it's used as imapc: remote in doveadm backup -R with imapc_list_prefix=INBOX Dovecot uses the following namespace to migrate into: namespace inboxes { inbox = yes location = mailbox Sent { auto = subscribe special_use = \Sent } ### some more folders omitted ### prefix = INBOX/ separator = / subscriptions = no } I use the following command line for the migration: doveadm -o imapc_features='rfc822.size fetch-headers' -o mail_prefetch_count=20 -o namespace/subs/location=mbox:/var/dovecot/subs -o imapc_user='someuser' -o imapc_password='secret' -o imapc_host=<cyrus-ip> -o imapc_list_prefix=INBOX -D backup -f -n INBOX/ -R -u someuser@somedomain imapc: The debug output contains the following lines: Nov 02 10:16:01 dsync(someuser@somedomain): Debug: brain S: Namespace INBOX/ has location imapc: Nov 02 10:16:01 dsync(someuser@somedomain): Debug: brain M: Local mailbox tree: INBOX guid=d8d5a3122bce5f63ef9101001ebfc848 uid_validity=1667223083 uid_next=1 subs=no last_change=0 last_subs=0 Nov 02 10:16:01 dsync(someuser@somedomain): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32691 subs=no last_change=0 last_subs=0 Nov 02 10:16:28 dsync(someuser@somedomain): Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?) The Cyrus server is solely used for the migration, no access from anywhere else! doveadm mailbox status -u someuser@somedomain all INBOX reports: INBOX messages=0 recent=0 uidnext=1 uidvalidity=1667223083 unseen=0 highestmodseq=1 vsize=0 guid=d8d5a3122bce5f63ef9101001ebfc848 firstsaved=never As all other folders seem ok, I blamed the failure with the INBOX on the imapc_list_prefix and altnamespace:no and tried, adding the INBOX content with an other doveadm sync run: doveadm -o imapc_features='rfc822.size fetch-headers' -o mail_prefetch_count=20 -o namespace/subs/location=mbox:/var/dovecot/subs -o imapc_user='someuser' -o imapc_password='secret' -o imapc_host=<cyrus-ip> -D sync -1 -f -m INBOX -n INBOX/ -R -u someuser@somedomain imapc: That gives the following output: Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: Namespace INBOX/ has location imapc: Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain M: Local mailbox tree: INBOX guid=d8d5a3122bce5f63ef9101001ebfc848 uid_validity=1667223083 uid_next=1 subs=no last_change=0 last_subs=0 Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: Local mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32691 subs=no last_change=0 last_subs=0 Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain M: Remote mailbox tree: INBOX guid=c92f64f79f0d1ed01e6d5b314f04886c uid_validity=1577952633 uid_next=32691 subs=no last_change=0 last_subs=0 Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: Remote mailbox tree: INBOX guid=d8d5a3122bce5f63ef9101001ebfc848 uid_validity=1667223083 uid_next=1 subs=no last_change=0 last_subs=0 Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain M: Mailbox INBOX: local=d8d5a3122bce5f63ef9101001ebfc848/0/1, remote=c92f64f79f0d1ed01e6d5b314f04886c/0/1: GUIDs conflict - will be merged later Nov 02 10:55:45 dsync(someuser@somedomain): Debug: Mailbox INBOX: Mailbox opened Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain M: Changing mailbox INBOX GUID d8d5a3122bce5f63ef9101001ebfc848 -> c92f64f79f0d1ed01e6d5b314f04886c Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): Waiting for dict to finish pending operations Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: Remote mailbox tree deletion: guid=c8882c0029ce5f63a9aa0000ca9feb0b type=mailbox timestamp=1667223081 name= local update=not found Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: Mailbox INBOX: local=c92f64f79f0d1ed01e6d5b314f04886c/0/1, remote=d8d5a3122bce5f63ef9101001ebfc848/0/1: GUIDs conflict - will be merged later Nov 02 10:55:45 dsync(someuser@somedomain): Debug: Mailbox INBOX: Mailbox opened Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: Ignore nonexistent mailbox GUID c92f64f79f0d1ed01e6d5b314f04886c with -1 sync Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain S: We don't have mailbox c92f64f79f0d1ed01e6d5b314f04886c Nov 02 10:55:45 dsync(someuser@somedomain): Debug: brain M: Ignoring missing remote box GUID c92f64f79f0d1ed01e6d5b314f04886c Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): Waiting for dict to finish pending operations Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): dict destroyed Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): Waiting for dict to finish pending operations Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): dict destroyed Nov 02 10:55:45 dsync(someuser@somedomain): Debug: imapc(10.44.88.3:143): Disconnected Nov 02 10:55:45 dsync(someuser@somedomain): Debug: User session is finished Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): dict destroyed Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): Waiting for dict to finish pending operations Nov 02 10:55:45 dsync(someuser@somedomain): Debug: dict(file): dict destroyed Nov 02 10:55:45 doveadm(40708): Debug: auth-master: conn unix:/run/dovecot/auth-userdb (pid=1,uid=0): Disconnected: Connection closed (fd=9) doveadm mailbox status -u someuser@somedomain all INBOX INBOX messages=0 recent=0 uidnext=1 uidvalidity=1667223083 unseen=0 highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c firstsaved=never The interesting part is that INBOX somehow seems to have two different GUIDs, depending on how I sync it (compare the two doveadm mailbox status outputs). In a tcpdump of the IMAP trafic I can see a STATUS "INBOX" (UIDNEXT UIDVALIDITY) but no EXAMINE "INBOX" or something else I would expect from the sync. I can provide the full output of doveadm backup/sync and a pcap of the IMAP trafic, but not to the public list. Any idea what's wrong with the sync of the INBOX or other suggestions / imapc parameters to use? Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0

3 17

"Mailbox isn't a valid MBOX file" Error
by Elise 06 Nov '22

06 Nov '22

Running currently two local mailclients: Outlook 2021 and Thunderbird 102.4.2 When using TB and remove a message on my main account, I get this error: [CANNOT] Mailbox isn't a valid mailbox On any other TB configured email accounts I can remove messages without any problems. First I thought that Outlook might block removal on messages, but the issue appears as well if I close that mail client. Can you tell what is causing this problem? Best regards, Jos Chrispijn

5 4

Any update on lazy load SNI?
by Pierre Allétru 03 Nov '22

03 Nov '22

Hi folks, We need to use SNI with Dovecot at a relatively large scale and I was wondering if there's any update on the ability to: 1 - Lazy load SNI certificates when they are needed instead of loading them all at once during startup/reload, thus taking a lot of memory and being very slow. 2 - Not having to reload Dovecot every time a new domain cert is added in conf. 3 - Or at least have Dovecot keep processing clients while the slow reload happens. This has been asked already in a 2016 thread: https://www.dovecot.org/pipermail/dovecot/2016-November/106075.html. Regarding point 3 there's a reply from Aki saying it was in the internal tasklist ( https://www.dovecot.org/pipermail/dovecot/2016-November/106089.html) Anybody knows if some progress has been made on these subjects? I can't anything in the docs or any recent information anywhere else. Thanks, -- Pierre Allétru 06 70 55 08 35 pierre.alletru(a)gmail.com

2 2

how to limit a global disk quota
by linuxguy＠gmx.net 03 Nov '22

03 Nov '22

1 0

The end of Dovecot Director?
by Steff Majeur 02 Nov '22

02 Nov '22

I recently stumbled upon the following commit on the Dovecot core Github repository: https://github.com/dovecot/core/commit/4a187116dc2311804be22724007d35732300… Apparently, Dovecot Director is going to be removed in the next major version of Dovecot and the commercial Dovecot cluster architecture will be its successor: https://github.com/dovecot/documentation/blob/a85b742ec4fc2744db30a6943b3c2… This would be a huge blow for many organizations around the world that are currently using Dovecot with Director in a shared storage environment. Can anyone of the Dovecot developers maybe enlighten us about the future of Dovecot? - Will there still be the Director feature in the next community release of Dovecot? - If not, will there be a community feature that is on par with the current Director feature? - For how long will Dovecot version 2.3 still be supported (security fixes, bug fixes)? Is there any EOL plan? Thanks for any clarification! Steff

31 56

Dovecot does not start on MacOS Monterey
by Doro Rose 30 Oct '22

30 Oct '22

Hi, this is a duplicate of the "Dovecot does not start on MacOS Monterey 12.01" thread. I’m still seeing this on dovecot 2.3.19.1. Is there a recommended workaround? The last post in that thread indicated that some modifications in `./src/lib/restrict-process-size.c` might be necessary. Best regards Doro

3 6

how to configure imapsieve to be used per user
by Marc 29 Oct '22

29 Oct '22

I only see configurations that are active for all users, how to configure this in the user sieve rules. I only need this for specific users.

7 11

oauth2 + postgres = Panic: file http-client.c: line 646 (http_client_context_close)
by Sebastiano Degan 27 Oct '22

27 Oct '22

I could to reply to the original thread: https://www.mail-archive.com/dovecot@dovecot.org/msg85204.html But I managed to get a stack trace of the error: root@mail:~ # lldb /usr/local/libexec/dovecot/auth (lldb) target create "/usr/local/libexec/dovecot/auth" Current executable set to '/usr/local/libexec/dovecot/auth' (x86_64). (lldb) run Process 54354 launched: '/usr/local/libexec/dovecot/auth' (x86_64) Error: stats: open(old-stats-user) failed: No such file or directory Panic: file http-client.c: line 646 (http_client_context_close): assertion failed: (cctx->clients_list == NULL) Process 54354 stopped * thread #1, name = 'auth', stop reason = signal SIGABRT frame #0: 0x000000080159f33a libc.so.7`__sys_thr_kill + 10 libc.so.7`__sys_thr_kill: -> 0x80159f33a <+10>: jb 0x80159d498 0x80159f340 <+16>: retq 0x80159f341: int3 0x80159f342: int3 (lldb) bt all * thread #1, name = 'auth', stop reason = signal SIGABRT * frame #0: 0x000000080159f33a libc.so.7`__sys_thr_kill + 10 frame #1: 0x0000000801517c74 libc.so.7`__raise + 52 frame #2: 0x00000008015c9109 libc.so.7`abort + 73 frame #3: 0x00000008012e606b libdovecot.so.0`default_fatal_finish(type=LOG_TYPE_PANIC, status=0) at failures.c:465:3 frame #4: 0x00000008012e3f7c libdovecot.so.0`fatal_handler_real(ctx=0x00007fffffffe610, format="file %s: line %d (%s): assertion failed: (%s)", args=0x00007fffffffe5f0) at failures.c:477:2 frame #5: 0x00000008012e3f14 libdovecot.so.0`default_fatal_handler(ctx=0x00007fffffffe610, format="file %s: line %d (%s): assertion failed: (%s)", args=0x00007fffffffe5f0) at failures.c:485:2 frame #6: 0x00000008012e4253 libdovecot.so.0`i_panic(format="file %s: line %d (%s): assertion failed: (%s)") at failures.c:530:2 frame #7: 0x000000080124f2ed libdovecot.so.0`http_client_context_close(cctx=0x000000080205c048) at http-client.c:646:2 frame #8: 0x000000080124ef22 libdovecot.so.0`http_client_global_context_ioloop_switched(prev_ioloop=0x0000000802030640) at http-client.c:710:3 frame #9: 0x00000008013102e4 libdovecot.so.0`io_loop_set_current(ioloop=0x0000000000000000) at ioloop.c:982:4 frame #10: 0x00000000010aa0be auth`driver_pgsql_set_state(db=0x0000000802092200, state=SQL_DB_STATE_IDLE) at driver-pgsql.c:103:3 frame #11: 0x00000000010a9ff0 auth`connect_callback(db=0x0000000802092200) at driver-pgsql.c:206:3 frame #12: 0x000000080130f3b2 libdovecot.so.0`io_loop_call_io(io=0x00000008020105c0) at ioloop.c:737:2 frame #13: 0x0000000801313e0b libdovecot.so.0`io_loop_handler_run_internal(ioloop=0x0000000802030640) at ioloop-kqueue.c:164:4 frame #14: 0x000000080130fa15 libdovecot.so.0`io_loop_handler_run(ioloop=0x0000000802030640) at ioloop.c:789:2 frame #15: 0x000000080130f8fa libdovecot.so.0`io_loop_run(ioloop=0x0000000802030640) at ioloop.c:762:3 frame #16: 0x00000000010a7eef auth`driver_pgsql_wait(_db=0x0000000802092200) at driver-pgsql.c:1279:3 frame #17: 0x00000000010a4c59 auth`sql_wait(db=0x0000000802092200) at sql-api.c:820:3 frame #18: 0x00000000010a7fc0 auth`driver_pgsql_get_flags(db=0x0000000802092200) at driver-pgsql.c:306:3 frame #19: 0x00000000010a2fa9 auth`sql_get_flags(db=0x0000000802092200) at sql-api.c:171:10 frame #20: 0x00000000010a585e auth`driver_sqlpool_get_flags(_db=0x0000000802092000) at driver-sqlpool.c:450:9 frame #21: 0x00000000010a2fa9 auth`sql_get_flags(db=0x0000000802092000) at sql-api.c:171:10 frame #22: 0x000000000108c675 auth`passdb_sql_init(_module=0x0000000802041ca8) at passdb-sql.c:281:10 frame #23: 0x000000000107d577 auth`passdb_init(passdb=0x0000000802041ca8) at passdb.c:247:3 frame #24: 0x0000000001052a89 auth`auth_passdb_init(passdb=0x0000000802041ba8) at auth.c:308:2 frame #25: 0x0000000001051dbc auth`auth_init(auth=0x0000000802041b70) at auth.c:322:3 frame #26: 0x0000000001051d21 auth`auths_init at auth.c:427:3 frame #27: 0x0000000001050610 auth`main_init at main.c:242:2 frame #28: 0x00000000010503ad auth`main(argc=1, argv=0x00007fffffffeaf8) at main.c:391:2 frame #29: 0x000000000104ff7d auth`_start(ap=<unavailable>, cleanup=<unavailable>) at crt1_c.c:75:7

1 0

Mailing list IMAP archive.
by dovecot＠ptld.com 27 Oct '22

27 Oct '22

I am trying to use the IMAP mailing list archive described on https://www.dovecot.org/mailing-lists/ Is this feature still functioning? Using Thunderbird i get an error message that the connection was refused. I don't know if Thunderbird isn't functioning properly (with handling no password) or if the service is down. My end goal is i want to search the archive for solutions before asking the list. Is there a browser based option to search the archives? I didn't see a search option on https://dovecot.org/pipermail/dovecot/

3 3

Re: The end of Dovecot Director?
by William Edwards 27 Oct '22

27 Oct '22

> Op 27 okt. 2022 om 04:25 heeft Timo Sirainen <timo(a)sirainen.com> het volgende geschreven: > > Director never worked especially well, and for most use cases it's just unnecessarily complex. I think usually it could be replaced with: > > * Database (sql/ldap/whatever) containing user -> backend table. > * Configure Dovecot proxy to use this database as passdb. > * For HA change dovemon to update the database if backend is down to move users elsewhere > * When backend comes up, move users into it. Set delay_until extra field for user in passdb to 5 seconds into future and kick the user in its old backend (e.g. via doveadm HTTP API). > > All this can be done with existing Dovecot. Should be much easier to build a project doing this than forking director. This is my train of thought as well. I believe the following would suffice for most setups. A database with: - Current vhost count per backend server. Alternatively, count the temporary user mappings. - Backend servers. - Temporary user mappings between user - backend server. This database is accessible by all Dovecot proxies in case there’s multiple. Steps when receiving a login: - Check if a temporary user mapping exists. - If so, proxy to the backend server in the temporary mapping. (To do: clean up mappings.) - If not, pick the backend server with the lowest vhost count, create a temporary mapping, then increase the vhost count of the chosen backend server. A monitoring service up/downs backend servers. E.g. by checking the port that we proxy to for each backend server. When a backend server is set to down, kick the user to force a reconnection. (Is that how Director ‘moves’ users?)

2 2