January 2024 - dovecot

fts indexer stuck on very large plaintext messages
by p h 29 Jan '24

29 Jan '24

Hi, I recently fts indexed a users mailbox with xapian which had 10-15 very large plaintext messages (>20Mb). The content of these messages was mostly a list of all files on a linux server (tripwire and backup reports). The indexer-worker process was completely stuck processing these messages, cpu pegged at 100% running continuously for multiple days. I was wondering if there is way to avoid this issue. It would be great if I could configure limits for the fts indexing process.

2 1

shared mailbox replication
by micha＠micha.waw.pl 28 Jan '24

28 Jan '24

We use a shared mailbox, where incoming mails from customers are stored. Every user can see it as "shared/contact(a)domain.name". When a new message comes, any user who is currently available just takes the message and does something with it. It's configured this way: namespace shared { type = shared separator = / prefix = shared/%%u/ location = maildir:%%h/.maildir:LAYOUT=fs:INDEX=~/.shared/%%u subscriptions = no list = children } It worked perfectly until I set up replication. Now we have a mess, because a message moved/deleted on one server are replicated beck form another. This is known issue. The documentation says it clear: "Warning Shared folder replication doesn’t work correctly. Mainly it can generate a lot of duplicate emails. This is because there’s currently a per-user lock that prevents multiple dsyncs from working simultaneously on the same user. But with shared folders multiple users can be syncing the same folder. So this would need additional locks (e.g. shared folders would likely need to lock the owner user, and public folders would likely need a per-folder lock or maybe a global public folder lock). There are no plans to fix this." I understand "There are no plans to fix this." But... maybe there is a workaround? I am sure someone before me had already this problem... :-)

1 0

Tests fail 2.3.21 in aarch64 build
by Peter 28 Jan '24

28 Jan '24

Building Dovecot 2.3.21 for aarch64 (building with qemu emulation) I get the following in the test stage: test-file-cache.c:266: Assert failed: file_cache_set_size(cache, 1024) == -1 Panic: file test-common.c: line 211 (test_expect_error_string_n_times): assertion failed: (expected_errors == 0) test-common.c:245: Assert failed: suppress == TRUE Error: Raw backtrace: ./test-lib(+0x693f0) [0x7f10b36943f0] -> ./test-lib(+0x6d8e4) [0x7f10b36988e4] -> ./test-lib(+0x75110) [0x7f10b36a0110] -> ./test-lib(+0x75168) [0x7f10b36a0168] -> ./test-lib(+0x15534) [0x7f10b3640534] -> ./test-lib(+0x152ec) [0x7f10b36402ec] -> ./test-lib(+0x3411c) [0x7f10b365f11c] -> ./test-lib(+0x15e64) [0x7f10b3640e64] -> /lib64/libc.so.6(+0x2c79c) [0x40000087579c] -> /lib64/libc.so.6(__libc_start_main+0x98) [0x40000087586c] -> ./test-lib(+0x163f0) [0x7f10b36413f0] qemu: uncaught target signal 6 (Aborted) - core dumped I'll provide more info on request. Is this just an issue with the tests or is it an issue with Dovecot itself? Peter

1 0

Azure AD / Entra Id OAuth2 issue, username not recognized
by meint＠meint.net 26 Jan '24

26 Jan '24

Dovecot version 2.3.16 Trying to use Azure AD / Entra Id with OAuth2 authentication, I have most of it working but somehow Dovecot fails to map / recognize the username, hopefully somebody can point out what I'm doing wrong? Excerpt from log: Jan 26 09:13:20 localhost dovecot: auth: Debug: http-client: conn [2603:1026:3000:108::4]:443 [1]: Got 200 response for request [Req1: POST https://login.microsoftonline.com/79b065d9-761f-4b79-a5cb-71a452f452af/oaut…] OK (took 220 ms + 82 ms in queue) Jan 26 09:13:20 localhost dovecot: auth: Debug: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): Password grant succeeded Jan 26 09:13:20 localhost dovecot: auth: Debug: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): Processing field token_type Jan 26 09:13:20 localhost dovecot: auth: Debug: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): Processing field scope Jan 26 09:13:20 localhost dovecot: auth: Debug: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): Processing field expires_in Jan 26 09:13:20 localhost dovecot: auth: Debug: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): Processing field ext_expires_in Jan 26 09:13:20 localhost dovecot: auth: Debug: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): Processing field access_token Jan 26 09:13:20 localhost dovecot: auth: Error: oauth2(test-external@secumailer.com,2a05:d018:ebe:d005:22b6:dc4b:41c6:7fec): oauth2 failed: Password grant failed: No username returned I'm using Resource Owner Password Grant flow. I have the following configured in conf.d/auth-oauth.conf.ext: passdb { driver = oauth2 mechanisms = plain args = /etc/dovecot/dovecot-oauth2.plain.conf.ext } I have the following configured in dovecot-oauth2.plain.conf.ext: grant_url = https://login.microsoftonline.com/<redacted>/oauth2/v2.0/token client_id = <redacted> client_secret = <redacted> username_attribute = email use_grant_password = yes scope = api://<redacted>/mail-relay debug = yes rawlog_dir = /tmp/oauth2 I have the following response from the oauth raw logs: out: 1706256800.309971 POST /<redacted>/oauth2/v2.0/token HTTP/1.1 1706256800.309971 Host: login.microsoftonline.com 1706256800.309971 Date: Fri, 26 Jan 2024 08:13:20 GMT 1706256800.309971 User-Agent: dovecot-oauth2-passdb/2.3.16 1706256800.309971 Content-Length: 255 1706256800.309971 Connection: Keep-Alive 1706256800.310013 Content-Type: application/x-www-form-urlencoded 1706256800.310018 1706256800.310056 grant_type=password&username=test-external(a)example.org&password=<redacted>&client_id=<redacted>&client_secret=<redacted>&scope=api:%2f%2f<redacted>%2fmail-relay in: 1706256800.530754 HTTP/1.1 200 OK 1706256800.530754 Cache-Control: no-store, no-cache 1706256800.530754 Pragma: no-cache 1706256800.530754 Content-Type: application/json; charset=utf-8 1706256800.530754 Expires: -1 1706256800.530754 Strict-Transport-Security: max-age=31536000; includeSubDomains 1706256800.530754 X-Content-Type-Options: nosniff 1706256800.530754 P3P: CP="DSP CUR OTPi IND OTRi ONL FIN" 1706256800.530754 x-ms-request-id: 629ce7bf-a5e9-4655-9019-83690c8b4400 1706256800.530754 x-ms-ests-server: 2.1.17122.2 - SEC ProdSlices 1706256800.530754 X-XSS-Protection: 0 1706256800.530754 Set-Cookie: fpc=Am8EyNTmX2ZOkMLj2MB82W4gKLwwAQAAAJ9gRd0OAAAA; expires=Sun, 25-Feb-2024 08:13:20 GMT; path=/; secure; HttpOnly; SameSite=None 1706256800.530754 Set-Cookie: x-ms-gateway-slice=estsfd; path=/; secure; samesite=none; httponly 1706256800.530754 Set-Cookie: stsservicecookie=estsfd; path=/; secure; samesite=none; httponly 1706256800.530754 Date: Fri, 26 Jan 2024 08:13:20 GMT 1706256800.530754 Content-Length: 1979 1706256800.530754 1706256800.530754 {"token_type":"Bearer","scope":"api://<redacted>/mail-relay","expires_in":5296,"ext_expires_in":5296,"access_token":"eyJ0eX ...<redacted>"} If I inspect the access token I'm getting: { "aud": "<redacted>", "iss": "<redacted>", "iat": 1706256500, "nbf": 1706256500, "exp": 1706262097, "acr": "1", "aio": "ATQAy/8VAAAANgbkS6NmHL3yGyROhLM28ooLX57pCjHjp0TGtd7EjwgVZy/i2aEqfV6sxiHeGZ0N", "amr": [ "pwd" ], "appid": "<redacted>", "appidacr": "1", "email": "test-external(a)example.org", "ipaddr": "<redacted>", "login_hint": "<redacted>", "name": "test-external", "oid": "657401f0-c9f0-402c-af6f-9ef063e7a281", "preferred_username": "test-external(a)example.org", "rh": "0.ATEA2WWweR92eUuly3GkUvRSr08CZ45J3dRLnCQJZCVUYBcxAKo.", "scp": "mail-relay", "sub": "ST9owcQQN2e0tBsHiVjvR8SCdlfUR_114tJk4C2bRxg", "tid": "79b065d9-761f-4b79-a5cb-71a452f452af", "unique_name": "test-external(a)sexample.org", "upn": "test-external(a)example.org", "uti": "v-ecYumlVUaQGYNpDItEAA", "ver": "1.0" } The email attribute is present in the access token yet Dovecot doesnt appear to recognize it. What am I doing wrong?

2 3

Replication going away?
by Vladimir Mishonov 26 Jan '24

26 Jan '24

Hello everyone. Just saw this commit in the official Github repo: https://github.com/dovecot/core/commit/4c04e4c30fd4817a8b0e11d04d9681173f69… Looking at the commit details, it appears that it completely removes the replication feature. I'm a bit perplexed by this change and am not sure what might be the justification for it. Personally, I find replication to be very useful, as it allows me to maintain a synchronized mirror of all of my mailboxes on my home server, for use as backup in case the primary server goes down for some reason. Perhaps there's some sort of replacement being planned for this feature? Or maybe the relevant code is simply going to be refactored to a plugin or external program, and there's nothing to worry about at all? In any case, I'd greatly appreciate if one of the developers could comment on this change. -- Kind regards, a concerned user of Dovecot (Vladimir)

34 66

Error activating filter set
by joe a 25 Jan '24

25 Jan '24

Some success creating and using filters, created via roundcube, but, when testing activate and deactivating filter sets, received an error. However, I can edit and add filters without apparent error, even "importing" from an existing filter set. var/log/dovecot-info.log shows, in part: ([%user] refers to the active dovecot user, whose account seems to operate perfectly, aside from filters). Obfuscated out of habit. Form is user(a)domain.com. "Jan 24 15:44:49 managesieve([%user])<23204><Pk9VIrcPfKx/AAAB>: Info: Failed to activate script `dovecot.orig': Internal error occurred. Refer to server log for more information. [2024-01-24 15:44:49]" It is unclear, among other things, what and where this "server log" is. tail -f /var/log/messages shows nothing during the various traipsing about in filters Not just foggy outdoors lately.

1 1

IMAP Sieve fileinto :create intermediate folders
by dovecot-4tg＠freakin.rocks 25 Jan '24

25 Jan '24

I apologize if this isn't the appropriate place for this question. Consider a Sieve script with this instruction: fileinto :create "folder 1.folder 2.folder 3"; That faithfully creates in Maildir the folder named '.folder 1.folder 2.folder 3'/ An IMAP Client shows this in tree format as: - folder 1 - - folder 2 - - - folder 3 But there isn't really a "folder 2". That is, there is no Maildir '.folder 1.folder 2' so what happens in the MUA is really subject to its coding. In Thunderbird, for example, 'folder 2' is grayed out. You can't subscribe to it or drop a file into that folder because it is disabled. With no mail actually targeted to that folder, that's generally not a problem, but Thunderbird does raise errors as it attempts to scan folders, correctly claiming that it can't access the folder because it doesn't exist. That's irritating to the user. So to the questions: 1) Is there anything in Dovecot or Pidgeonhole Sieve that might allow us to auto-create the missing intermediate folders? Maybe using a vnd.dovecot.* extension or some sieve capability? 2) Aside from being inelegant, is there anything wrong with a cron script that creates missing Maildir folders? While I am developer, I haven't yet looked into the proper way to create and configure folders user folders. 3) Would it be more appropriate to create a new extension, implementation of some RFC, Lua script, or some other type of plugin that might get triggered on a fileinto operation? Again, I haven't looked deeply in this direction and could use a few RTM references. 4) Might there already exist an external script or custom extension that does this? 5) Might this be a worthy enhancement request for Dovecot or Pidgeonhole Sieve? Thanks! Dovecot 2.3.7.2 / Pidgeonhole 0.5.7.2 / Ubuntu v20

1 0

lda or lmtp for sieve?
by Joe Acquisto 25 Jan '24

25 Jan '24

I noticed that many places in the documentation and in examples gleaned from the wilderness, refer to the LDA protocol when discussing sieve. The documentation also mentions that lmtp is preferred over lda, and seems to say in places that sieve will operate without issue in either case. Does it matter to sieve implementation if one uses only lmtp?

7 13

Can Dovecot use an MH data store?
by Steve Litt 25 Jan '24

25 Jan '24

Hi all, I know Dovecot can use a Maildir mail store and a Mailbox mail store. Can it also use an MH mail store? If so, is there a way of getting procmail to dump messages into the correct MH folder? Thanks, SteveT Steve Litt Autumn 2023 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21

1 0

Re: "Connection reset by peer" errors with Outlook
by Joseph Tam 24 Jan '24

24 Jan '24

From: Steve Dondley <s(a)dondley.com> > I have no idea what is triggering it for so many different users from legit > email addresses. Still investigating. But this appears to be a fail2ban > problem, not a dovecot problem. My logs are filled with failed authentication from Outlook clients. The clients seem to be trying different usernames (with/without domains), and maybe SSL/TLS flavours. My guess is Outlook is doing some autodiscovery/autoconfiguration thing, and occasionally hits the right combo and successfully authenticates. I'm not sure I would characterise this as a fail2ban problem. Fail2ban is doing what it says on the side of the tin: looking for repeated authentication failures, and blocking those that fail too many times. The real problem is Outlook fumble around for the correct settings, and mimicking a brute force attack. I've had great difficulty getting some Outlook clients to configure exactly the settings it should have (like excluding domain names from usernames). Try running his command line using Windows-R (not from cmd.exe). outlook.exe /PIM NoEmail This will avoid the auto-setup process that railroads you into frustration. MacOSX Mail app tries the same stuff, but at least you can turn that behaviour off and stop it from second guessing your settings. Joseph Tam <jtam.home(a)gmail.com>

2 1