- dovecot - dovecot.org

exim+dovecot+sieve issue
by JM 18 Aug '19

18 Aug '19

Hi, We have mail server in production, with Exim + Dovecot + Mailman + OpenLDAP + Roundcube webmail server. Openldap keeps mail accounts in db, exim uses lmtp transport to deliver messages to dovecot. Mailman is responsible for mailing lists, Roundcube webmail has managsieve module to create sieve filters. Exim router,transport for dovecot > localuser: > driver = accept > domains = +local_domains : +virt_domains > check_local_user > transport = dovecot_lmtp > cannot_route_message = Unknown account > no_more > dovecot_lmtp: > driver = lmtp > socket = /var/spool/dovecot/lmtp > batch_max = 500 > rcpt_include_affixes It worked perfectly, but after upgrade from Dovecot 2.2 to Dovecot 2.3 major version, I've found a lot of strange errors in exim panic.log > mail exim[7821] Failed to get user name for uid 1175 > mail exim[7880]: Failed to get user name for uid 1173 etc where uids are ldap accounts and recognized by system (id 1175 shows user's name and group) Even worse, some clients start to receive notify email with strange content, when sending email to mailing lists > user(a)domain.com > Delay reason: error in redirect data: malformed address: > someuser1(a)domain.com may not follow someuser2(a)domain.com in " > someuser3(a)domain.com ..some crap here.." , where someuser addresses are > not related to recipient address and unknown for sender. After some investigation,I've found out that dovecot's sieve produces these errors (I suspect sieve vacation, when user creates vacation via Roundcube webmail). If set "submission_host = exim ip" in dovecot.conf, exim doesn't produce errors in panic.log anymore, but clients still receive email with malformed address. Please help me to find the root of the problem (some broken sieve filter in user's maildir, misconfiguration or breaking changes in new Dovecot version?) Current software Gentoo with 4.14.90 kernel Dovecot-2.3.7.1 Exim-4.92.1 Mailman 2.1.29 Openldap-2.4.47 Roundcube-1.3.8 Dovecot.conf attached Regards, Juri

2 1

2.3.7 + stats
by Matt Bryant 17 Aug '19

17 Aug '19

Is there any additional documentation/information around the new stats module. Have added some metrics just to see what they produce ## ## Metrics ### metric imap { event_name = imap_command_finished #source_location = example.c:123 #categories = fields = name args running_usecs bytes_in bytes_out #filter { # field_key = wildcard #} } metric sql { event_name = sql_query_finished } metric auth { event_name = auth_request_finished fields = user transport error successful } and get the following [root@stargate dovecot]# doveadm stats dump metric_name field count sum min max avg median stddev %95 imap duration 370 2007904499 130 62955249 5426768.92 2068 16436817.37 60026465 imap name 370 0 0 0 0.00 0 0.00 0 imap args 0 0 0 0 0.00 0 0.00 0 imap running_usecs 370 2007865330 81 62955127 5426663.05 1991 16436816.76 60026329 imap bytes_in 370 5366 2 173 14.50 8 19.04 35 imap bytes_out 370 2119971 0 941517 5729.65 41 53760.89 2082 sql duration 18 28991 991 2305 1610.61 1660 377.36 2305 auth duration 12 26046980 8146 7079879 2170581.67 847730 2457811.23 7079879 auth user 12 0 0 0 0.00 0 0.00 0 auth transport 12 0 0 0 0.00 0 0.00 0 auth error 0 0 0 0 0.00 0 0.00 0 auth successful 0 0 0 0 0.00 0 0.00 0 the main wiki page on stats/events doesnt really hold much detail whats stores for each event the above fields dont make much sense and top no longer works out of the box [root@stargate dovecot]# doveadm stats top usage: doveadm [-Dv] [-f <formatter>] stats <command> [<args>] dump [-s <stats socket path>] [-r] [-f <fields>] has is been removed ? do you need to specify something additional now ??? rgds Matt

3 2

James spooling messages problem
by Adam Raszkiewicz 16 Aug '19

16 Aug '19

For some reason my James setup is creating some kind of spooling queue of sending messages. It will queue 3 of them and then forth “sent” message will make 1st one to be sent outside. Any thoughts? Thanks -- Adam

1 0

SASL: encoded packet size too big
by Eugene 16 Aug '19

16 Aug '19

Hello! Dovecot uses it's own SASL implementation, doesn't it? Aug 14 23:45:23 example.com auth[10428]: GSSAPI client step 1 Aug 14 23:45:23 example.com auth[10428]: encoded packet size too big (813804546 > 65536) Aug 14 23:45:23 example.com dovecot[10085]: auth-worker(10428): Error: LDAP: Can't connect to server: ldap://ipa2.example.com Aug 14 23:45:23 example.com dovecot[10085]: auth: Error: auth worker: Aborted USER request for eugene: Lookup timed out Aug 14 23:45:23 example.com dovecot[10085]: imap: Error: auth-master: login: request [3847225345]: Login auth request failed: Internal auth failure (auth connected 60000 msecs ago, request took 60000 msecs, client-pid=10362 client-id=1) Looks like cyrus-sasl encountered same problem earlier. https://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2017-March/003001.html I never have such an issue with ldapsearch. So, I assume there is a similar problem in Dovecot SASL implementation. -- Eugene Bright IT engineer Tel: + 79257289622

3 6

Re: Dovecot - Microsoft Azure AD
by Aki Tuomi 15 Aug '19

15 Aug '19

Hi! Dovecot supports Lua userdb, which can be used to implement custom user databases, maybe this might work for you? See https://doc.dovecot.org/configuration_manual/authentication/lua_based_authe… for more details. Aki > On 15/08/2019 12:16 Lennart Boettcher <lennart.boettcher(a)secpoint.onmicrosoft.com> wrote: > > > Hello, > > Thank you for the quick reply. > > > > > > I have expressed myself wrongly. Our idea was to use the Azure-AD as userdb by doing the user lookup with the help of Microsoft's Graph API. OAuth2 would then of course only be the authorization procedure to access the user accounts using the Graph API. > > > > > > One would then implement a graph-userdb and no oauth-userdb. OAuth is, as you correctly mentioned, only an authorization mechanism. > > > > > > Here is a link to the GraphAPI: https://docs.microsoft.com/de-de/graph/api/overview?view=graph-rest-1.0 > > > And here is another link to the Graph Explorer, with which you can see how the GraphAPI works: https://developer.microsoft.com/en-us/graph/graph-explorer > > > > > We already use this procedure for the passdb lookup and it works very well. > > > > > Greetings > > Lennart Boettcher > > > > > ------------------------------ > > From: Aki Tuomi <aki.tuomi(a)open-xchange.com> > Sent: 14 August 2019 14:57 > To: Lennart Boettcher <lennart.boettcher(a)secpoint.onmicrosoft.com>; Lennart Boettcher via dovecot <dovecot(a)dovecot.org> > Subject: Re: Dovecot - Microsoft Azure AD > > > > On 14/08/2019 15:36 Lennart Boettcher via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > Hello, > > > > I am currently trying to connect my Dovecot mail server to Microsoft's Azure-AD and use it as password and user database. I am using version 2.3.7.1. > > > > > > > > > > > > Using the Azure-AD as passdb already works. In this context I noticed that the scope implementation is not yet merged. > > > > > > > > > > > > Since I haven't found any hints for an OAuth2 userdb implementation yet, I wanted to ask if there are any plans for an implementation. > > > > > > > > > > Greetings > > > > Lennart Boettcher > > > > > > > > > > Dovecot 2.3 supports oauth2. I don't know how "oauth2 user database" would work, since oauth2 is an authentication mechanism. I suggest you use LDAP or static userdb, or set mail_* settings for user settings. > > Aki >

1 0

Dovecot not responding to external clients
by Edwin Humphries 15 Aug '19

15 Aug '19

Hi all I have Dovecot on my Linux-Mint workstation, running IMAP for the Thunderbird client thereon, but previously also for my phone and laptop clients as well. Since upgrading to Linux-Mint 19, however, although the Thunderbird client on the workstation works just fine, the other clients can't connect. I've disabled the firewall, and nmap shows the ports open, so it seems like some setting in the server. Can anyone help? > Dovecot version is 2.2.33.2 (d6601f4ec) > > dovecot -n output: > > # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.21 (92477967) > # OS: Linux 4.15.0-55-generic x86_64 Linux Mint 19.1 Tessa > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_privileged_group = mail > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > driver = pam > } > protocols = " imap" > ssl_cert = </etc/dovecot/private/dovecot.pem > ssl_client_ca_dir = /etc/ssl/certs > ssl_key = # hidden, use -P to show it > userdb { > driver = passwd > } > -- Regards, Edwin Humphries

2 2

Autoexpunge not working for Junk?
by Amir Caspi 14 Aug '19

14 Aug '19

Hi all, I set up dovecot a couple of months ago and am having trouble getting autoexpunge=30d to work on my Trash and Junk mailboxes. Not sure why not because I'm not getting error messages in my log. Running "doveadm search -u <user> mailbox Junk savedbefore 30d" shows me many messages (I've got messages back to mid-May, and a couple of other users have them back to early April, although if this setting were working, there should be nothing earlier than June 24). Running a manual doveadm expunge works fine... it's just autoexpunge that seems to not be running at all. I'm using sendmail as the MTA and procmail as the LDA, so dovecot is running purely for IMAP/POP service. Any help is much appreciated. Thanks! doveconf -n: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.21.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: REDACTED auth_username_format = %Ln first_valid_uid = 1000 mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mailbox_list_index = yes mbox_write_locks = fcntl namespace compat1 { alias_for = hidden = yes list = no location = prefix = mail/ separator = / } namespace compat2 { alias_for = hidden = yes list = no location = prefix = ~/mail/ separator = / } namespace compat3 { alias_for = hidden = yes list = no location = prefix = ~%u/mail/ separator = / } namespace inbox { inbox = yes location = mailbox Archive { special_use = \Archive } mailbox "Deleted Messages" { autoexpunge = 30 days special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { autoexpunge = 30 days special_use = \Junk } mailbox "Junk E-mail" { autoexpunge = 30 days special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { autoexpunge = 30 days special_use = \Junk } mailbox Trash { autoexpunge = 30 days special_use = \Trash } prefix = separator = / } passdb { driver = pam } pop3_uidl_format = %08Xv%08Xu ssl_cert = # REDACTED ssl_cipher_list = # REDACTED ssl_dh_parameters_length = # REDACTED ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } --- Amir

7 14

segmentation fault in fs_list_get_path
by David M. Johnson 14 Aug '19

14 Aug '19

Hi, There seems to be a straightforward bug in src/lib-storage/list/mailbox-list-fs.c:79. set->index_dir is unchecked prior to dereferencing (unlike on line 126 in the same file, where it is properly checked). This manifested on a FreeBSD server running dovecot 2.3.6 when clients tried to retrieve mail with subscriptions like `~/bar/baz`. This caused the `imap` child to crash, e.g. (slightly anonymized) Core was generated by `imap: [foo w.x.y.z EXAMINE]'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000000011593af4 in fs_list_get_path (_list=0x12444848, name=0x12416880 "/home/foo/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe150) at mailbox-list-fs.c:79 79 *set->index_dir == '\0') (gdb) bt #0 0x0000000011593af4 in fs_list_get_path (_list=0x12444848, name=0x12416880 "/home/foo/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe150) at mailbox-list-fs.c:79 #1 0x0000000011559dd0 in mbox_list_get_path (list=0x12444848, name=0x124da410 "~/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe2c8) at mbox-storage.c:96 #2 0x000000001150ed0b in mailbox_list_get_path (list=0x12444848, name=0x124da410 "~/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe2c8) at mailbox-list.c:1387 #3 0x00000000114f8aaf in get_path_to (box=0x124da048, type=MAILBOX_LIST_PATH_TYPE_INDEX, internal_path=0x124da250, path_r=0x7fffffffe2c8) at mail-storage.c:2662 #4 0x00000000114f89e9 in mailbox_get_path_to (box=0x124da048, type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe2c8) at mail-storage.c:2678 #5 0x00000000114f928c in mailbox_create_missing_dir (box=0x124da048, type=MAILBOX_LIST_PATH_TYPE_INDEX) at mail-storage.c:2826 #6 0x00000000115cf049 in index_storage_mailbox_alloc_index (box=0x124da048) at index-storage.c:243 #7 0x00000000115cf43f in index_storage_mailbox_open (box=0x124da048, move_to_memory=false) at index-storage.c:297 #8 0x000000001155a715 in mbox_mailbox_open_finish (mbox=0x124da048, move_to_memory=false) at mbox-storage.c:413 #9 0x000000001155a94b in mbox_mailbox_open_existing (mbox=0x124da048) at mbox-storage.c:452 #10 0x0000000011559309 in mbox_mailbox_open (box=0x124da048) at mbox-storage.c:489 #11 0x00000000115a446e in mailbox_list_index_open_mailbox (box=0x124da048) at mailbox-list-index.c:720 #12 0x00000000114f43ed in mailbox_open_full (box=0x124da048, input=0x0) at mail-storage.c:1294 #13 0x00000000114f4117 in mailbox_open (box=0x124da048) at mail-storage.c:1350 #14 0x000000000103d788 in select_open (ctx=0x12443228, mailbox=0x12416810 "~/bar/baz", readonly=true) at cmd-select.c:287 #15 0x000000000103d307 in cmd_select_full (cmd=0x12443048, readonly=true) at cmd-select.c:415 #16 0x0000000001034afa in cmd_examine (cmd=0x12443048) at cmd-examine.c:8 #17 0x000000000104a520 in command_exec (cmd=0x12443048) at imap-commands.c:201 #18 0x000000000104804e in client_command_input (cmd=0x12443048) at imap-client.c:1164 #19 0x00000000010483cf in client_command_input (cmd=0x12443048) at imap-client.c:1227 #20 0x00000000010469fc in client_handle_next_command (client=0x12442848, remove_io_r=0x7fffffffe8c7) at imap-client.c:1269 #21 0x00000000010463c0 in client_handle_input (client=0x12442848) at imap-client.c:1283 #22 0x0000000001044134 in client_input (client=0x12442848) at imap-client.c:1329 #23 0x0000000011b6bd1c in io_loop_call_io (io=0x1244f240) at ioloop.c:703 #24 0x0000000011b6fc08 in io_loop_handler_run_internal (ioloop=0x1242b0a0) at ioloop-kqueue.c:160 #25 0x0000000011b6c37e in io_loop_handler_run (ioloop=0x1242b0a0) at ioloop.c:755 #26 0x0000000011b6c146 in io_loop_run (ioloop=0x1242b0a0) at ioloop.c:728 #27 0x0000000011a87dcb in master_service_run (service=0x12436000, callback=0x1060080 <client_connected>) at master-service.c:781 #28 0x000000000105f870 in main (argc=1, argv=0x7fffffffeb20) at main.c:523 (gdb) p set->index_dir $3 = 0x0 The following one-liner fixes the immediate problem (although I didn't look closely to see what set->index_dir means in the context of MAILBOX_LIST_PATH_TYPE_INDEX): --- src/lib-storage/list/mailbox-list-fs.c~ 2019-04-30 06:25:06.000000000 -0600 +++ src/lib-storage/list/mailbox-list-fs.c 2019-08-02 16:23:57.254087000 -0600 @@ -76,6 +76,7 @@ if (mailbox_list_try_get_absolute_path(_list, &name)) { if (type == MAILBOX_LIST_PATH_TYPE_INDEX && + set->index_dir != NULL && *set->index_dir == '\0') return 0; *path_r = name; David

2 1

auth module logging
by AP 14 Aug '19

14 Aug '19

Hi, As per my discussion with cmouse on irc, I'm currently just using dovecot for its auth mechanism, etc capabilities (alas, Exim does not support argon directly). Tis a fun little side project. :) I have the config pared down as far as I think is reasonable (see below) and all is working well except that dovecot is very silent. Errors hit the logs but I would appreciate seeing successful auths happen for the additional piece of mind. Cmouse and I couldn't find a way to do it on irc and it appears that the capability is missing. Successul /logins/ can be logged but auths, by themselves, cannot. I would appreciate if the ability was added. Dovecot 2.3.7.1 is in use. Thanks # Pigeonhole version 0.5.7.1 (db5c74be) # OS: Linux 5.2.4 x86_64 Debian 10.0 # Hostname: bob auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1:16 log_timestamp = "%Y-%m-%d %H:%M:%S %z " mail_log_prefix = "%{pid}<%{session}> %{service}(%{user}): " passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } service anvil { unix_listener anvil-auth-penalty { mode = 00 } } service auth-worker { user = $default_internal_user } service auth { unix_listener auth-client { group = Debian-exim mode = 0660 } } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } AP

3 3

dovecot-uidlist invalid data
by Király Balázs 14 Aug '19

14 Aug '19

Hi! I'm struggling with the following error: Aug 4 21:32:00 mx02 dovecot: imap(xxx(a)xxx.tld)<17693><w14vpU+PCCSwPxpo>: Error: Mailbox INBOX: Broken file /home/vmail/xxx.tld/xxx/dovecot-uidlist line 6246: Invalid data: Aug 4 21:49:22 mx02 dovecot: imap(xxx(a)xxx.tld)<21879><YNhW40+PLoBOGLlU>: Error: Mailbox INBOX: Broken file /home/vmail/xxx.tld/xxx/dovecot-uidlist line 6249: Invalid data: It seems the first part is not incremented properly and sometimes it has a jump in it, like the line 6246: 18810 :1564935891.M816284P8904.mx01.m.ininet.hu,S=12145,W=12409 18812 :1564947092.M542714P2651.mx01.m.ininet.hu,S=12275,W=12517 Even though I deleted all the dovecot* files the error comes back. I have the maildirs through NFS, but I set to store the INDEX files locally. I have failover configuration, that means when one of my servers die another one takes the active IP and start to serve dovecot users, so its always one server that serves all the users, no load balacing. Any idea what am I missing or how could I fixed this problem? uname -a Linux mx02.m.mydomain.tld 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5 (2019-06-19) x86_64 GNU/Linux dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-5-amd64 x86_64 Debian 10.0 nfs # Hostname: mx02.m.mydomain.tld auth_mechanisms = plain login auth_verbose = yes default_client_limit = 10000 default_process_limit = 1000 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no mail_fsync = always mail_location = maildir:/home/vmail/%d/%n:INDEX=/var/indexes/%d/%n mail_plugins = " quota" mail_privileged_group = mail mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota sieve = /home/vmail/default.sieve sieve_default = /home/vmail/default.sieve sieve_extensions = +editheader sieve_global_dir = /var/lib/dovecot/sieve } protocols = " imap pop3" service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = $default_internal_user } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service imap { process_limit = 1024 } service pop3 { process_limit = 1024 } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/postfix/ssl/mail.mydomain.tld-dovecot.crt ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota sieve" } protocol imap { mail_max_userip_connections = 20 mail_plugins = " quota quota imap_quota" } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = " quota quota" } Thanks, Balazs

2 1