March 2020 - dovecot - dovecot.org

mail_crypt crashing randomly during LMTP
by Andi Krischer 31 Mar '20

31 Mar '20

Hello everybody, hope you’re all well! We have a really strange bug with our dovecot setup. After weeks and month of debugging we’re finally reached the point when we ask you for help. Our basic setup is quite simple: a single dovecot 2.3.10 (0da0eff44) install on Debian 10 with LDAP as user/passdb and sieve, imap_sieve and mail_crypt as mail_plugins. For the mail_crypt we have a special rule which allows us to enable crypting for each user individually. See this line from dovecot-ldap.conf.ext: pass_attrs = userPassword=password,=userdb_mail_plugins=%{if;%{real_lport};eq;993;imap_sieve;} %{if;%{real_lport};eq;143;imap_sieve;} notify mail_log %{if;%{ldap:encryptMail};eq;TRUE;mail_crypt; },=userdb_mail_crypt_private_password=%w As you can see for IMAP connections we enable imap_sieve and if the LDAP attribute “encryptMail” is set to TRUE we enable “mail_crypt” as mail_plugin. This works fine except that around 5-10 times a day the dovecot server crashes during a LMTP connection. It crashes in the last steps of LMTP which result that the mail get delivered to Maildir but Postfix doesn’t get a positive response and delivers the mail again after a few minutes. So all of our users get some mails twice or even more times. This is what the log says in the moment of crashing: dovecot: lmtp(53852): Panic: Module context mail_crypt_user_module missing *** backtrace *** (see: https://pastebin.com/YCiFtxmy) dovecot: lmtp(53852): Fatal: master: service(lmtp): child 53852 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Here I’ve pasted the LTMP session in mail.log: https://pastebin.com/A4ughNUF Obviously it looks like it has something to do with our mail_crypt setup. But the thing is that although we’re monitoring this for months we cannot find a condition which cause this. It seems to happen more frequent for mails with lot of people in To or CC (large headers), but definitively not all of them… Some more things we’ve already thought on: First my guess was that the mail_crypt plugin isn’t loaded in some case but in the backtrace are things from “lib10_mail_crypt_plugin.so”, so it should be loaded, right? On the other hand a non-crashing LTMP session looks exactly the same in log except it has three additional lines in mail.log: Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so My college already had a look in the source code, but this didn’t make us much smarter. Here his references: mail context is defined here: https://github.com/dovecot/core/blob/81b5b188c478ec36bea8bda8fcad1e5f32ac61… and removed again here: https://github.com/dovecot/core/blob/81b5b188c478ec36bea8bda8fcad1e5f32ac61… Does any of the experts on this list have an idea what could cause these crashes and how to fix them? Thanks in advance! Best, Andi

2 5

limit for user exceeded
by Maciej Milaszewski 31 Mar '20

31 Mar '20

Hi System debian 8.11 dovecot-2.2.36.4 and I have some warnings in log likes: Warning: Inotify watch limit for user exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_watches cat /proc/sys/fs/inotify/max_user_watches 8192 in sysctl i change #fs.inotify.max_user_watches= 8192 #fs.inotify.max_user_instances = 16384 fs.inotify.max_user_watches= 16384 fs.inotify.max_user_instances = 24576 One questions - should these values be equal?

2 2

IMAP error
by Roy Lemmon 30 Mar '20

30 Mar '20

Hi, I am trying to setup a dovecot server on my laptop to serve my emails to emacs gnus. The emails are pulled from the mail server using isync into particular directories. I am using a Guix system. The dovecot service is running. I can telnet into it with: telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot ready. However if I then do: b select inbox b BAD Error in IMAP command received by server I have not logged in as I have not set any username or password. I assumed that access was open on localhost. Is this not correct ? I am not sure how to check further. I have checked the configuration and it is pointing to my email directories, imap is enabled etc. Thanks Roy.

2 2

Panic/Assert dns-lookup.c
by tim＠linux-daus.de 30 Mar '20

30 Mar '20

Hi, currently we deploying Dovecot as imap/pop3 proxy. Every few minutes some panic/assert occurred (we connect roughly 7k - 8k user at one imap proxy with a connection rate of 200/s). We activate core dumps. Concerning the sensitive information in the dump we would prefer to not share the dump (e.g. i found our ssl private key in the dump). Log/Stack trace: Mar 30 15:54:06 imap16 dovecot: auth: Panic: file dns-lookup.c: line 371 (dns_client_lookup_common): assertion failed: (param != NULL && *param != '\0') Mar 30 15:54:06 imap16 dovecot: auth: Error: Raw backtrace: #0 t_askpass[0x7f27a219b5f0] -> #1 backtrace_append[0x7f27a219b860] -> #2 backtrace_get[0x7f27a219b9c0] -> #3 i_syslog_error_handler[0x7f27a21a6840] -> #4 i_syslog_fatal_handler[0x7f27a21a6970] -> #5 i_fatal[0x7f27a20fc3b7] -> #6 dns_client_connect[0x7f27a216ffb0] -> #7 dns_client_lookup[0x7f27a21702a0] -> #8 auth_request_proxy_finish[0x55c930e9b200] -> #9 auth_request_handler_reply[0x55c930e9cee0] -> #10 auth_policy_check[0x55c930e93a10] -> #11 auth_request_success[0x55c930e9bcf0] -> #12 auth_request_verify_plain_callback_finish[0x55c930e9a650] -> #13 auth_request_verify_plain_callback[0x55c930e9a7a0] -> #14 authdb_ldap_deinit[0x7f279faa9f10] -> #15 db_ldap_result_iterate_deinit[0x7f279faa7f70] -> #16 io_loop_call_io[0x7f27a21c0490] -> #17 io_loop_handler_run_internal[0x7f27a21c1e20] -> #18 io_loop_handler_run[0x7f27a21c05c0] -> #19 io_loop_run[0x7f27a21c0810] -> #20 master_service_run[0x7f27a212d5b0] -> #21 main[0x55c930e8dd10] -> #22 __libc_start_main[0x7f27a14901f0] -> #23 _start[0x55c930e8e2c0] -> #24 [no start/end information] Mar 30 15:54:06 imap16 dovecot: auth: Fatal: master: service(auth): child 6133 killed with signal 6 (core dumped) Config: # 2.3.9.2 (844fc8246): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.9 (db4e9a2f) # OS: Linux 4.9.0-12-amd64 x86_64 Debian 9.12 # Hostname: imap16.domain.de auth_default_realm = domain.de auth_failure_delay = 0 auth_mechanisms = plain login cram-md5 auth_username_format = %{if;%d;eq;domain.de;%n(a)olddomain.de;%u} auth_verbose = yes base_dir = /var/run/dovecot/ default_client_limit = 4096 default_internal_user = pop default_process_limit = 400 default_vsz_limit = 1 G doveadm_password = # hidden, use -P to show it first_valid_uid = 48 import_environment = TZ last_valid_uid = 48 login_trusted_networks = 192.168.11.0/24 mail_gid = pop mail_plugins = " mail_log notify zlib quota" mail_uid = pop passdb { args = /etc/dovecot/conf.d/dovecot-ldap-domain-proxy.conf.ext driver = ldap result_failure = return-fail result_success = continue-ok } passdb { args = allow_real_nets=192.168.11.0/24 driver = static result_failure = continue-ok } passdb { args = /etc/dovecot/conf.d/dovecot-ldap-domain-protocol-deny.conf.ext driver = ldap result_failure = return-ok result_success = return-fail } passdb { args = /etc/dovecot/passdb-domain-ldap-cram.conf.ext driver = ldap mechanisms = CRAM-MD5 result_failure = continue-fail result_success = continue-ok } passdb { args = /etc/dovecot/passdb-domain-ldap.conf.ext driver = ldap mechanisms = LOGIN,PLAIN result_failure = return-fail result_success = continue-ok } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size zlib_save = gz zlib_save_level = 6 } protocols = " imap pop3" service auth { unix_listener auth-client { group = dovecot_auth mode = 0660 user = $default_internal_user } } service doveadm { group = pop inet_listener { port = 12345 } user = pop } service imap-login { process_min_avail = 24 service_count = 0 } service pop3-login { process_min_avail = 24 service_count = 0 } ssl = required ssl_cert = </etc/ssl/certs/star_domain_de.crt ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-CAMELLIA128-SHA:DHE-RSA-CAMELLIA256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:CAMELLIA128-SHA:CAMELLIA256-SHA ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it verbose_proctitle = yes

2 4

Namespace problem? duplicated folders...
by Gregory Sloop 30 Mar '20

30 Mar '20

I'm sure this is related to name-spaces, but for some reason [brain damage perhaps? :) ] I can't seem to figure out how to fix it. In TBird, for example, I have a folder tree that looks like this Inbox -| Folder-A Folder-B Folder-C Folder-A Folder-B Folder-C And Folder-A, Folder-B, Folder-C are the same folders, just shown twice. [In two different hierarchies.] I'm using mbox files to store mail. [If that matters.] Here's what I have in my conf files for namespace defs namespace { inbox = yes #hidden = yes prefix = INBOX/ separator = / } namespace inbox { mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } } I've had this problem for literally years, and it's not been that big of a deal (mostly irritating) - and I've tried to "fix" it before without any success. But now it's causing some issues and I'd really like to get this nagging issue gone. Since it's been so long since I last tried to fix it, I really can't recall any of settings changes I made in an attempt to fix it. I'd guess this is a trivial issue, but if so, can someone point me in the right direction. Thanks in advance! -Greg

2 5

Re: Help with migration to Dovecot from raw files
by Stefano Antonelli 30 Mar '20

30 Mar '20

On 2020-03-28 23:54, Stefano Antonelli wrote: > I've attached my script in case it's helpful to anyone in a similar > situation. It took quite a bit of effort to get this right, highly > depends on the format of the original mailstore, and I've only tested > this on one smaller mailbox (<400 messages). And I wasn't concerned > with any client having to download everything again since this was a > fresh import. > > In any case, it worked for me. Err had a few issues on a larger 4gb mailbox: 1. small bug in the script, new script attached. Change: # Grab the flags from the filename. F=`echo $fname | cut -c 26-27` to: # Grab the flags from the filename. F=0x`echo $fname | cut -c 26-27` without the 0x in front, the masking operation interprets $F as octal. 2. there are cases where the old mail server wrote files with identical timestamps even though the dates of the email are clearly different. I presume this was just buggy behaviour and could explain why I couldn't successfully download via imap. 201011072323050332-000001060000028C3D2E9D97.imap 1289200985.M000000P12345.icewarp,S=5614,W=5614:2,RS Received: Tue, 14 Sep 2010 19:44:47 -0700 201011072323050330-000001060000028A3D2C9676.imap 1289200985.M000000P12345.icewarp,S=2317,W=2317:2,RS Received: Sun, 12 Sep 2010 18:51:44 -0700 I may consider fixing this. 3. for some reason there were some off by one file sizes that resulted in errors like this: dovecot.index.cache: UID 278: Broken physical size in mailbox ... failed: Cached message size smaller than expected (1972 < 1973 ,box=INBOX, UID=278) In my case 298 out of 22023 files were affected. Script to fix this is after using the import-from-icewarp-files.sh script is attached. I couldn't find a good way to find only maildir messages by filename. Seems difficult to parse. When I renamed the files, I used icewarp as the server name and that's how I searched for files: FILES=`find /path/to/mail/user -name \*\.icewarp\,\*` If there's a better way to do this, I would like to know for future reference. Thanks, Stef

1 0

Help with migration to Dovecot from raw files
by Stefano Antonelli 29 Mar '20

29 Mar '20

Hello list, I recently migrated to Dovecot from a very old version of IceWarp running on failing hardware running Windows XP. I had a lot of issues downloading mail through imap. What I have instead are the email files directly off the harddrive from IceWarp's mail store. I rearranged the mail files to look like a Dovecot maildir and tried to use these emails directly. Dovecot did not complain, but I ran into my first problem. The emails were out of order on the client side. None of my imap clients store email. They all just fetch headers and download mail as it's required. So a resync is not a problem. I just can't figure out how and I don't know if the filenames are a problem. The files I'm working with have this format: 2020032512105639B0-000001000000D2D45079615C.imap The first part is the date and all the the filenames appear to be unique. Is it okay to leave them this way or should I rename them in the same fashion as Dovecot? If so, is there a tool I can use or do I need to write a script? If I have to write a script, could someone help me decipher this filename: 1585197800.M440284P10160.hostname,S=2812,W=2875:2,S I understand the first bit is a unix timestamp. I couldn't find out what M440284P10160 means or how I might create that number myself to use in a renaming script. Additionally, I'd like to customise the hostname without changing the machine's hostname and I understand that the S and W can also be omitted. I'm not using quotas. I couldn't find anywhere in the documentation where it talked about any configuration options for the filenames. Are there any or do I have to patch the source? I was also planning on directly backing up the email using rsync, but the changing filenames with flags is obviously an issue. I understand that dovecot can store flags in a file instead, if the mailbox is shared or public. Is there any way to have this behaviour with a private mailbox? Ideally the filenames are not touched after creation in which case rsync will work nicely to my NAS. The last issue is that since Dovecot went live, I've started receiving mail. For the time being, I have left my "yet to be migrated" mail outside of maildir. So Dovecot doesn't know about it. I will need to merge this newly received mail with my "yet to migrated mail" once I sort out exactly how to migrate that mail. And I would really like all the mail to sort properly. From what I can tell doveadm is the tool for the job, but I just can't make sense of the documentation. Thanks, Stef Dovecot version: 2.3.4.1 (f79e8e7e4)

2 3

managesieve user
by André Rodier 28 Mar '20

28 Mar '20

Hello, I am updating dovecot, from Debian Stretch to Buster, and I have an issue with the managesieve service: > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth: > connect(login) in directory / failed: Permission denied > (euid=120(<getpwuid() error>) egid=128(<getgrgid() error>) missing +x > perm: /, we're not in group 127, dir owned by 0:127 mode=0770) > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth: > connect(login) in directory / failed: Permission denied > (euid=120(<unknown>) egid=128(<unknown>) missing +x perm: /, we're > not in group 127, dir owned by 0:127 mode=0770) The two folders I found with these permissions are /run/dovecot/login and /run/dovecot/token-login. Both folders belongs to root:dovecot If I change the permissions of the following folders, like this: chmod 0775 /run/dovecot/login chmod 0775 /run/dovecot/token-login Then, dovecot managesieve is working, but this is just a test, I suppose the service should run with a specific user? Do you have any idea on what is wrong here? If I restart dovecot, I have an error message that states the permissions are wrong, and they are set again to 0770 Thanks, André Rodier

2 2

Forward a copy
by ＠lbutlr 28 Mar '20

28 Mar '20

<https://wiki.dovecot.org/Pigeonhole/Sieve/Examples> has this example: if header :contains "subject" ["order", "buy"] { redirect "orders(a)company.xn--dom-9o0a; } Can I simply add fileinto :create “mailbox”; before that redirect line so that a COPY of the message is forwarded? And does redirect imply stop, since I don;’t see stop in the example. -- They say only the good die young. If it works the other way too I'm immortal

2 1

doveadm rebuild: Failed to add attachment keywords
by Patrick Ben Koetter 27 Mar '20

27 Mar '20

Greetings, I'm using the current dovecot release 2.3.10 and I try to rebuild mailboxes with broken attachment paths. When I run this command: doveadm rebuild attachments -u user(a)example.com ALL I runs fine for 6199 of 6223 messages and fails then at message 6200 with this message: doveadm(user(a)example.com): Error: Mailbox INBOX: UID=6200: read(attachments-connector(/MailStore/mail/mailboxes/CFEE0E67-6269-4AD3-8DED-6AB71A8E4BD7/dbox/mailboxes/INBOX/dbox-Mails/u.6200)) failed: read(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200[base64:19 b/l]) failed: open(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200) failed: No such file or directory (read reason=) doveadm(user(a)example.com): Error: Mailbox INBOX: UID=6200: read(attachments-connector(/MailStore/mail/mailboxes/CFEE0E67-6269-4AD3-8DED-6AB71A8E4BD7/dbox/mailboxes/INBOX/dbox-Mails/u.6200)) failed: read(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200[base64:19 b/l]) failed: open(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200) failed: No such file or directory (read reason=) doveadm(user(a)example.com): Error: Mailbox INBOX: UID=6200: Failed to add attachment keywords: mail_get_parts() failed: Mailbox INBOX: UID=6200: read(attachments-connector(/MailStore/mail/mailboxes/CFEE0E67-6269-4AD3-8DED-6AB71A8E4BD7/dbox/mailboxes/INBOX/dbox-Mails/u.6200)) failed: read(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200[base64:19 b/l]) failed: open(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200) failed: No such file or directory (read reason=) 6200 error Assuming the attachments are still there, what would I need to do to 'fix' that? And if I can't fix it, can 'rebuild' skip the error and go on rebuilding the rest? TIA, p@rick -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG,80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief Aufsichtsratsvorsitzender: Florian Kirstein

2 1