March 2020 - dovecot - dovecot.org

Panic/Assert dns-lookup.c
by tim＠linux-daus.de 30 Mar '20

30 Mar '20

Hi, currently we deploying Dovecot as imap/pop3 proxy. Every few minutes some panic/assert occurred (we connect roughly 7k - 8k user at one imap proxy with a connection rate of 200/s). We activate core dumps. Concerning the sensitive information in the dump we would prefer to not share the dump (e.g. i found our ssl private key in the dump). Log/Stack trace: Mar 30 15:54:06 imap16 dovecot: auth: Panic: file dns-lookup.c: line 371 (dns_client_lookup_common): assertion failed: (param != NULL && *param != '\0') Mar 30 15:54:06 imap16 dovecot: auth: Error: Raw backtrace: #0 t_askpass[0x7f27a219b5f0] -> #1 backtrace_append[0x7f27a219b860] -> #2 backtrace_get[0x7f27a219b9c0] -> #3 i_syslog_error_handler[0x7f27a21a6840] -> #4 i_syslog_fatal_handler[0x7f27a21a6970] -> #5 i_fatal[0x7f27a20fc3b7] -> #6 dns_client_connect[0x7f27a216ffb0] -> #7 dns_client_lookup[0x7f27a21702a0] -> #8 auth_request_proxy_finish[0x55c930e9b200] -> #9 auth_request_handler_reply[0x55c930e9cee0] -> #10 auth_policy_check[0x55c930e93a10] -> #11 auth_request_success[0x55c930e9bcf0] -> #12 auth_request_verify_plain_callback_finish[0x55c930e9a650] -> #13 auth_request_verify_plain_callback[0x55c930e9a7a0] -> #14 authdb_ldap_deinit[0x7f279faa9f10] -> #15 db_ldap_result_iterate_deinit[0x7f279faa7f70] -> #16 io_loop_call_io[0x7f27a21c0490] -> #17 io_loop_handler_run_internal[0x7f27a21c1e20] -> #18 io_loop_handler_run[0x7f27a21c05c0] -> #19 io_loop_run[0x7f27a21c0810] -> #20 master_service_run[0x7f27a212d5b0] -> #21 main[0x55c930e8dd10] -> #22 __libc_start_main[0x7f27a14901f0] -> #23 _start[0x55c930e8e2c0] -> #24 [no start/end information] Mar 30 15:54:06 imap16 dovecot: auth: Fatal: master: service(auth): child 6133 killed with signal 6 (core dumped) Config: # 2.3.9.2 (844fc8246): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.9 (db4e9a2f) # OS: Linux 4.9.0-12-amd64 x86_64 Debian 9.12 # Hostname: imap16.domain.de auth_default_realm = domain.de auth_failure_delay = 0 auth_mechanisms = plain login cram-md5 auth_username_format = %{if;%d;eq;domain.de;%n(a)olddomain.de;%u} auth_verbose = yes base_dir = /var/run/dovecot/ default_client_limit = 4096 default_internal_user = pop default_process_limit = 400 default_vsz_limit = 1 G doveadm_password = # hidden, use -P to show it first_valid_uid = 48 import_environment = TZ last_valid_uid = 48 login_trusted_networks = 192.168.11.0/24 mail_gid = pop mail_plugins = " mail_log notify zlib quota" mail_uid = pop passdb { args = /etc/dovecot/conf.d/dovecot-ldap-domain-proxy.conf.ext driver = ldap result_failure = return-fail result_success = continue-ok } passdb { args = allow_real_nets=192.168.11.0/24 driver = static result_failure = continue-ok } passdb { args = /etc/dovecot/conf.d/dovecot-ldap-domain-protocol-deny.conf.ext driver = ldap result_failure = return-ok result_success = return-fail } passdb { args = /etc/dovecot/passdb-domain-ldap-cram.conf.ext driver = ldap mechanisms = CRAM-MD5 result_failure = continue-fail result_success = continue-ok } passdb { args = /etc/dovecot/passdb-domain-ldap.conf.ext driver = ldap mechanisms = LOGIN,PLAIN result_failure = return-fail result_success = continue-ok } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size zlib_save = gz zlib_save_level = 6 } protocols = " imap pop3" service auth { unix_listener auth-client { group = dovecot_auth mode = 0660 user = $default_internal_user } } service doveadm { group = pop inet_listener { port = 12345 } user = pop } service imap-login { process_min_avail = 24 service_count = 0 } service pop3-login { process_min_avail = 24 service_count = 0 } ssl = required ssl_cert = </etc/ssl/certs/star_domain_de.crt ssl_cipher_list = ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-CAMELLIA128-SHA:DHE-RSA-CAMELLIA256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:CAMELLIA128-SHA:CAMELLIA256-SHA ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it verbose_proctitle = yes

2 4

Namespace problem? duplicated folders...
by Gregory Sloop 30 Mar '20

30 Mar '20

I'm sure this is related to name-spaces, but for some reason [brain damage perhaps? :) ] I can't seem to figure out how to fix it. In TBird, for example, I have a folder tree that looks like this Inbox -| Folder-A Folder-B Folder-C Folder-A Folder-B Folder-C And Folder-A, Folder-B, Folder-C are the same folders, just shown twice. [In two different hierarchies.] I'm using mbox files to store mail. [If that matters.] Here's what I have in my conf files for namespace defs namespace { inbox = yes #hidden = yes prefix = INBOX/ separator = / } namespace inbox { mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } } I've had this problem for literally years, and it's not been that big of a deal (mostly irritating) - and I've tried to "fix" it before without any success. But now it's causing some issues and I'd really like to get this nagging issue gone. Since it's been so long since I last tried to fix it, I really can't recall any of settings changes I made in an attempt to fix it. I'd guess this is a trivial issue, but if so, can someone point me in the right direction. Thanks in advance! -Greg

2 5

Re: Help with migration to Dovecot from raw files
by Stefano Antonelli 30 Mar '20

30 Mar '20

On 2020-03-28 23:54, Stefano Antonelli wrote: > I've attached my script in case it's helpful to anyone in a similar > situation. It took quite a bit of effort to get this right, highly > depends on the format of the original mailstore, and I've only tested > this on one smaller mailbox (<400 messages). And I wasn't concerned > with any client having to download everything again since this was a > fresh import. > > In any case, it worked for me. Err had a few issues on a larger 4gb mailbox: 1. small bug in the script, new script attached. Change: # Grab the flags from the filename. F=`echo $fname | cut -c 26-27` to: # Grab the flags from the filename. F=0x`echo $fname | cut -c 26-27` without the 0x in front, the masking operation interprets $F as octal. 2. there are cases where the old mail server wrote files with identical timestamps even though the dates of the email are clearly different. I presume this was just buggy behaviour and could explain why I couldn't successfully download via imap. 201011072323050332-000001060000028C3D2E9D97.imap 1289200985.M000000P12345.icewarp,S=5614,W=5614:2,RS Received: Tue, 14 Sep 2010 19:44:47 -0700 201011072323050330-000001060000028A3D2C9676.imap 1289200985.M000000P12345.icewarp,S=2317,W=2317:2,RS Received: Sun, 12 Sep 2010 18:51:44 -0700 I may consider fixing this. 3. for some reason there were some off by one file sizes that resulted in errors like this: dovecot.index.cache: UID 278: Broken physical size in mailbox ... failed: Cached message size smaller than expected (1972 < 1973 ,box=INBOX, UID=278) In my case 298 out of 22023 files were affected. Script to fix this is after using the import-from-icewarp-files.sh script is attached. I couldn't find a good way to find only maildir messages by filename. Seems difficult to parse. When I renamed the files, I used icewarp as the server name and that's how I searched for files: FILES=`find /path/to/mail/user -name \*\.icewarp\,\*` If there's a better way to do this, I would like to know for future reference. Thanks, Stef

1 0

Help with migration to Dovecot from raw files
by Stefano Antonelli 29 Mar '20

29 Mar '20

Hello list, I recently migrated to Dovecot from a very old version of IceWarp running on failing hardware running Windows XP. I had a lot of issues downloading mail through imap. What I have instead are the email files directly off the harddrive from IceWarp's mail store. I rearranged the mail files to look like a Dovecot maildir and tried to use these emails directly. Dovecot did not complain, but I ran into my first problem. The emails were out of order on the client side. None of my imap clients store email. They all just fetch headers and download mail as it's required. So a resync is not a problem. I just can't figure out how and I don't know if the filenames are a problem. The files I'm working with have this format: 2020032512105639B0-000001000000D2D45079615C.imap The first part is the date and all the the filenames appear to be unique. Is it okay to leave them this way or should I rename them in the same fashion as Dovecot? If so, is there a tool I can use or do I need to write a script? If I have to write a script, could someone help me decipher this filename: 1585197800.M440284P10160.hostname,S=2812,W=2875:2,S I understand the first bit is a unix timestamp. I couldn't find out what M440284P10160 means or how I might create that number myself to use in a renaming script. Additionally, I'd like to customise the hostname without changing the machine's hostname and I understand that the S and W can also be omitted. I'm not using quotas. I couldn't find anywhere in the documentation where it talked about any configuration options for the filenames. Are there any or do I have to patch the source? I was also planning on directly backing up the email using rsync, but the changing filenames with flags is obviously an issue. I understand that dovecot can store flags in a file instead, if the mailbox is shared or public. Is there any way to have this behaviour with a private mailbox? Ideally the filenames are not touched after creation in which case rsync will work nicely to my NAS. The last issue is that since Dovecot went live, I've started receiving mail. For the time being, I have left my "yet to be migrated" mail outside of maildir. So Dovecot doesn't know about it. I will need to merge this newly received mail with my "yet to migrated mail" once I sort out exactly how to migrate that mail. And I would really like all the mail to sort properly. From what I can tell doveadm is the tool for the job, but I just can't make sense of the documentation. Thanks, Stef Dovecot version: 2.3.4.1 (f79e8e7e4)

2 3

managesieve user
by André Rodier 28 Mar '20

28 Mar '20

Hello, I am updating dovecot, from Debian Stretch to Buster, and I have an issue with the managesieve service: > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth: > connect(login) in directory / failed: Permission denied > (euid=120(<getpwuid() error>) egid=128(<getgrgid() error>) missing +x > perm: /, we're not in group 127, dir owned by 0:127 mode=0770) > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth: > connect(login) in directory / failed: Permission denied > (euid=120(<unknown>) egid=128(<unknown>) missing +x perm: /, we're > not in group 127, dir owned by 0:127 mode=0770) The two folders I found with these permissions are /run/dovecot/login and /run/dovecot/token-login. Both folders belongs to root:dovecot If I change the permissions of the following folders, like this: chmod 0775 /run/dovecot/login chmod 0775 /run/dovecot/token-login Then, dovecot managesieve is working, but this is just a test, I suppose the service should run with a specific user? Do you have any idea on what is wrong here? If I restart dovecot, I have an error message that states the permissions are wrong, and they are set again to 0770 Thanks, André Rodier

2 2

Forward a copy
by ＠lbutlr 28 Mar '20

28 Mar '20

<https://wiki.dovecot.org/Pigeonhole/Sieve/Examples> has this example: if header :contains "subject" ["order", "buy"] { redirect "orders(a)company.xn--dom-9o0a; } Can I simply add fileinto :create “mailbox”; before that redirect line so that a COPY of the message is forwarded? And does redirect imply stop, since I don;’t see stop in the example. -- They say only the good die young. If it works the other way too I'm immortal

2 1

doveadm rebuild: Failed to add attachment keywords
by Patrick Ben Koetter 27 Mar '20

27 Mar '20

Greetings, I'm using the current dovecot release 2.3.10 and I try to rebuild mailboxes with broken attachment paths. When I run this command: doveadm rebuild attachments -u user(a)example.com ALL I runs fine for 6199 of 6223 messages and fails then at message 6200 with this message: doveadm(user(a)example.com): Error: Mailbox INBOX: UID=6200: read(attachments-connector(/MailStore/mail/mailboxes/CFEE0E67-6269-4AD3-8DED-6AB71A8E4BD7/dbox/mailboxes/INBOX/dbox-Mails/u.6200)) failed: read(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200[base64:19 b/l]) failed: open(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200) failed: No such file or directory (read reason=) doveadm(user(a)example.com): Error: Mailbox INBOX: UID=6200: read(attachments-connector(/MailStore/mail/mailboxes/CFEE0E67-6269-4AD3-8DED-6AB71A8E4BD7/dbox/mailboxes/INBOX/dbox-Mails/u.6200)) failed: read(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200[base64:19 b/l]) failed: open(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200) failed: No such file or directory (read reason=) doveadm(user(a)example.com): Error: Mailbox INBOX: UID=6200: Failed to add attachment keywords: mail_get_parts() failed: Mailbox INBOX: UID=6200: read(attachments-connector(/MailStore/mail/mailboxes/CFEE0E67-6269-4AD3-8DED-6AB71A8E4BD7/dbox/mailboxes/INBOX/dbox-Mails/u.6200)) failed: read(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200[base64:19 b/l]) failed: open(/MailStore/mail/attachment/b9/52/b952b91441e53d3bb57453af2df66bb003e551f4-e94b170ff8bb685e943f0000162bf795-5d0b3129034c8957284d00007dfcf92d-6200) failed: No such file or directory (read reason=) 6200 error Assuming the attachments are still there, what would I need to do to 'fix' that? And if I can't fix it, can 'rebuild' skip the error and go on rebuilding the rest? TIA, p@rick -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG,80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief Aufsichtsratsvorsitzender: Florian Kirstein

2 1

Deliver log with "msgid=?" when email come from Exchange/Office 365
by Alessio Cecchi 27 Mar '20

27 Mar '20

Hi, I noticed that all emails that come from Exchange/Office 365 have the message-id, in deliver log, reported as msgid=? <DBBPRDD0(a)DBBPR023.eurprd08.prod.outlook.com> instead of expected msgid=<DBBPRDD0(a)DBBPR023.eurprd08.prod.outlook.com> My deliver_log format is: deliver_log_format = deliverytime=%{delivery_time}, msgid=%m, sender=%e, from=%f, subject="%s": %$ The behaviour is the same on Dovecot 2.2 and 2.3, and only dovecot have the "?", we have a mail filter before dovecot and the message-id is registred fine, without the ? in front. Is an issue of my setup or a bug in dovecot? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

1 0

Not able to start dovecot
by Adam Raszkiewicz 26 Mar '20

26 Mar '20

Freshly compiled dovecot 2.3.9.3 on Centos 7 for some reason not able to start sudo systemctl status dovecot.service ● dovecot.service - LSB: Starts the Dovecot POP3/IMAP service Loaded: loaded (/etc/rc.d/init.d/dovecot; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2020-03-18 16:32:17 UTC; 15s ago Docs: man:systemd-sysv-generator(8) Process: 2868 ExecStart=/etc/rc.d/init.d/dovecot start (code=exited, status=127) Mar 18 16:32:17 systemd[1]: Starting LSB: Starts the Dovecot POP3/IMAP service... Mar 18 16:32:17 dovecot[2868]: /etc/rc.d/init.d/dovecot: line 34: log_info_msg: command not found Mar 18 16:32:17 dovecot[2868]: /etc/rc.d/init.d/dovecot: line 35: start_daemon: command not found Mar 18 16:32:17 dovecot[2868]: /etc/rc.d/init.d/dovecot: line 36: evaluate_retval: command not found Mar 18 16:32:17 systemd[1]: dovecot.service: control process exited, code=exited status=127 Mar 18 16:32:17 systemd[1]: Failed to start LSB: Starts the Dovecot POP3/IMAP service. Mar 18 16:32:17 systemd[1]: Unit dovecot.service entered failed state. Mar 18 16:32:17 systemd[1]: dovecot.service failed. I have compiled the same code on Centos 6 and had no problem to run it. Any thoughts on that? Thanks, Adam

4 4

Understanding doveadm rebuild
by Patrick Ben Koetter 26 Mar '20

26 Mar '20

Greetings, I'm trying to understand if the doveadm rebuild command is suited to fix a problem we have on a customers platform. The customer uses sdbox in combination with SIS and for $REASONS dovecot cannot find the attachements anymore. They are there, but the references to them in the messages seem to be wrong. Is that what the rebuild subcommand has been made for? Will it e.g. scan the messages for attachment references and try to find them in the SIS storage location and then fix the references in messages? TIA, p@rick -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG,80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief Aufsichtsratsvorsitzender: Florian Kirstein

1 0