January 2019 - dovecot

Permission error on stats writer
by Matt Simpson 08 Jan '19

08 Jan '19

I’m running Dovecot 2.3.4 on FreeBSD 11.2 with qmail. In my mail log, I see messages like this on every delivery Jan 8 10:30:02 v1 qmail: 1546961402.314222 delivery 533: success: lda(matt,)Error:_net_connect_unix(/var/run/dovecot/stats-writer)_failed:_Permission_denied/did_0+0+1/ Searching for this error, I found a message on a support forum for a different server that suggested adding this to dovecot.conf: service stats { unix_listener stats-reader { user = vmail group = vmail mode = 0660 } unix_listener stats-writer { user = vmail group = vmail mode = 0660 } } The person reporting the error said that this corrected it. But since I’m running qmail, I have no vmail user/group. I’m guessing that I need to replace user/group with the values that some mail delivery process is running with. I’ve tried a couple of things with no success. Any ideas? $ doveconf -n # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 (60b0f48d) # OS: FreeBSD 11.2-RELEASE-p7 amd64 # Hostname: v1.redhorse.me auth_stats = yes auth_verbose = yes default_vsz_limit = 128 M lock_method = flock mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = prefix = } passdb { args = imap driver = pam } plugin { recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_pipe_exec_timeout = 10s sieve_plugins = sieve_extprograms } postmaster_address = postmaster(a)redhorse.me protocols = imap service auth { unix_listener auth-master { group = qnofiles mode = 0660 user = alias } user = root } service imap-login { process_min_avail = 3 vsz_limit = 94 M } ssl_cert = </usr/local/etc/letsencrypt/live/bitchpack.us-0001/fullchain.pem ssl_cipher_list = EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes syslog_facility = local0 userdb { driver = passwd } verbose_proctitle = yes protocol lda { mail_plugins = " sieve" } ls -l /var/run/dovecot total 20 srw------- 1 root wheel 0 Jan 8 10:29 anvil srw------- 1 root wheel 0 Jan 8 10:29 anvil-auth-penalty srw------- 1 dovecot wheel 0 Jan 8 10:29 auth-client srw------- 1 dovecot wheel 0 Jan 8 10:29 auth-login srw-rw---- 1 alias qnofiles 0 Jan 8 10:29 auth-master -rw------- 1 root wheel 32 Jan 8 10:29 auth-token-secret.dat srw-rw-rw- 1 dovecot wheel 0 Jan 8 10:29 auth-userdb srw------- 1 dovecot wheel 0 Jan 8 10:29 auth-worker srw------- 1 root wheel 0 Jan 8 10:29 config srw-rw---- 1 root dovecot 0 Jan 8 10:29 dict srw-rw---- 1 root dovecot 0 Jan 8 10:29 dict-async srw------- 1 root wheel 0 Jan 8 10:29 director-admin srw-rw-rw- 1 root wheel 0 Jan 8 10:29 dns-client srw------- 1 root wheel 0 Jan 8 10:29 doveadm-server lrwx------ 1 root wheel 35 Jan 8 10:29 dovecot.conf -> /usr/local/etc/dovecot/dovecot.conf drwxr-xr-x 2 root wheel 512 Jan 8 10:29 empty srw-rw---- 1 root dovecot 0 Jan 8 10:29 imap-hibernate srw------- 1 root wheel 0 Jan 8 10:29 imap-master srw-rw-rw- 1 root wheel 0 Jan 8 10:29 imap-urlauth srw------- 1 dovecot wheel 0 Jan 8 10:29 imap-urlauth-worker srw-rw-rw- 1 root wheel 0 Jan 8 10:29 indexer srw------- 1 dovecot wheel 0 Jan 8 10:29 indexer-worker srw------- 1 dovecot wheel 0 Jan 8 10:29 ipc srw------- 1 root wheel 0 Jan 8 10:29 log-errors drwxr-x--- 2 root dovenull 512 Jan 8 10:29 login srw------- 1 root wheel 0 Jan 8 10:29 master -rw------- 1 root wheel 6 Jan 8 10:29 master.pid srw------- 1 root wheel 0 Jan 8 10:29 old-stats prw------- 1 root wheel 0 Jan 8 10:29 old-stats-mail prw------- 1 root wheel 0 Jan 8 10:29 old-stats-user srw------- 1 root wheel 0 Jan 8 10:29 replication-notify prw------- 1 root wheel 0 Jan 8 10:29 replication-notify-fifo srw------- 1 dovecot wheel 0 Jan 8 10:29 replicator srw------- 1 root wheel 0 Jan 8 10:29 stats-reader srw-rw---- 1 root dovecot 0 Jan 8 10:29 stats-writer drwxr-x--- 2 root dovenull 512 Jan 8 10:29 token-login

2 1

Apple mail fails with Submission
by Ruud Voorjans 08 Jan '19

08 Jan '19

Dear all, I'm running dovecot # 2.3.2.1 - Pigeonhole version 0.5.2 () - OS: Linux 4.18.0-12-generic x86_64 Ubuntu 18.10 with Submission. It works great except with apple mail (Iphone). I get an error with the MTA (postfix): ""postfix/submission/smtpd[32552]: warning: non-SMTP command from mail.example.org[xx.xx.xx.xx]: Content-Transfer-Encoding: 7bit"" with other mail-client(s) (Outlook (Desktop and Iphone app)) i have no problem and it proxy-sends the e-mail beautiful out to the recipient. I think it is some kind of bug or maybe it just isn't supported (yet), cause as you said it is still "pretty basic". Best Regards

5 10

Fwd: Re: gcc -> clang
by Rupert Gallagher 08 Jan '19

08 Jan '19

The compiler returns many warnings, and the test returns two IPv6-related errors. I am attaching both logs as reference. > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Thursday, January 3, 2019 9:53 AM, Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > >> We compile all core code with both gcc and clang. What sort of interesting things did you find? >> >> Aki >> >>> On 03 January 2019 at 11:50 Rupert Gallagher via dovecot < dovecot(a)dovecot.org> wrote: >>> >>> Please, use clang instead of gcc. Code quality can only profit from it. I just compiled 2.3.4 and compiler stderr is full of interesting problems. >> >> --- >> Aki Tuomi

3 3

doveadm + HA
by Maciej Milaszewski IQ PL 07 Jan '19

07 Jan '19

Hi I have two server directors in ring and 5 dovecot servers (2.2.36) IP for IMAP and POP3 is a VIP (keepalived) What is the best solutions to get realy HA for 5 dovecot servers ? Maby corosync+pacemeker ? But this solution is too problematic and hardcore Why I need HA ? Doveadmin is too lazy and doveadm director does not know that one machine broke down and still sends traffic

2 1

Dovecot no logs by sieve user script
by Christian Wolf 07 Jan '19

07 Jan '19

Dear mailinglist, I have the problem that I had to migrate a setup and since then my user's sieve script does no longer log anything. I have sieve_user_log unset and sieve points to a folder (ready for ManageSievev service). In the old config I had sieve point to the symlink and sieve_dir point to the folder of all scripts. As sieve_dir is deprecated I use the sieve directive only (see below). According to my understanding of https://wiki2.dovecot.org/Pigeonhole/Sieve/ Configuration this should result in user logging in ~/.dovecot.sieve.log. This file is never generated by dovecot and if I create it manually, it is not filled. Can you tell me, what I am missing here in order to get the per-user logs running? Thank you very much Christian # dovecot --version # dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Mailbox managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = create special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster@wolf- stuttgart.net protocols = " imap sieve sieve" service auth { unix_listener /var/spool/postfix/ private/auth { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_cert = </etc/letsencrypt/live/imap/fullchain.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { mail_plugins = " sieve" }

2 3

special_use folders still created
by Marc Roos 07 Jan '19

07 Jan '19

dovecot-2.2.36-3.el7.x86_64 The below configuration seems to work for Thunderbird and emclient, when moving a message to Junk. Emclient uses "Junk E-mail" and Thunderbird uses "Junk". On the server is only the file .Junk created as expected. When I move a message to Archive in Thunderbird. On the server the files .Archives and .Archives.2018 are created (Thunderbird creates a subfolder in Archives) How is this possible? On the server there is already an .Archive folder so it should create .Archive.2018 not? imap_capability = +SPECIAL-USE # drafts folders merging mailbox Drafts { special_use = \Drafts auto = create } # archive folders merging mailbox Archive { special_use = \Archive auto = create } mailbox Archives { special_use = \Archive auto = no } # spam folders merging mailbox Junk { special_use = \Junk auto = create } mailbox Spam { special_use = \Junk auto = no } mailbox "Junk E-mail" { special_use = \Junk auto = no }

4 3

reprocessing mail
by Marc Lucke 06 Jan '19

06 Jan '19

Hey guys, I had some corruptions whereby dovecot would bomb out on mail files in my maildir. I found that if piped the files to dovecot-lda the all email was clean &lost. I even retrieved email I thought might have been lost for good. I have some better ideas now, but I still processed a few thousand emails like that now & would like to do something like this: * stop dovecot * iterate over each file ** get date ** touch date (linux - to set the date/time) ** rename the file with the correct epoch ** update dovecot-uidlist with correct epoch * start dovecot Will that work? Or is there something I've missed or should do? - Marc

1 0

Sieve "OOO" configuration
by Jerry 05 Jan '19

05 Jan '19

I am able to get sieve issuing an "out of office"message correctly. However, I want to configure it to send an "OOO" message only during certain dates, say on weekends, or only between certain hours. I can do that manually; however, was wondering if there is any automatic method available that could handle this chore. Thanks! -- Jerry

2 2

zlib - some emails maybe double or partial gzipped
by admin 05 Jan '19

05 Jan '19

Hi, I found these in my logs: Jan 5 15:58:56 mail dovecot: imap-login: Login: user=<xy(a)z.com>, method=PLAIN, rip=1.2.3.4, lip=2.3.4.5, mpid=5852, TLS, session=<K2oaPLd+885fWs6U> Jan 5 15:58:56 mail dovecot: imap-login: Login: user=<xy(a)z.com>, method=PLAIN, rip=1.2.3.4, lip=2.3.4.5, mpid=5854, TLS, session=<uxAjPLd+ys5fWs6U> Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=17247,W=17549:2,Sab,Z to /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4172:2,Sab,Z Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Corrupted index cache file /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/dovecot.index.cache: Broken physical size for mail UID 99 in mailbox Sent: read(zlib(/var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=17247,W=17549:2,Sab,Z)) failed: Cached message size larger than expected (17247 > 4268, box=Sent, UID=99) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Panic: file istream.c: line 175 (i_stream_read): assertion failed: (old_size <= _stream->pos - _stream->skip) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x95e92) [0x7f5068d3ee92] -> /usr/lib/dovecot/libdovecot.so.0(+0x95f8d) [0x7f5068d3ef8d] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5068cd4a91] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x290) [0x7f5068d49f20] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f5068d4a72d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x72) [0x7f5068d25d12] -> /usr/lib/dovecot/libdovecot.so.0(+0x76fd1) [0x7f5068d1ffd1] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f5068d49ce3] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f5068d4a72d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x72) [0x7f5068d25d12] -> /usr/lib/dovecot/libdovecot.so.0(+0x76fd1) [0x7f5068d1ffd1] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f5068d49ce3] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f5068d4a72d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f5068d29904] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x32f) [0x7f50690b52ef] -> dovecot/imap(+0x1ebbc) [0x556dd2cd7bbc] -> dovecot/imap(+0x1cfb6) [0x556dd2cd5fb6] -> dovecot/imap(imap_fetch_more+0x39) [0x556dd2cd70e9] -> dovecot/imap(cmd_fetch+0x33b) [0x556dd2cc8d9b] -> dovecot/imap(command_exec+0xa5) [0x556dd2cd4735] -> dovecot/imap(+0x199c2) [0x556dd2cd29c2] -> dovecot/imap(+0x19a4c) [0x556dd2cd2a4c] -> dovecot/imap(client_handle_input+0x1b5) [0x556dd2cd2e55] -> dovecot/imap(client_input+0x86) [0x556dd2cd33c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f5068d539f2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f5068d55029] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f5068d53a8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f5068d53c38] Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Fatal: master: service(imap): child 5852 killed with signal 6 (core dumps disabled) # and again: Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4172:2,Sab,Z to /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4268:2,Sab,Z Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Corrupted index cache file /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/dovecot.index.cache: Broken physical size for mail UID 254 in mailbox Sent: read(zlib(/var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4172:2,Sab,Z)) failed: Cached message size smaller than expected (4172 < 4268, box=Sent, UID=254) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Panic: file istream.c: line 212 (i_stream_read): assertion failed: (!i_stream_is_buffer_invalid(_stream)) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x95e92) [0x7f1dd9f6be92] -> /usr/lib/dovecot/libdovecot.so.0(+0x95f8d) [0x7f1dd9f6bf8d] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f1dd9f01a91] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x190) [0x7f1dd9f76e20] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f1dd9f7772d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f1dd9f56904] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_init_stream+0x388) [0x7f1dda2a7538] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x66fc0) [0x7f1dda25efc0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_stream_because+0x5d) [0x7f1dda22ee1d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x66d87) [0x7f1dda25ed87] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_virtual_size+0x33) [0x7f1dda22ea83] -> dovecot/imap(+0x1eae3) [0x5595eb2dfae3] -> dovecot/imap(+0x1cfb6) [0x5595eb2ddfb6] -> dovecot/imap(imap_fetch_more+0x39) [0x5595eb2df0e9] -> dovecot/imap(cmd_fetch+0x33b) [0x5595eb2d0d9b] -> dovecot/imap(command_exec+0xa5) [0x5595eb2dc735] -> dovecot/imap(+0x199c2) [0x5595eb2da9c2] -> dovecot/imap(+0x19a4c) [0x5595eb2daa4c] -> dovecot/imap(client_handle_input+0x1b5) [0x5595eb2dae55] -> dovecot/imap(client_input+0x86) [0x5595eb2db3c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f1dd9f809f2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f1dd9f82029] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f1dd9f80a8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f1dd9f80c38] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f1dd9f07fd3] -> dovecot/imap(main+0x328) [0x5595eb2cde68] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f1dd9b572e1] -> dovecot/imap(_start+0x2a) [0x5595eb2cdfea] Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Fatal: master: service(imap): child 5854 killed with signal 6 (core dumps disabled) Trying to read them with zcat they turned out to be binary. I figured out they might accidentally be compressed twice - they are. How could this happen? I used this script: dovecot-maildir-compress.sh (https://gist.github.com/cviebrock/e37f80a3223dfee19051c936c5ebe509) which appends an additional flag 'Z' after processing/conmpressong the mail and only compresses those without the flag: > find=$(find "$maildir" -type f -name "*,S=*" -mtime +30 ! -name "*,*:2,*,*Z*" -printf "%f\n") ... > if echo $flags | grep ','; > then > newname=$filename"Z" > else > newname=$filename",Z" which is actually neat. Now two possible things: (1) I am not quite sure yet but my best guess is that moving messages from one folder to another (via mail client) the message looses it's 'Z' flag. (2) compressing might have failed to rename but successfully compressed the file and the another consecutive run of the script compressed the file again. How could I recursively (a) check each maildir file if e.g. the output of(zcat $file | zcat) is ASCII or !binary (b) find all mails which are compressed - i.e. valid gzip - but miss the 'Z' flag ? (c) find all mails that are valid gzip but dont contain ASCII if / dont uncompress to mail ? There are backups in place for all files I get a error for, but I want to recursively fix the problem. Any hint? Thanks in advance! -M -- info(a)awib.it

1 0

Searching small(ish) sites on ubuntu
by Dirk Koopman 05 Jan '19

05 Jan '19

I run a small dovecot site with ~5 users, some of whom have many large mailboxes. I used to use lucene until the packagers decided that it broke some licencing rules and stopped including it. I have dismally failed to configure Solr, plus it is huge and potentially exposes too many services as well. Squash is deprecated (but still used?) in 2.2+. I am currently on 2.2.33. So what can I use? Or do I have to bite the bullet and install from source, thus losing the advantages of packaging and automatic updates? Dirk

1 0