- dovecot - dovecot.org

Errors with missing links to files when using Single Instance Storage and zlib (Dovecot 2.3.4)
by Daniel Schütze 18 Dec '18

18 Dec '18

I'm running Dovecot 2.3.4 with Single Instance Storage (SIS) and zlib and I am frequently seeing errors where files are missing in the attachments directory even though the zipped file in the hash directory is actually there. This appears not to have been an issue in Dovecot 2.3.1 Requested output of the server below. The error looks like this in the log Dec 18 08:58:58 dovecot01 dovecot: imap(username)<3692><RKnPGEh9fC3AqAAB>: Error: Mailbox INBOX/FOLDER: UID=41: read(attachments-connector(zlib(/usr/home/vmail/mail/username//mdbox/storage/m.411))) failed: read(/usr/home/vmail/attachments/91/08/91085773a0774909207852b8f9c98dbe22fe5a31-e88ba0276aae135cd2050000d09efc50[base64:19 b/l]) failed: open(/usr/home/vmail/attachments/91/08/91085773a0774909207852b8f9c98dbe22fe5a31-e88ba0276aae135cd2050000d09efc50) failed: No such file or directory (FETCH BODY[]) I'm also seeing errors when replication fails dovecot: dsync-local(username)<bzg3MqjhGFxmFwAAzRvpBw>: Error: dsync(servername): read(attachments-connector(zlib(/usr/home/vmail/mail/username//mdbox/storage/m.413))) failed: read(/usr/home/vmail/attachments/45/42/4542dcf385f6b4a4dc8752b96db1d7ad8a8023d5-ac735007f6bd135c93440100cd1be907[base64:19 b/l]) failed: open(/usr/home/vmail/attachments/45/42/4542dcf385f6b4a4dc8752b96db1d7ad8a8023d5-ac735007f6bd135c93440100cd1be907) failed: No such file or directory (last sent=mail, last recv=mail (EOL)) It is not the case that every replication fail has a corresponding imap access fail. I am running Dovecot 2.3.4 on FreeBSD11.2 and I did not see this error on Dovecot 2.3.1 FreeBSD 10.4. Both installs had the same configuration for Dovecot. Both installs were made from source code and not packages (as the need for the non standard fts and mysql options). During my testing I had the 2.3.1 install receive mail and replicate to the 2.3.4 and that was working fine (except for tcp connection issues which I saw was fixed in the changelog). Unfortunately having gone into testing with real account and mail being sent to the 2.3.4 install I'm seeing the above errors. My clients are using Thunderbird on desktops and Roundcube for webmail. My users report they can see the e-mails fine in their webmail (Roundcube) and only Thunderbird is giving issues. I admit I don't know how this can be the case as both systems use the same account details. In everycase so far there is a file in the hashes directory I have hand crafted the missing file (i.e. link) using ln and this restores access in Thunderbird. Clearly this is an big issue for my install and I'd appreciate any help! At the moment it looks like my course will be to try and script creation (recreation?) of the links from the error logs (I've seen people talking about this online) but that isn't really what I'm after. DOVECOT CONFIG # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 (60b0f48d) # OS: FreeBSD 11.2-RELEASE-p5 amd64 # Hostname: dovecot01.cwa.uk.com dict { acl = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 145 last_valid_uid = 145 mail_access_groups = mail mail_attachment_dir = /usr/home/vmail/attachments mail_attachment_min_size = 64 k mail_gid = vmail mail_location = mdbox:~/mdbox:INDEX=/indexdisk/indexes/%n mail_plugins = " fts fts_solr acl zlib notify replication" mail_privileged_group = mail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = yes location = mdbox:%%h/mdbox:INDEXPVT=~/mdbox/shared/%%n prefix = shared/%%n/ separator = / subscriptions = yes type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl fts = solr fts_autoindex = yes fts_solr = url=http://localhost:8983/solr/dovecot/ mail_replica = tcp:192.168.0.138:12345 replication_dsync_parameters = -d -l 30 -U sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_duplicate_default_period = 1h sieve_duplicate_max_period = 1d sieve_global_dir = /usr/home/vmail/sieve/global/ sieve_global_path = /usr/home/vmail/default.sieve sieve_vacation_default_period = 1d zlib_save = gz zlib_save_level = 6 } protocols = imap sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { mode = 0777 } } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = 12345 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " fts fts_solr acl zlib notify replication sieve" submission_host = 192.168.0.1:25 } protocol imap { mail_plugins = " fts fts_solr acl zlib notify replication imap_acl imap_zlib" }

1 0

Change default mode for attachment files
by Олег Кривоносов 18 Dec '18

18 Dec '18

Hi I migrated my dovecot server from Maildir to mdbox with saving attachments to external files and enabled SIS a few days ago. Everything works fine except one issue. I have both virtual and system users. Dovecot saves attachments with mode 0600. So I catch such errors for system users: dovecot: lda(wasya): Error: fs-sis: Couldn't read hash file /var/mail/attachments/2e/dc/hashes/2edc6009bc9b6ba38febb581df03986dcaf134d9: Permission denied I want to fix this issue by changing default file mode to 0660 via group policy. I found such feature in the changelog (v2.1.rc2 2012-01-06): lib-fs: Added "mode" parameter to "posix" backend to specify mode for created files/dirs. Unfortunately I can't find any information about this feature in the dovecot documentation. Can I use it to setup mode 0660 for attachments in the dovecot configuration? uname -a Linux diplo.trueconf.ru 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux dovecot --version 2.2.27 (c0f36b0) 10-mail.conf dovecot configuration: mail_access_groups = vmail mail_location = mdbox:~/mdbox mdbox_preallocate_space = yes mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 5 M mail_attachment_dir = /var/mail/attachments mail_attachment_min_size = 32k mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} Thank you

1 0

Possible attack?
by Daniel Miller 18 Dec '18

18 Dec '18

I found an error in my log today... Dec 17 12:03:30 bubba dovecot: imap(user1(a)amfes.com)<23017><VzQFnjx9WNAKO2EC>: Error: fts_solr: received invalid uid '0' Dec 17 12:04:44 bubba dovecot: imap(user2(a)amfes.com)<25004><FeHDSj19i2ysOCn7>: Fatal: master: service(imap): child 25004 killed with signal 11 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) I've now enabled core dumps (I think) and restarted - if it comes back hopefully I can get a backtrace. But reading that fts_solr message, and some other comments, leads me to wonder - could this be caused by someone/thing trying to authenticate as root? On that theory - I tried doing so via telnet - and received: Dec 17 15:06:02 bubba dovecot: auth: Error: plain(ultradeitytypeperson(a)amfes.com,127.0.0.1,<4kQr0z99UMZ/AAAB>): user not found from any userdbs Dec 17 15:06:02 bubba dovecot: imap: Error: Authenticated user not found from userdb, auth lookup id=3522297857 (auth connected 1 msecs ago, handshake 0 msecs ago, request took 1 msecs, client-pid=29572 client-id=1) I have root's email aliased to a valid user's email. I'm not sure how I'm able to authenticate as root - there isn't a root user defined in my LDAP database and that should be the only auth backend enabled for Dovecot. Or do I need to explicitly block local users from /etc/passwd on the server? The only auth databases shown in doveconf -n: userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } and "master-users" doesn't list root either. -- Daniel

1 0

Re: ECDSA client question
by Michael A. Peters 18 Dec '18

18 Dec '18

On 12/16/18 7:52 AM, Tributh via dovecot wrote: > > > Am 16.12.18 um 12:13 schrieb Michael A. Peters: >> Hi, for those who have adopted ECDSA, >> >> Are there still any commonly used IMAPS/POP3S clients that still can not >> handle ECDSA certificates? >> >> I know you can set up Dovecot dor dual cert, I am just trying to >> determine if there still is a real world need to. > > Nearly every client can handle ECDSA, but it depends on the size of the > certificate. > I used years ago ECDSA-384bit certificates, which covered most of the > clients. It came to the point to disable RSA in that time, but than came > Android7.0. This Version can only handle ECDSA-256bit certificates or RSA. > > The coverage of Android7.0 is still over 20%. Google reacted fast and > repaired this bug in 7.1, which is still not coming to most of the phones. > > Cheers > Torsten > Wow - My phone is running Android 6, I just checked with Dad - his phone (Motorola) is running Android 7.0 - the version with the bug. We don't replace phones just because new versions are available, we replace them when they stop working, and when we do we usually get refurbished because we hate how much electronic waste is in the world. I have to admit, the tin foil hat of mine just got an alert. We know there are unexplained constants in the NIST curves including P-256 - what if NSA was partially responsible for this bug (back room deal to avoid anti-trust prosecution, similar deal with IBM was made in the 70s I believe also involving cryptography) so that Android apps that use ECDSA (beyond just the mail client, e.g. chat apps) would use P-256 for compatibility and are maybe vulnerable to MITM for the key exchange. I want Ed25519 now.

2 1

Overrideing pop delete?
by ＠lbutlr 17 Dec '18

17 Dec '18

Is it possible to override the POP3 delete on download command and make sure that messages stay on the server for at least X hours or X days? It is important that the messages be around long enough to hit a snapshot cycle (using rsnapshot to backup ever hour). -- Greedo didn't shoot first, motherfucker!

3 7

Bug in IDLE implementation for virtual mailbox
by Pali Rohár 17 Dec '18

17 Dec '18

Hello! I found bug in Dovecot's IDLE implementation when virtual mailbox is in use. IDLE does not notify about new emails when email appears in newly created mailbox and IDLE was issued in virtual folder which matches "*" wildcard and that mailbox was created after opening virtual mailbox. This problem is present in version 2.2.34 which is available in Debian Stretch system with backports. Reproducer: 1. Create virtual "All" mailbox in Gmail-like style: $ cat All/dovecot-virtual * -Drafts -Spam -Trash ALL 2. Ensure that (real non-virtual) mailbox "Test" does not exist. 3. Create sieve rule which deliver some email to "Test" mailbox. require [ "fileinto", "mailbox" ]; ... fileinto :create "Test"; 4. Open dovecot imap connection, select virtual "All" mailbox and then enter IDLE command. 4. Send email which matches that sieve filter. Email will be processed by dovecot sieve. As "Test" mailbox does not exist, dovecot sieve would first create it and then deliver email into it. Now dovecot imap should notify IDLE client that new message into virtual "All" mailbox was delivered (as it collects emails from all - * - mailboxes). But dovecot for some unknown reason does not notify via IDLE that there is a new email. To get notifications, it is needed to re-open that "All" mailbox again. -- Pali Rohár pali.rohar(a)gmail.com

2 4

Downgrade dovecot if required
by Durga Prasad Malyala 17 Dec '18

17 Dec '18

Hello all, I have version 2.3.3-2 of dovecot and could see version 2.3.4-2 available from the repos. How is feedback after upgrade to 2.3.4-2? Any issues? Can I revert back to older version if I face any problems? Cheers/DP

2 1

ssh_dh?
by Daniel Miller 17 Dec '18

17 Dec '18

Don't know if this was corrected in 2.3.4 (haven't upgraded yet but didn't see it in the notes) - but in 2.3.3 I see this in my log: imap-login: Error: Diffie-Hellman key exchange requested, but no DH parameters provided. Set ssh_dh=</path/to/dh.pem So...either there's an undocumented feature of SSH-over-IMAP (that's Dovecot - always on the cutting edge!) or someone had a coffee shortage during a coding session... -- Daniel

5 5

Sieve scripts not backed up
by Marc Weustink 17 Dec '18

17 Dec '18

Hi all. I try to backup out sieve scripts through doveadm -o stats_writer_socket_path= backup -R -u marc -N tcp:<remote.host>:4191 All mail is backed up but no scripts. What am I doing wrong ? Thanks Marc doveconf -n # 2.3.4 (0ecbaf23d): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 (60b0f48d) # OS: Linux 4.4.0-137-generic x86_64 Ubuntu 16.04.5 LTS autofs # Hostname: xxx dict { acl = pgsql:/etc/dovecot/dovecot-dict-sql-local.conf.ext } doveadm_password = # hidden, use -P to show it mail_debug = yes mail_gid = vmail mail_home = /home/mail/backup/%n mail_location = maildir:/home/mail/backup/%n/mail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace Shared { location = maildir:/home/mail/backup/Common/mail:INDEXPVT=/home/mail/backup/%n/mail/common prefix = Shared/ separator = / subscriptions = no type = public } namespace Users { list = children location = maildir:/home/mail/backup/%%n/mail:INDEXPVT=/home/mail/backup/%n/mail/shared/%%n prefix = Users/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = sieve service auth { unix_listener auth-userdb { mode = 0777 } } service dict { unix_listener dict { group = vmail user = vmail } } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql }

2 1

Upgrade to 2.3.1 has failed
by C. Andrews Lavarre 17 Dec '18

17 Dec '18

Problem: We had Dovecot v2.2 working just fine under openSUSE Leap 42.3. But we upgraded openSUSE to Leap 15.0. In the process, Dovecot got upgraded from 2.2 to 2.3.1. It no longer works and I haven't figured out how to downgrade to the older working version. The key issue seems to be the change to requiring dh.pem and changing s sl_protocols to ssl_min_protocols. I think I've navigated both correctly, but it still doesn't work. The error is auth: Error: stats: open(old-stats-user) failed: Permission denied as a consequence of which we get imap-login: Error: Failed to initialize SSL server context: Can't load SSL certificate: There is no valid PEM certificate. We have followed the instructions at https://wiki.dovecot.org/S SL/DovecotConfiguration 1. We have created /etc/dovecot/dh.pem (yes it took five hours) 2. We have edited 10-ssl.conf as directed by the Wiki: ssl = yes ssl_cert = /etc/certbot/live/privustech.com/fullchain.pem ssl_key = /etc/certbot/live/privustech.com/privkey.pem ssl_dh = /etc/dovecot/dh.pem #(yes, it took five hours to create...) ssl_min_protocol = TLSv1 ssl_cipher_list = ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH ssl_prefer_server_ciphers = no 3. We have checked 10-ssl.conf against the 2.3 default at https://github.com/dovecot/core/blob/master/doc/example-config/conf.d/10-ss… 4. We do NOT include the less than (<) symbol before the paths because then dovecot fails to load complaining it cannot find the files. 5. we have checked all the pem keys, certificates, and dh files with cat, they all exist and are in the expected hash format. 6. We have followed the instructions to set their permissions root:root 0444 and 0400 accordingly. 7. We have rebooted the host. Any help or clues would be most appreciated. Kind regards, Andy

7 20