August 2015 - dovecot

backing up email / saving maildir on external hard drives
by Kevin Laurie 15 Aug '15

15 Aug '15

Hello, Yesterday I tried to back up a 40GB maildir . I tried to move the maildir from home to external HDD but failed. Decided then to compress it(which took several hours). Now changing the disk format from FAT to exFAT to allow the transfer for the large compressed file. How does one back up emails on a external drive? Some advice would be greatly appreciated.

7 12

need help debugging deleted mails
by Vu Ngoc VU 13 Aug '15

13 Aug '15

hello, one of our users complained about the fact that he doesn't receive some messages (randomly) that his teamates have. Grepping for the message-ID in the mail.log, I can see lmtp says it is saved : Aug 12 15:23:23 imap-01 dovecot: lmtp(4943, johndoe(a)example.net) fFuCO09Hy1VPEwAA2t/0+w: sieve: vsize= msgid=<037e01d0d502$0e3687d0$2aa39770$(a)example.net>: stored mail into mailbox 'INBOX' The only difference with others recipients is the 'sieve:... stored mail' instead of 'saved mail to' when there is no sieve. But when he asked me, the message was no more in his maildir (I checked with doveadm search). As we have checked and disabled all the filters in his MUA and on the server (sieve) and this issue still continue, I did mkdir a dovecot.rawlog inside his home. With the help of rawlog, I can see that the message can be found in a .out file (by grepping the Message-ID). And by reading this .out log raw log, the entire message were sent to the client. So I guess that : * the message was written to disk * it was sent to the MUA But I need your help because I don't get how this message disappear from his maildir. My assumptions was that the client must do some commands like : * STORE the \Deleted flag to this message UID * and then, later do EXPUNGE but I did : * grep -i expunge * 20150812* => no match * grep -iE '\+flags .*\\deleted' => a few matches but not with the appropriate UID snippet from the 20150812-133858-13317.in file: 8nk1 UID FETCH 208209 (UID FLAGS RFC822.SIZE BODY.PEEK[HEADER] INTERNALDATE) p7uv IDLE DONE 6yk3 NOOP 83g6 IDLE DONE yhbd UID FETCH 208209 (UID FLAGS BODY.PEEK[]) fy16 IDLE DONE snippet from the 20150812-133858-13317.out file: * 56295 FETCH (UID 208209) wn2p OK Fetch completed. * 56295 FETCH (UID 208209 FLAGS (\Deleted \Seen \Recent) RFC822.SIZE 17138 INTERNALDATE "12-Aug-2015 15:23:22 +0200" BODY[HEADER] {2836} Considering these points: * I cannot find any match with 208209 early in raw logs * I don't see any expunge for the whole day * I've grepped for \Deleted (insensitive) and didn't find matching like a interval of messages UID. What did I miss to find out how/when the message got \Deleted flag and was removed from his maildir? A few about my setup : * system is ubuntu 14.04 inside OpenVZ containers (Proxmox) * dovecot version is 2.2.9 from Ubuntu packages * 2 director with Internet and local ip addresses * 2 backends with only local addresses * these dovecot access storage servers by NFSv3 (that's why we have director) * imap/lmtp are proxied Thanks a lot for reading, I'm really disappointed about this problem. For the moment, I'll delete his script even if all rules are disabled (via roundcube, and I checked the rule is commented in the file). And also I'll try to watch what happens with inotify on the storage server.

4 6

backing up IMAP server on a hard drive
by Kevin Laurie 13 Aug '15

13 Aug '15

Hi, I am trying to back up my IMAP server to a hard drive. Later I intend to extract all mails for attachments. What do you reckon is the best too to perform this ? Imapsync or Thunderbird (or something else, please recommend) One problem I am having with imapsync is the setting for host2 (being a localhost computer). How does one set a parameter for host2 being a desktop computer. Thanks Kevin

3 6

Error when reading attachment - one byte overhead?
by Ачилов Рашид Нурмухамед ович 13 Aug '15

13 Aug '15

After converting maildir to mdbox mail periodically obtained these messages: Aug 13 11:22:27 mailsrv dovecot: imap(user-aaa): Error: read(/usr/share/smbusers/user-aaa/mail/attachments/a0/8e/a08e3e63e11aae3f37020bdca0da7d-7b39ab07ea73ba559d450000dec08558[base64:18 b/l]) failed: Stream is larger than expected (434427 > 434426, eof=1) Aug 13 11:22:27 mailsrv dovecot: imap(user-aaa): Error: read(attachments-connector(/usr/share/smbusers/user-aaa/mail/storage/m.23)) failed: read(/usr/share/smbusers/user-aaa/mail/attachments/a0/8e/a08e3e63e11aae3f37020bdca0da7d-7b39ab07ea73ba559d450000dec08558[base64:18 b/l]) failed: Stream is larger than expected (434427 > 434426, eof=1) (FETCH BODY[] for mailbox Разное полезное/Для памяти UID 20) Aug 13 11:22:27 mailsrv dovecot: imap(user-aaa): Disconnected: FETCH read() failed in=16923 out=497431 How to correct it? Conversion was did with 2.2.8 version, next will installed 2.2.18. -- С Уважением. Ачилов Рашид Нурмухамедович (AchilovRN(a)granch.ru) Главный специалист по защите информации ООО НПФ "Гранч", тел: +7 (383) 233-35-12, доб. 107

1 0

SiS does not work
by Ачилов Рашид Нурмухамед ович 13 Aug '15

13 Aug '15

How do you do. I have tried to use SiS to deduplicate attachments in mail storage. Unsuccesful. When goes mail with reasonable size, and SiS try to start, I have receive error, when this mail must be delivered for group of receivers (i.e. through aliases). Below is error log, valuable parts of mail addresses masked by X and Y chars Aug 6 16:11:12 mailsrv dovecot: lmtp(xxxxx): AL1SNrkyw1VPBAAABJcpmA: msgid=<2015080614471325760993(a)chinacoalchem.com> (from zsy(a)chinacoalchem.com) saved mail to INBOX This receiver has received mail. But next one does not. And to receive I must restart _sendmail_ (not dovecot!). I must restart sendmail as many times as there were recipients in envelope. Aug 6 16:11:12 mailsrv dovecot: lmtp(yyyy): Error: read(/usr/share/smbusers/xxxxx/mail/attachments /e4/3d/e43db6e8af6fb1df4a08f9c1f3f31e93e61781c4c1d2395e04dd96a583aa-01bd5236b932c3554f04000004972998[base64 :19 b/l]) failed: Stream is smaller than expected (0 < 4080128) Aug 6 16:11:12 mailsrv dovecot: lmtp(yyyy): Error: read(attachments-connector(/usr/share/smbusers/xxxxx/mail/storage/m.35)) failed: read(/usr/share/smbusers/xxxxx/mail/attachments/e4/3d/e43db6e8af6f b1df4a08f9c1f3f31e93e61781c4c1d2395e04dd96a583aa-01bd5236b932c3554f04000004972998[base64:19 b/l]) failed: Stream is smaller than expected (0 < 4080128) Aug 6 16:11:12 mailsrv dovecot: lmtp(yyyy): Error: copy: i_stream_read(attachments-connector(/usr/share/smbusers/xxxxx/mail/storage/m.35)) failed: read(/usr/share/smbusers/xxxxx/mail/attachments/e4/ 3d/e43db6e8af6fb1df4a08f9c1f3f31e93e61781c4c1d2395e04dd96a583aa-01bd5236b932c3554f04000004972998[base64:19 b/l]) failed: Stream is smaller than expected (0 < 4080128) Aug 6 16:11:12 mailsrv dovecot: lmtp(yyyy): Error: read(attachments-connector(/usr/share/smbusers/xxxxx/mail/storage/m.35)) failed: read(/usr/share/smbusers/xxxxx/mail/attachments/e4/3d/e43db6e8af6f b1df4a08f9c1f3f31e93e61781c4c1d2395e04dd96a583aa-01bd5236b932c3554f04000004972998[base64:19 b/l]) failed: Stream is smaller than expected (0 < 4080128) (uid=0, box=INBOX) Aug 6 16:11:12 mailsrv dovecot: lmtp(yyyy): AL1SNrkyw1VPBAAABJcpmA: msgid=<2015080614471325760993(a)chinacoalchem.com> (from zsy(a)chinacoalchem.com) save failed to INBOX: Internal error occurred. Refer to server log for more information. [2015-08-06 16:11:12] Why LMTP, when try to deliver mail to local user yyyy tries to look up into a home directory of local user xxxxx? All these and similar home directories has a 'mail' subdir, which keeps mail and rights on this subdir is 0700. To avoid this completely misunderstood occassion, I have temporary set mail_attachment_min_size to 2048kb - but now I cannot receive advantage from SiS. -- С Уважением. Ачилов Рашид Нурмухамедович (AchilovRN(a)granch.ru) Главный специалист по защите информации ООО НПФ "Гранч", тел: +7 (383) 233-35-12, доб. 107

2 2

userdb / user map with director
by Will Yardley 10 Aug '15

10 Aug '15

This is with dovecot-ee 2.2.18.2 on RHEL6.... To handle backend and frontend on same machine, I'm using the following hack, as previously outlined here: [in director instance config] passdb { # See thread ending in: # http://dovecot.org/pipermail/dovecot/2012-June/083817.html # for why this has to be sql instead of 'static' driver = sql args = /etc/dovecot/dovecot-sql.conf } [in /etc/dovecot/dovecot-sql.conf] driver = sqlite connect = /etc/dovecot/empty.db password_query = select 'y' as proxy, \ NULL as password, \ 'y' as nopassword, \ case '%a' \ when '110' then '10110' \ when '995' then '10110' \ when '143' then '10143' \ when '993' then '10143' end \ as port; This works, but I was getting a failure on usernames, since director instance didn't have a userdb. So I tried the following: [in config] passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } userdb { driver = prefetch } and appended the following to the SQL query: password_query = select '%u' AS user, \ Then I see the following error in the log when I try to run the map "Trying to iterate users, but userdbs don't support it" and doveadm -i director director map returns: Error: User listing returned failure Error: user listing failed [...] I'm sure I'm missing something obvious, so is there an easy fix to make this mapping work? w

1 0

Dovecot sends "OK Logged in" with empty tag
by Simon Thelen 10 Aug '15

10 Aug '15

Hello, I'm currently writing my own IMAP client and testing it against Dovecot, and when doing PLAIN AUTH Dovecot sends the "OK Logged in" with an empty tag at the start. I tried reproducing the conversation with netcat, but Dovecot sends a correctly tagged OK at the end of the AUTH so I assume it's timing related. I've attached pcap dumps of the TCP conversation in both situations as well as the output of dovecot -n. Side Note: Why does Dovecot send the updated capabilities before OK'ing the authentication? Seems strange to me as the client can't assume that the authentication proceeded successfully until it actually receives the OK so it has to assume that those capabilities are pre login. -- Simon Thelen

1 1

Using a separate passdb per service
by Gerry 10 Aug '15

10 Aug '15

Situation: one front-facing server running Dovecot as IMAP/POP3/ ManageSieve proxy, a mixture of IMAP servers (Dovecot, Exchange, ...) in the back-end. Dovecot's passdb does lookups against MySQL which contains a simple user/host mapping, the actual authentication happens on the back-end IMAP servers. The configuration is more or less as described here: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy Now I would like to add a Postfix instance on the front-facing server which listens on the submission port and authenticates users via SASL using the local Dovecot's UNIX socket. The idea being that a user only needs to remember one single hostname, one username and one password for all mail-related services. The problem is that Dovecot is operating in proxy mode, which means that the password_query returns NULL as the password and explicitly returns a field "nopasswd" containing "Y". Thus, users can not authenticate against the UNIX socket. What I think I want to do is convince Dovecot to use one passdb for the imap/pop3/managesieve services and different one for the "auth" service. The configuration snippet below doesn't work, but it should illustrate what I want to achieve: > protocols = imap pop3 sieve > > service auth { > passdb sql { > driver = sql > args = /etc/dovecot/mysql-auth-sasl.conf.ext > } > > unix_listener /var/spool/postfix/private/auth { > user = postfix > group = postfix > mode = 0666 > } > } > > # IMAP/POP3/ManageSieve auth against MySQL > passdb sql { > driver = sql > args = /etc/dovecot/mysql-auth-default.conf.ext > } Example mysql-auth-sasl.conf.ext > driver = mysql > connect = host=127.0.0.1 dbname=mail user=mail password=somethingrandom > password_query = SELECT password AS password FROM users WHERE login = '%u' Example mysql-auth-default.conf.ext: > driver = mysql > connect = host=127.0.0.1 dbname=mail user=mail password=somethingrandom > password_query = SELECT NULL AS password, 'Y' as nopassword, host, 'Y' AS proxy FROM users WHERE login = '%u' Any pointers? Gerry

3 4

How to import an mbox to an existing user as a new mailbox
by Paul Hoffman 09 Aug '15

09 Aug '15

Greetings. I have a bunch of mbox files that I want to import to an existing user, each into a new mailbox. That is, I want to import the file project1.mbox to user Foo, into a newly-created mailbox called "Project 1". Is there a dovecot tool to do this on the server? --Paul Hoffman

4 10

Trying to back
by Kevin Laurie 09 Aug '15

09 Aug '15

Hi, I am trying to backup multiple mail accounts by using imapsync and Dovecot on my local machine. I am using maildir as a mail storage format and currently have the following setting on my local machine :- mail_location = maildir:~/Maildir I am afraid that if I imapsync multiple accounts they will all get stored in the same location. Is there a way to seperate the accounts based on the email address? Appreciate if someone could advise. Thanks! Kevin

3 3