August 2013 - dovecot

[Dovecot] Logging pre-rewrite user ID for application-specific passwords
by Steven Murdoch 22 Sep '13

22 Sep '13

I have set up Dovecot v2.2.5 with application-specific passwords, along the lines of this blog post (http://www.dgsiegel.net/news/2013_05_21-application_specific_passwords_for_…) My users file looks like: user-foo:{BLF-CRYPT}$2...:42:42::/home/user::allow_nets=127.0.0.1/32 user=user user-bar:{BLF-CRYPT}$2...:42:42::/home/user::user=user user:{BLF-CRYPT}$2...:42:42::/home/user::nologin In this way, user-foo can log in with one password (but only from localhost), user-bar can log in from any host with a different password, but both actually being treated as the same user. When these users log in through, the log entries look like: Aug 26 23:03:01 hostname dovecot: imap-login: Login: user=<user>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=12345, TLS Whereas I would like to see the original user name (prior to the re-write), i.e. user-foo and user-bar so I can keep track of which user names are in use and from where. Is there some way to do this? I couldn't see any of the variables being suitable. There is some information in the debug logs, but I would prefer not to need to leave this on when I don't need them. Thanks in advance. Steven

2 1

[Dovecot] Dovecot never release preallocated space in mdbox
by Stéphane BERTHELOT 22 Sep '13

22 Sep '13

Hello everyone, I am currently evaluating dovecot for our new email production servers (20k+ mailboxes) and found out something strange. I'm using those settings on Dovecot 2.2.4 (x86_64 / Slackware / compiled from sources) mdbox_rotate_size = 128M mdbox_rotate_interval = 1d mdbox_preallocate_space = yes with virtual users and location like : mail_location = mdbox:~/mdbox I don't think the remaining config is relevant but ask me if you need some other parts. Using test accounts for 2 weeks now I've figured that the 128M preallocated space is never 'hole punched" (to use a similar term than "man fallocate" on Linux), even when rotating m.* files. From what I understand those files will never be appended again because of the mdbox_rotate_interval. Then doveadm purge creates new files so old ones would never grow again. Here is an example of a mdbox storage using ls -ls (which shows allocated VS used space) total 4065176 1884 -rw------- 1 mail mail 1926656 Jul 29 10:55 dovecot.map.index 4 -rw------- 1 mail mail 460 Jul 29 11:26 dovecot.map.index.log 48 -rw------- 1 mail mail 44304 Jul 29 10:55 dovecot.map.index.log.2 131072 -rw------- 1 mail mail 133165066 Jul 19 15:31 m.10 131072 -rw------- 1 mail mail 133507393 Jul 19 15:32 m.13 131072 -rw------- 1 mail mail 134155182 Jul 19 15:33 m.14 131072 -rw------- 1 mail mail 134213403 Jul 19 15:30 m.2 131072 -rw------- 1 mail mail 46464 Jul 21 04:30 m.21 131072 -rw------- 1 mail mail 134215030 Jul 19 15:30 m.3 131072 -rw------- 1 mail mail 25852 Jul 25 01:54 m.32 131072 -rw------- 1 mail mail 2360 Jul 26 00:05 m.34 131072 -rw------- 1 mail mail 169073 Jul 27 23:18 m.35 131072 -rw------- 1 mail mail 31624 Jul 27 01:55 m.36 131072 -rw------- 1 mail mail 134216982 Jul 28 04:30 m.37 131076 -rw------- 1 mail mail 134217804 Jul 28 04:30 m.38 131072 -rw------- 1 mail mail 134217341 Jul 28 04:30 m.39 131072 -rw------- 1 mail mail 134213719 Jul 19 15:30 m.4 131072 -rw------- 1 mail mail 29740970 Jul 28 04:30 m.40 131072 -rw------- 1 mail mail 129175917 Jul 28 04:30 m.41 131072 -rw------- 1 mail mail 133174937 Jul 28 04:30 m.42 131072 -rw------- 1 mail mail 633436 Jul 28 04:30 m.43 131072 -rw------- 1 mail mail 3154623 Jul 28 04:30 m.44 131072 -rw------- 1 mail mail 3676879 Jul 28 04:30 m.45 131072 -rw------- 1 mail mail 468158 Jul 28 04:30 m.46 131072 -rw------- 1 mail mail 26964 Jul 28 04:30 m.47 131072 -rw------- 1 mail mail 3574599 Jul 28 04:30 m.48 131072 -rw------- 1 mail mail 3789133 Jul 28 04:30 m.49 131072 -rw------- 1 mail mail 134215016 Jul 19 15:30 m.5 131072 -rw------- 1 mail mail 1280074 Jul 28 04:30 m.50 131076 -rw------- 1 mail mail 635459 Jul 28 22:47 m.51 131072 -rw------- 1 mail mail 1459418 Jul 29 10:55 m.52 131072 -rw------- 1 mail mail 132941013 Jul 29 11:26 m.53 131072 -rw------- 1 mail mail 134213475 Jul 19 15:30 m.7 131072 -rw------- 1 mail mail 132240074 Jul 19 15:31 m.9 There's a lot of "lost" space since preallocated space would only be reclaimed when *all* emails in m.X file have refcount=0 and after a doveadm purge call, if I read well the dovecot docs. On mailboxes patterns with low incoming mail (< 100kb / day) this would waste much space. Of course I can decrease rotate size a lot but it would then produce a lot of files and would certainly become similar performance-wise to sdbox/maildir/... There would certainly be smart to use something similar to "FALLOC_FL_PUNCH_HOLE" on rotation (when doing close() ?) so that when we're sure there won't be anymore data appended to file that the allocated space == used space. I will disable space preallocation for our next tests since it wastes much storage for us ; did you have any feedback on how much it may affect performance ? I found in this ML archives some messages about the implementation but didn't see anyone clearly stating how much better preallocation is. Thanks, best regards, Stephane Berthelot.

4 5

[Dovecot] Deleted mails in dsync backups
by Wolfgang.Friebel＠desy.de 22 Sep '13

22 Sep '13

Hello we are doing regular backups of the mdbox mail folders using dsync -u username backup mdbox:/somedir/BACKUPS/username (still with version 2.1.10, but 2.2.5 is in testing phase) We do observe that the directory size in the BACKUPS directory is growing with respect to the original mail folders. I believe this is due to deleted mails not being purged in the backup. We are doing regular doveadm purge -A cleanups, but this does (of course) not affect the backups and subsequent dsync backup runs seem not to clean up deleted mails in the backup either. Will the behaviour of the dsync backup command deal with deleted mails in the 2.2.5 release? If not, what could we do to get rid of the deleted mails in the backup (or is there another reason for the growth in size) We have e.g. after 9 month of running with 2.1.10 du -sk username 588520 du -sk /somedir/BACKUPS/username 863024 -- Wolfgang Friebel Deutsches Elektronen-Synchrotron DESY

2 1

[Dovecot] Rfc: Limit the size of an email IN the mailbox
by Ralf Hildebrandt 22 Sep '13

22 Sep '13

Currently we're in the process of migrating Dovecot mailboxes to Exchange. This is all working very well, except for the occasional user, who used his/her Drafts folder as a file storage: They started writing an email containing lots of image attachments (and thus exceeding our maximum_message_size by far). So the mail never gets sent, but stays in the Drafts folder. Once I try moving that mail to exchange, it won't accept the message since it's too big. I think it would be interesting to have a limit on the total size of an IMAP object, just like in SMTP. Thought? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 2

[Dovecot] INBOX protected foldername?
by Hajo Locke 22 Sep '13

22 Sep '13

Hello, have a little problem with folders with name INBOX created by Users. Some Clients use a Prefix for their Folders. In our old setup (dovecot 1.2) it was possible to create a Folder INBOX in mbox-space with some Subfolders. These Subfolders of INBOX showed up in Mailclient at toplevel. Now we updatet to 2.1.7. In higher dovecot versions INBOX seems to be a kind of protected foldername. dovecot is not offering this folder in folderlist. If i rename it, the folder is available again. There are only a few users, so i could fix this manually. But how to explain this? I did not find a setting etc. which could explain this behaviour. Our Config is pretty standard: mail_location is unchanged some dovecot versions we use: mail_location = mbox:~/mail:INBOX=/var/mail/%u most of conf is out of the box and also used in older versions. only new is namespace inbox, but should also be default: namespace inbox { inbox = yes } Is there an explaination for this behaviour? Thanks, Hajo

4 10

[Dovecot] . (dot) in maildir folder names
by Christoph Anton Mitterer 21 Sep '13

21 Sep '13

Hi. Uhm... yeah as the topic implies I want to have . (dots) in my folder names... Unfortunately dovecot's maildirmake hasn't a -f switch as the one from courier/maildrop, but that one in turn is buggy[0] and doesn't encode any characters at all. In principle, maildir++ should allow[1] having (encoded) dots in folder names,... but I don't quite understand how the encoding actually works, cause UTF-7 seems to be already the encoded (without padding and such)... but . is . in UTF-7 as well. Does dovecot support this? At least my mail clients (well I only tried Evolution so far which sucks in so many areas)... couldn't create it... but I guess it's rather a stupid client issue. Anyone knows how it would be encoded? Interestingly, and this might be actually an inssue in doveocot when I create maildirs with non-ASCII chars without encoding these, e.g. maildir/.München/ (instead of the correctly encoded (.M&APw-nchen) Dovecot seems to "export" these to the clients,... at least Evolution shows it in the list of subscribable folders (even correctly as "München")... but it doesn't allow me to actually subscribe to it. Cheers & thx, Chris. [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=709555 [1] http://www.courier-mta.org/maildir.html#id351803

5 24

[Dovecot] dovecot v-2.0, postfix+mysql configuration issue.
by Vimuth 20 Sep '13

20 Sep '13

Greetings Gentlemen, Im in the attempt of configuring a postfix mail server on RHEL6.2 with the said components in the subject. I followed a couple of guides and I beleive that Im almost there, but I get the following error when I try to send a mail to a user who is in the mysql database as follows, Dovecot version that is being used is 2.0.X. echo test | mail vimuth(a)mydomain.rock [root@box1 ~]# tail -f /var/log/maillog Mar 26 07:00:44 box1 dovecot: auth: Error: sql(vimuth(a)mydomain.rock): User query failed: Table 'mailserver.users' doesn't exist (using built-in default user_query: SELECT home, uid, gid FROM users WHERE username = '%n' AND domain = '%d') Mar 26 07:00:44 box1 dovecot: lda: Error: user vimuth(a)mydomain.rock: Auth USER lookup failed Mar 26 07:00:44 box1 dovecot: lda: Fatal: Internal error occurred. Refer to server log for more information. Mar 26 07:00:44 box1 postfix/pipe[6291]: A9E0E23B96: to=<vimuth(a)mydomain.rock>, relay=dovecot, delay=14255, delays=14255/0.02/0/0.12, dsn=4.3.0, status=deferred (temporary failure) Mar 26 07:05:44 box1 postfix/qmgr[5675]: 2E5AD23B66: from=<root(a)box1.mydomain.rock>, size=421, nrcpt=1 (queue active) Mar 26 07:05:44 box1 dovecot: auth: Error: mysql: Query failed, retrying: Table 'mailserver.users' doesn't exist Mar 26 07:05:44 box1 dovecot: auth: Error: sql(vimuth(a)mydomain.rock): User query failed: Table 'mailserver.users' doesn't exist (using built-in default user_query: SELECT home, uid, gid FROM users WHERE username = '%n' AND domain = '%d') Mar 26 07:05:44 box1 dovecot: lda: Error: user vimuth(a)mydomain.rock: Auth USER lookup failed Mar 26 07:05:44 box1 dovecot: lda: Fatal: Internal error occurred. Refer to server log for more information. Mar 26 07:05:44 box1 postfix/pipe[6312]: 2E5AD23B66: to=<vimuth(a)mydomain.rock>, relay=dovecot, delay=16007, delays=16007/0.03/0/0.08, dsn=4.3.0, status=deferred (temporary failure) As I understand, this has something to do with the way I have setup my database. Im gonna post what are in my most important configuration files and also of course in the database. here's *"/etc/dovecot/dovecot-sql.conf.ext"* driver = mysql connect = host=127.0.0.1 dbname=mailserver user=mailuser password=redhat default_pass_scheme = MD5 password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; here's */etc/postfix/main.cf* # TLS parameters smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_key_file = /etc/pki/tls/private/smtpd.key smtpd_tls_cert_file = /etc/pki/tls/smtpd.crt smtpd_tls_CAfile = /etc/pki/tls/certs/cacert.pem smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_timeout = 3600s tls_random_source = dev:/dev/urandom # set smtpd restrictions smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination</pre> virtual_mailbox_domains = mysql:/etc/postfix/ mysql-virtual-mailbox-domains.cf virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf mailbox_command = /usr/libexec/dovecot/deliver mailbox_transport = dovecot virtual_transport = dovecot dovecot_destination_recipient_limit = 1 Now to the *database * mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mail | | mailserver | | mysql | +--------------------+[ mysql> use mailserver; Database changed mysql> select * from mailserver; mysql> show tables; +----------------------+ | Tables_in_mailserver | +----------------------+ | virtual_aliases | | virtual_domains | | virtual_users | +----------------------+ mysql> select * from virtual_users; +----+-----------+----------------------------------+----------------------+ | id | domain_id | password | email | +----+-----------+----------------------------------+----------------------+ | 1 | 1 | e2798af12a7a0f4f70b4d69efbc25f4d | vimuth(a)mydomain.rock | +----+-----------+----------------------------------+----------------------+ 1 row in set (0.00 sec) Please can someone help. If you need more information please let me know. I'm ready to tear down the entire implementation and start from the scratch. Many thanks in advance. /Vimuth

5 5

[Dovecot] Unable to use dovecot-antispam plugin
by Xavier Claude 14 Sep '13

14 Sep '13

Hello everybody, I'm trying to use the Dovecot antispam on Debian Wheezy (dovecot version 2.1.7), but it's not working, each time I put a mail in the "Spam" folder, it is send in its original folder. What bother me the most is that I don't see anything in the log. Here is my dovecot config related to the antispam plugin : plugin { antispam_allow_append_to_spam = yes antispam_backend = dspam antispam_debug_target = syslog antispam_dspam_args = --user;%Lu;--deliver=spam,innocent;--source=error;--signature=%%s antispam_dspam_binary = /usr/bin/dspam antispam_dspam_notspam = --class=innocent antispam_dspam_result_header = X-DSPAM-Result antispam_dspam_spam = --class=spam antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam = SPAM;Spam antispam_trash = trash;Trash;Deleted Items; Deleted Messages sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocol imap { mail_plugins = " antispam" } And in the log, the only thing I see is the plugin initialization : Jul 27 13:18:05 twoflowers imap: antispam: plugin initialising (2.0-notgit) Jul 27 13:18:05 twoflowers imap: antispam: "SPAM" is exact match spam folder Jul 27 13:18:05 twoflowers imap: antispam: "Spam" is exact match spam folder Jul 27 13:18:05 twoflowers imap: antispam: no unsure folders Jul 27 13:18:05 twoflowers imap: antispam: "trash" is exact match trash folder Jul 27 13:18:05 twoflowers imap: antispam: "Trash" is exact match trash folder Jul 27 13:18:05 twoflowers imap: antispam: "Deleted Items" is exact match trash folder Jul 27 13:18:05 twoflowers imap: antispam: " Deleted Messages" is exact match trash folder Jul 27 13:18:05 twoflowers imap: antispam: allowing APPEND to spam folders Jul 27 13:18:05 twoflowers imap: antispam: dspam binary set to /usr/bin/dspam Jul 27 13:18:05 twoflowers imap: antispam: dspam result set to X-DSPAM-Result Jul 27 13:18:05 twoflowers imap: antispam: dspam extra arg --user Jul 27 13:18:05 twoflowers imap: antispam: dspam extra arg xavier Jul 27 13:18:05 twoflowers imap: antispam: dspam extra arg --deliver=spam,innocent Jul 27 13:18:05 twoflowers imap: antispam: dspam extra arg --source=error Jul 27 13:18:05 twoflowers imap: antispam: dspam extra arg --signature=%s Jul 27 13:18:05 twoflowers imap: antispam: signature header line is "X-DSPAM-Signature" Thank you for your help. -- Xavier Claude claude.xavier(a)gmail.com

2 5

[Dovecot] 2.2.4 + metadata plugin: autoconf failed
by Andreas Schulze 12 Sep '13

12 Sep '13

Hello, (had disabled maildelivery for the list, so I can't reply) I changed the minimal version from 2.65 to 2.59 then I run ./autogen.sh && ./configure --foo ... sles9: (autoconf-2.59) ---------------------- + cd dovecot-metadata-plugin-2a17386d4dbc + ./autogen.sh + creating m4/ ... + running autoreconf ... configure.ac:13: error: possibly undefined macro: AC_PROG_CC_C99 If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. autoreconf: /usr/bin/autoconf failed with exit status: 1 sles10: (autoconf-2.59) ----------------------- + cd dovecot-metadata-plugin-2a17386d4dbc + ./autogen.sh + creating m4/ ... + running autoreconf ... configure.ac:13: error: possibly undefined macro: AC_PROG_CC_C99 If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. autoreconf: /usr/bin/autoconf failed with exit status: 1 sles11: (autoconf-2.63) ----------------------- + cd dovecot-metadata-plugin-2a17386d4dbc + ./autogen.sh + creating m4/ ... + running autoreconf ... libtoolize: putting auxiliary files in `.'. libtoolize: copying file `./ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'. libtoolize: copying file `m4/libtool.m4' libtoolize: copying file `m4/ltoptions.m4' libtoolize: copying file `m4/ltsugar.m4' libtoolize: copying file `m4/ltversion.m4' libtoolize: copying file `m4/lt~obsolete.m4' configure.ac:7: installing `./config.guess' configure.ac:7: installing `./config.sub' configure.ac:4: installing `./install-sh' configure.ac:4: installing `./missing' src/Makefile.am:33: DOVECOT_PLUGIN_DEPS does not appear in AM_CONDITIONAL src/Makefile.am:41: DOVECOT_PLUGIN_DEPS does not appear in AM_CONDITIONAL src/Makefile.am:10: `dovecot_pkglib_LTLIBRARIES' is used but `dovecot_pkglibdir' is undefined src/Makefile.am: installing `./depcomp' src/Makefile.am:13: variable `libdovecot_metadata_la_SOURCES' is defined but no program or src/Makefile.am:13: library has `libdovecot_metadata_la' as canonic name (possible typo) autoreconf: automake failed with exit status: 1 openSUSE_Factory: (autoconf-2.69) --------------------------------- + cd dovecot-metadata-plugin-2a17386d4dbc + ./autogen.sh + creating m4/ ... + running autoreconf ... libtoolize: putting auxiliary files in `.'. libtoolize: copying file `./ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'. libtoolize: copying file `m4/libtool.m4' libtoolize: copying file `m4/ltoptions.m4' libtoolize: copying file `m4/ltsugar.m4' libtoolize: copying file `m4/ltversion.m4' libtoolize: copying file `m4/lt~obsolete.m4' configure.ac:7: installing './config.guess' configure.ac:7: installing './config.sub' configure.ac:4: installing './install-sh' configure.ac:4: installing './missing' src/Makefile.am:33: error: DOVECOT_PLUGIN_DEPS does not appear in AM_CONDITIONAL src/Makefile.am:41: error: DOVECOT_PLUGIN_DEPS does not appear in AM_CONDITIONAL src/Makefile.am:10: error: 'dovecot_pkglib_LTLIBRARIES' is used but 'dovecot_pkglibdir' is undefined src/Makefile.am: installing './depcomp' src/Makefile.am:13: warning: variable 'libdovecot_metadata_la_SOURCES' is defined but no program or src/Makefile.am:13: library has 'libdovecot_metadata_la' as canonical name (possible typo) autoreconf: automake failed with exit status: 1 -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 Nürnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 Nürnberg, Paumgartnerstr. 6-14 | Registergericht Nürnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr Jörg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen

1 1

[Dovecot] Received header only showing localhost
by Matt Devaney 05 Sep '13

05 Sep '13

Hi all, Im a new dovecot user and a first-time poster so please forgive me if this is a stupid question, or even (as I suspect it might be) not really a Dovecot question. My system is the fairly typical mix of postfix, amavis-new, and Dovecot. For some reason Ive noticed that when I look at the mail headers for delivered messages the Received header is missing the detail of any external servers that the mail went through, and additionally the tag appears twice; for example the Welcome to the dovecot mailing list email has: Received: from localhost (localhost [127.0.0.1]) by droplet.the-wyvern.net (Postfix) with ESMTP id DDDD163F73 for <myname(a)mydomain.net>; Mon, 26 Aug 2013 16:40:19 +0000 (UTC) Received: from droplet.the-wyvern.net ([127.0.0.1]) by localhost (droplet.the-wyvern.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6AFMiG0TAHUv for <myname(a)mydomain.net>; Mon, 26 Aug 2013 16:40:17 +0000 (UTC) Searching through the dovecot.markmail archive I can see this doesnt seem to be too unusual in that there are other examples where folk have posted message headers with the above format, however Im wondering if theres a good reason for it and if I can turn it off? I quite like being able to see where my mail came from :) If theres a man / wiki page to read somewhere that explains it then please feel free to tell me to RTM (.. long as you also point me in the right direction :p ) Thanks, Matt

2 4