March 2013 - dovecot - dovecot.org

[Dovecot] Override quota settings with LDAP
by Hanno Hirschberger 20 Mar '13

20 Mar '13

Hi all, my current Dovecot setup is fetching both the imap quota limit and the mail location settings from LDAP entries. user_attrs = [...] UDBmailQuota=quota_rule=*:bytes=%$, mailLocation=mail That is because we have 3 users who wants to fetch their mail from /var/mail/<username>. All the other users have maildir with maildir quota. The LDAP entry for /var/mail user looks like this: mailLocation: mbox:/var/empty:INBOX=/var/mail/<username>:INDEX=MEMORY Works fine with one exception: the mail quota for /var/mail users. The dovecot log gets spammed with: Mar 20 13:37:04 lmtp(2494, username): Warning: quota: Namespace '' is not Maildir, skipping for Maildir++ quota Mar 20 13:37:04 lmtp(2494, username): Error: opendir(/var/empty/.imap) failed: No such file or directory ... which is only logical, because there's no maildir for that user. I want to get rid of these error messages and like to ask you if there's a way to override the "quota = maildir:User quota" setting in my Dovecot configuration for the 3 /var/mail users. For example with an extra LDAP field? Sadly I don't really know how to realize this. Or is there any other / better way? You can look at my doveconf -n output here: http://pastebin.com/mJZ2CXbg Best regards, Hanno

2 1

[Dovecot] shared folders and "subscriptions = yes"
by Lars Uhlmann 20 Mar '13

20 Mar '13

Example (dovecot 1.2.15): ,-[dovecot.conf] | namespace public { | separator = / | prefix = management | location = maildir:/path/to/management | subscriptions = yes | } | | namespace public { | separator = / | prefix = sales | location = maildir:/path/to/sales | subscriptions = yes | } `----- To have all IMAP-Users automatically updated their shared folder structure I use "subscriptions = yes". This works with the side effect, that all IMAP-Users can see _all_ folders in every namespace even if they do not have "lookup" access. This is of course not wanted. Is it possible to have dovecot check the ACL file in the root of a namespace before it parses the subscrptions file? regards Lars

2 1

[Dovecot] dsync migration questions
by Michael Smith (DF) 20 Mar '13

20 Mar '13

We are currently in the process of replacing one of our customer mail systems with a dovecot solution. However, one of the sticking points right now is how to get the old mail to the new system. On the dovecot side, we are using mdbox storage. On the old system we are using qpopper/mbox mailboxes in the following setup. All user inboxes are in /mnt/mail/mail_spool. Only select, privileged, users have mail folders in /mnt/mail/mail_spool/FOLDERS/user/. I know, insane setup, but it's almost 18 years legacy now. Like this... /mnt/mail/mail_spool/user1.mbox /mnt/mail/mail_spool/user2.mbox /mnt/mail/mail_spool/user3.mbox /mnt/mail/mail_spool/user4.mbox /mnt/mail/mail_spool/FOLDERS/user2/folder1.mbox /mnt/mail/mail_spool/FOLDERS/user2/folder2.mbox Also, the UIDs between the old system and the new dovecot solution are different. User1 may have uid 2134 on the old system, but have uid 193847 on the new system. We've tried the following. Note: User Anakin has a single message already in the new inbox. mkdir -p /mnt/mail/mail_spool/.migrate/ ln /mnt/mail/mail_spool/anakin /mnt/mail/mail_spool/.migrate/inbox chmod 644 /mnt/mail/mail_spool/.migrate/inbox dsync -D -v -R -u anakin(a)domain.com backup mbox:/mnt/mail/mail_spool/.migrate/ ... chmod 600 /mnt/mail/mail_spool/.migrate/inbox /bin/rm -rf /mnt/mail/mail_spool/.migrate dsync(root): Debug: Loading modules from directory: /usr/lib64/dovecot dsync(root): Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so dsync(root): Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so dsync(root): Debug: auth input: anakin uid=1000001228 gid=1000001228 home=/var/spool/mail/CC/1000001228 quota_rule=*:storage=0:messages=0 dsync(root): Debug: Added userdb setting: plugin/quota_rule=*:storage=0:messages=0 dsync(root): Debug: Effective uid=1000001228, gid=1000001228, home=/var/spool/mail/CC/1000001228 dsync(root): Debug: Quota root: name=User quota backend=dict args=:file:/var/spool/mail/CC/1000001228/dovecot.quota dsync(root): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 dsync(root): Debug: dict quota: user=anakin, uri=file:/var/spool/mail/CC/1000001228/dovecot.quota, noenforcing=0 dsync(root): Debug: fs: root=/var/spool/mail/CC/1000001228/mail, index=/var/spool/mail/CC/1000001228/index, control=, inbox= dsync(root): Debug: Namespace : Using permissions from /var/spool/mail/CC/1000001228/mail: mode=0700 gid=-1 dsync(root): Debug: Effective uid=1000001228, gid=1000001228, home=/var/spool/mail/CC/1000001228 dsync(root): Debug: Quota root: name=User quota backend=dict args=:file:/var/spool/mail/CC/1000001228/dovecot.quota dsync(root): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 dsync(root): Debug: dict quota: user=anakin, uri=file:/var/spool/mail/CC/1000001228/dovecot.quota, noenforcing=0 dsync(root): Debug: mbox: INBOX defaulted to /mnt/test/mail_spool/.migrate/inbox dsync(root): Debug: fs: root=/mnt/test/mail_spool/.migrate, index=, control=, inbox=/mnt/test/mail_spool/.migrate/inbox dsync(anakin): Debug: Namespace : Using permissions from /mnt/test/mail_spool/.migrate: mode=0777 gid=-1 dsync(anakin): Error: Can't delete mailbox INBOX: INBOX can't be deleted. dsync(anakin): Debug: Namespace : Permission lookup failed from /var/spool/mail/CC/1000001228/mail/mailboxes/INBOX_b6845831821c42517c490000a72406a4 dsync(anakin): Debug: Namespace : Using permissions from /var/spool/mail/CC/1000001228/mail: mode=0700 gid=-1 dsync(anakin): Debug: Namespace : Permission lookup failed from /var/spool/mail/CC/1000001228/mail/mailboxes/INBOX_b6845831821c42517c490000a72406a4 dsync(anakin): Debug: Namespace : Using permissions from /var/spool/mail/CC/1000001228/mail: mode=0700 gid=-1 dsync(anakin): Info: INBOX: only in dest dsync(anakin): Info: INBOX: only in source dsync(anakin): Error: Trying to open a non-listed mailbox with guid=814b6332541c425169490000a72406a4 dsync(anakin): Error: msg iteration failed: Couldn't open mailbox 814b6332541c425169490000a72406a4 dsync(anakin): Error: Trying to open a non-listed mailbox with guid=814b6332541c425169490000a72406a4 Nothing was migrated. In some other combinations, trying to reproduce them again, it does migrate the old mail, but first destroys the existing mail in the new mailbox. Our goal is to cut mail delivery over to the new dovecot system (which will result in existing emails before migration), and then migrate the old mail *while* maintaining message UIDs. This is the easiest way to ensure no "lost" email with the migration. What are we doing wrong, and how do we achieve our goals? Thanks in advance. -- Michael Smith

2 1

[Dovecot] Logon with Client Certificate and OTP fallback
by dovecot.pkoch＠dfgh.net 20 Mar '13

20 Mar '13

Dear Dovecot experts, we have unusual authentication requirements, namely: - almost all of our user are using a smartcard to connect with our mailserver. Thunderbird is our friend here as it will use the smartcard as an additional certificate store and Thunderbird will do client certificate based authentication when connecting via SSL with a mailserver - there's no way (at least that I know of) for our iPad users to use a smartcard, so our iPad and iPhone users use OTP token (http://www.gooze.eu/catalog/otp-tokens-oath-0) For 10 years we are using a very simple POP3-server where I replaced the authentication routines with my own functions and these function are doing the following: 1) if the connection was made without SSL refuse to connect 2) if the client was using a client certificate extract the username from the certificate. Since we cannot stop Thunderbird from sending a username/passowrd-combination we compare the username from the certificate with the username Thunderbird has sent. If the usernames matches we allow access and ignore the password. 3) if the client did not sent a client certificate we calculate the current OneTimePassword of the users OTP-token and compare that with the password Thunderbird has sent. If the passwords matches we allow access. If a OTP-password was validated it can be used for 24 hours so our OTP-users must use their tokens only once a day. Now we would like to switch from POP3 to IMAP, so here are my questions: - can Dovecot be CONFIGURED to do the above. I compiled it yesterday and SSL client authentication works as expected. OTP seems to be supported via PAM and I can write a PAM-modul for our OTP-tokens, so OTP would be possible too. But how do I tell Dovecot to do OTP-auth as a fallback for ClientCert-auth. - I already looked into the sourcecode of Dovecot. Seems like all the authentication routines were built as pluggable modules. Is there any documentation out there on how how to built my own auth-module. Integrating the OTP-auth directly into Dovecot would be a lot easier (for me) than creating a PAM-module, so I would prefer that. If there was interest from other people I would add some configuration options to Dovecot (for example the location of the OTP-token list). Otherwise I would just hardcode everything into the source. Peter

3 3

[Dovecot] ACL to make mailboxes populated by master account Read Only for regular users.
by Alex Cherniak 20 Mar '13

20 Mar '13

We have a solution using Dovecot as a secondary mail archive. All mailboxes are populated/groomed by master account and the actual users have only read access. This is achieved by a simple ACL approach. dovecot.conf has protocol imap { mail_plugins = acl quota imap_quota zlib } plugin { acl = vfile:/etc/dovecot/acls:cache_secs=300 } /etc/dovecot/acls/.DEFAULT file is trivial: user=master lrwstipekxa owner lr It used to work with Dovecot 2.0.4 for years, but after upgrade to 2.0.18 users now have full access to folders created by master account and can delete, add and move mails. Should it behave this way? How can I "secure" mailboxes again? Any help is appreciated.

2 1

[Dovecot] Core dump in 2.2.rc2 with FETCH 1 BODY.PEEK[ALL]
by Steffen Kaiser 20 Mar '13

20 Mar '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Usingt ./sbin/dovecot --version 2.2.rc2 (7a08461c5559) with Maildir, 2 FETCH 1 BODY.PEEK[ALL] dumps core, bt full is: #0 0x00007fa166e721b5 in raise () from /lib/libc.so.6 No symbol table info available. #1 0x00007fa166e74fc0 in abort () from /lib/libc.so.6 No symbol table info available. #2 0x00007fa16721ac47 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:191 backtrace = 0x181e2d0 "/usr/local/dovecot-2.2.rc2/lib/dovecot/libdovecot.so.0(+0x78c20) [0x7fa16721ac20] -> /usr/local/dovecot-2.2.rc2/lib/dovecot/libdovecot.so.0(+0x7a0dd) [0x7fa16721c0dd] -> /usr/local/dovecot-2.2.rc2/lib"... #3 0x00007fa16721c0dd in i_internal_fatal_handler (ctx=0x7fffa3de5d90, format=0x7fa1675bbde1 "file %s: line %d: unreached", args=0x7fffa3de5d70) at failures.c:652 status = 0 #4 0x00007fa16721af90 in i_panic (format=0x7fa1675bbde1 "file %s: line %d: unreached") at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 24, fp_offset = 48, overflow_arg_area = 0x7fffa3de5e70, reg_save_area = 0x7fffa3de5da0}} #5 0x00007fa16759d415 in imap_msgpart_parse (section=0x181e278 "ALL", msgpart_r=0x18476c0) at imap-msgpart.c:291 msgpart = 0x1851c40 pool = 0x1851c20 i = 0 next_digit = true ret = 0 #6 0x000000000041e734 in imap_fetch_body_section_init (ctx=0x7fffa3de5fa0) at imap-fetch-body.c:358 body = 0x18476b8 list_args = 0x7fa1672152a7 list_count = 0 str = 0x0 p = 0x181e26e "" error = 0x7fffa3de6460 "\001" __FUNCTION__ = "imap_fetch_body_section_init" #7 0x000000000041d064 in fetch_body_init (ctx=0x7fffa3de5fa0) at imap-fetch.c:670 No locals. #8 0x000000000041b6b5 in imap_fetch_init_handler (init_ctx=0x7fffa3de5fa0) at imap-fetch.c:62 handler = 0x1827e10 lookup_name = 0x181e270 "BODY" p = 0x181e264 ".PEEK[ALL]" __FUNCTION__ = "imap_fetch_init_handler" #9 0x000000000040e6f5 in imap_fetch_cmd_init_handler (ctx=0x1847068, cmd=0x1846f60, name=0x181e260 "BODY.PEEK[ALL]", args=0x7fffa3de5fe8) at cmd-fetch.c:36 init_ctx = {fetch_ctx = 0x1847068, pool = 0x1846f40, name = 0x181e260 "BODY.PEEK[ALL]", args = 0x1829978, error = 0x0} __FUNCTION__ = "imap_fetch_cmd_init_handler" #10 0x000000000040e86d in fetch_parse_args (ctx=0x1847068, cmd=0x1846f60, arg=0x1829978, next_arg_r=0x7fffa3de6078) at cmd-fetch.c:69 str = 0x181e260 "BODY.PEEK[ALL]" macro = 0x0 #11 0x000000000040eef3 in cmd_fetch (cmd=0x1846f60) at cmd-fetch.c:258 client = 0x1846380 ctx = 0x1847068 args = 0x1829918 next_arg = 0x7fa166e75130 list_arg = 0x1847048 search_args = 0x1850c30 qresync_args = {qresync_sample_seqset = 0x1, qresync_sample_uidset = 0x41a22c} messageset = 0x1829a78 "1" send_vanished = false ret = 1 #12 0x000000000041a18f in command_exec (cmd=0x1846f60) at imap-commands.c:156 hook = 0x1827d70 ret = false #13 0x00000000004190f2 in client_command_input (cmd=0x1846f60) at imap-client.c:775 client = 0x1846380 command = 0x20041a274 __FUNCTION__ = "client_command_input" #14 0x00000000004193f2 in client_command_input (cmd=0x1846f60) at imap-client.c:836 client = 0x1846380 command = 0x1827858 __FUNCTION__ = "client_command_input" #15 0x0000000000419512 in client_handle_next_command (client=0x1846380, remove_io_r=0x7fffa3de61bd) at imap-client.c:874 No locals. #16 0x0000000000419592 in client_handle_input (client=0x1846380) at imap-client.c:886 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #17 0x0000000000419725 in client_input (client=0x1846380) at imap-client.c:928 cmd = 0x183c8a8 output = 0x1846d98 bytes = 26 __FUNCTION__ = "client_input" #18 0x00007fa167231e1b in io_loop_call_io (io=0x1846e60) at ioloop.c:387 ioloop = 0x1826700 t_id = 2 #19 0x00007fa167233aaa in io_loop_handler_run (ioloop=0x1826700) at ioloop-epoll.c:215 ctx = 0x1826a80 events = 0x1826af0 event = 0x1826af0 list = 0x1846eb0 io = 0x1846e60 tv = {tv_sec = 1725, tv_usec = 816704} events_count = 6 msecs = 1725817 ret = 1 i = 0 j = 0 call = true __FUNCTION__ = "io_loop_handler_run" #20 0x00007fa167231ebe in io_loop_run (ioloop=0x1826700) at ioloop.c:406 No locals. #21 0x00007fa1671caa45 in master_service_run (service=0x18265a0, callback=0x4262c6 <client_connected>) at master-service.c:550 No locals. #22 0x00000000004265df in main (argc=1, argv=0x1826390) at main.c:400 set_roots = {0x42f700, 0x0} login_set = {auth_socket_path = 0x181e048 "\200\340\201\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x426078 <login_client_connected>, failure_callback = 0x426252 <login_client_failed>, request_auth_token = 1} service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUThQFl3r2wJMiz2NAQI+zwf8CLsphLKRsB2pgFAuoNT9UAcs74DVC5CT JqNw1X4IOau9fY+m0fyZC/xOowjZiPn0yz8ZIYG/6l5dFZxNKzuyUBwh1pHVqcP7 WOrM71+ZW43gT6hibCtaxyW61BQ3XoK/nIDVZM67eVcy7G/DFuGd9RJsV4Bh3iuO g3lXqaeFAeghukMKrvU/YwIXFVyjXsjjy8ZPm3QEtUEKWN8OXaveJ3O4VT/K0VWG 234D47rBT3KFt/kqwTn95LIp10DZzzch2i/QBe992acGi78Mz4F0+VT+lJFa6nlO ZIzAeqhQRXoM8a86N0RJHri4xIMCzPHXjx/5lWL0TDe6N4ITfzEMCQ== =PrvL -----END PGP SIGNATURE-----

3 3

[Dovecot] Lot of dovecot process writes
by Angel L. Mateo 20 Mar '13

20 Mar '13

Hello, While monitoring my servers' behaviour (ubuntu 12.04 with dovecot 2.1.9, with about 70000 users) I have found that lots of disk usage is done by dovecot process (the parent process). Since lmtp, imap and pop3 connections are handle by their own process, what could be the reason of this? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

3 4

[Dovecot] Maildir or Mdbox and expunge messages.
by Evgeny Basov 20 Mar '13

20 Mar '13

Hello. I tested dovecot's work with expunge messages on Mdbox over strace (see at tail of message). As I can see dovecot process opens old storage m.* file, reads it content, opens new temporary file, writes in this one content and rename this new one to m.(*+1). How fast this algorithm works on system with about 10000 users and 5Tb data? I will use mdbox_rotate_interval for delayed expunge but I think that simple delete file in Maildir must be faster than expunge from Mdbox. Please, tell me about real experience work with Mdbox on big loaded systems. ++++++++++++++++ 0.000017 open("/var/vmail/example.org/user/storage/.temp.1362474108.P7152Q0M909647.mailsys", O_RDWR|O_CREAT|O_TRUNC, 0666) = 11 0.000063 umask(077) = 066 0.000015 fallocate(11, 01, 0, 10485760) = 0 0.000054 fstat(11, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 0.000043 pread(10, "\1\2N 0000000000000285\nFC"..., 8192, 16) = 1465 0.000025 write(11, "2 M1e C5135b47c\n\1\2N 000"..., 691) = 691 0.000041 stat("/var/vmail/example.org/user/indexes/storage/dovecot.map.index.log.2", 0x7fff235b34c0) = -1 ENOENT (No such file or directory) 0.000026 alarm(180) = 0 0.000018 fcntl(9, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0 0.000020 alarm(0) = 180 0.000018 stat("/var/vmail/example.org/user/indexes/storage/dovecot.map.index.log", {st_mode=S_IFREG|0600, st_size=3656, ...}) = 0 0.000031 fstat(9, {st_mode=S_IFREG|0600, st_size=3656, ...}) = 0 0.000041 write(11, "\n\1\3\nR5135ad97\nV29b\nG5840870697ad"..., 62) = 62 0.000023 fdatasync(11) = 0 0.079275 stat("/var/vmail/example.org/user/storage/m.8", 0x7fff235b3640) = -1 ENOENT (No such file or directory) 0.000061 rename("/var/vmail/example.org/user/storage/.temp.1362474108.P7152Q0M909647.mailsys", "/var/vmail/example.org/user/storage/m.8") = 0 0.000164 write(9, "\200\200\200\203\0\0\10\20\210\0\0\0\200\200\200\207@\0\0\20\0\0\0\0\0\0\0\0\10\0\0\0"..., 152) = 152 0.000094 fcntl(9, F_SETLK, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 0.000042 unlink("/var/vmail/example.org/user/storage/m.7") = 0 0.000059 stat("/var/vmail/example.org/user/indexes/storage/dovecot.map.index.log", {st_mode=S_IFREG|0600, st_size=3808, ...}) = 0 0.000055 fstat(9, {st_mode=S_IFREG|0600, st_size=3808, ...}) = 0

2 1

[Dovecot] Stability of extraction of attachment
by Angel L. Mateo 20 Mar '13

20 Mar '13

Hello, We are migrating our dovecot servers from 2.1.9 to 2.1.15 (or 2.2 if there is a stable version soon) and maildir format to mdbox. Looking for mdbox options I have found options: * mail_attachment_dir * mail_attachement_min_size * mail_attachment_fs * mail_attachment_hash but this options are described with the message: # WARNING: This feature hasn't been tested much yet. Use at your own risk. What is the state of these options? Could I use it in a production environment? In case of yes, what is the best deduplicating option? Let it to the FS? Or configuring with "sis posix" or "sis-queue posix"? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

2 1

[Dovecot] Exposing masteruser info to cllients via lmtp-proxy connections
by Alexandr Sabitov 20 Mar '13

20 Mar '13

Hello, Please somebody suggest me how to remove the "*masteruser" appendix from the dovecot director database while using masteruser authentication. My situation is: - all users get authenticated in LDAP on the dovecot-director frontend - then they are forwarded to dovecot backend as username*masteruser ( this is a way how to proxy users with encrypted passwords). To enable it a added an extra field in LDAP-dovecot configuration: destuser=%u*masteruser This works OK, but problems are: - I have lmtp-proxy on the dovecot frontend and I have to use the same "%u*masteruser" field in lmtp passdb because the director thinks that me(a)mail.com and me(a)mail.com*masteruser are different users and would send lmtp-proxy connections to other dovecot backend. Here is a problem: "Delivery-To" and "Received" fields in any email contains "*masteruser" ! I need to remove any mentions about "masteruser" authentication from client emails. Also another little problem: - If I run "doveadm status" tool then to get user's data I need to specify user as "username*masteruser" Thank you in advance. Alexandr Sabitov Netregistry

2 1