October 2014 - dovecot

Wiki advice on running getmail on INBOX access - how does that work?
by c128 mail 18 Oct '14

18 Oct '14

Hi, I'm currently running getmail in a cron job every 2 minutes, so I was quite intrigued by this on the wiki: http://wiki2.dovecot.org/HowTo/TriggerGetmailOnIMAPAccess Thing is - I couldn't see how it would work and, when I tried it, it didn't work (at least not for me). It details using incrontab to monitor /home/username/.maildir/cur in order to trigger getmail. However, "cur" won't change unless populated with mail...as an initial result of actually running getmail? Seems like a chicken and egg situation. I reckon I'm missing something fundamental :-), but what is it? Ta.

2 4

Recap: last_login plugin with MySQL
by Luigi Rosa 18 Oct '14

18 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 With many thanks to Gedalya and Sven Hartge Situation: Dovecot installation with userbase in a MySQL table with the same structure described in Postfix.Admin installation; filesystem permission is flat with a single user (vmail) who owns maildir mailbox files and directories. Procedure to implement last-login plugin to update mailbox table. The lastlogin value is an integer in time() format Alter mailbox SQL table to add a `lastlogin` unsigned int field Add this to 10-master.conf service dict { unix_listener dict { mode = 0660 user = vmail } } In 90-plugin.conf add this: plugin { last_login_dict = proxy::lastlogin last_login_key = last-login/%u # default } dict { lastlogin = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } Enable last login for IMAP and POP3: protocol imap { mail_plugins = $mail_plugins last_login } protocol pop3 { mail_plugins = $mail_plugins last_login } Finally edit dovecot-dict-sql.conf.ext and add this lines: connect = {same values as dovecot-sql.conf.ext} map { pattern = shared/last-login/$user table = mailbox username_field = username value_field = lastlogin fields { username = $user } } Ciao, luigi - -- / +--[Luigi Rosa]-- \ After things have gone from bad to worse, the cycle will repeat itself. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJUQp/PAAoJEO5WT/qgw4yKLscP/1ysmNOP23fQeqN2I7gxfhKp 4TF0Buf0rxKjxzjE5jcw4T/vDKhdjj2uaYtpUkE188l+tmGdeQ55ENyXEml/Svvw eEs9Kc2UJTlSQpQFpspUezvOtPwgb0L3720rCqs61BshGFgpZsiUEpPW8mHeVoo1 j5zmo9TVdrSPr8XqLzoR+fdIamZRzZH2pLA9x9cgzsVMmqSlMm4Xv3f/JMloNs98 fwWo13Ji5uFCRbbMmNUDCDOpd4wPYZ7ASS64Jf9nR4DZ75fnQmRgvsbl3EWicrqT 94Z4hlSEkOYWxDzooFta37+Sp61IjQWkWFiM+0260Q5u8IArpatW+tT2eD3jzaxP Gu0HB8jg+bT9cXXtfo1HyA+JYswEtwKiUvUKa/09j6oDGgBD2KR1yhDJNsZsJXkB bEKsACdSztEvfe9j91AzW8S/W3Uk5c7l9RH1CXVNTCbQXaeagScAguh94KKP0oiW P43mBkuFg2mM+5+bXz0U8EYfswt/zYkp/pKFajEwyXIWdeUICPmFJA+GKpBXeJPn CL8nJCnUpKNsy85+Z6MXYXIv6a0D0utocdsQzR8gjG2whNnv7EFeHa2hyagKfG5g ydOSG6tKlWHCOSFaTogmUVZVXEGsE1gUmJZMeoyZVI5jEl3b3KJSTxB7QHtMsm5q NOyZ+PdNJtyDC8n4T48t =MJj6 -----END PGP SIGNATURE-----

1 0

[Dovecot] Disk Encryption
by Simon Brereton 17 Oct '14

17 Oct '14

Hi As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL. What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore. Is this possible? Is there a terrible reason to do it even if it is possible? I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent. Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely. Thanks. Simon

11 28

High memory usage
by Hanna Ruotsalainen 17 Oct '14

17 Oct '14

Hi I recently (about 2 weeks ago) moved over my users over to a new version of dovecot (2.0.9) and now the memory usage on the new server seems much higher than it was before. The new server is a CentOS 6 VM with 4 GB of RAM and 4 CPUs and according to top, it's used up all of the RAM and all of the Swap. Here's a sample of the output from top (sorted by allocated memory). top - 10:42:36 up 22 days, 21:57, 1 user, load average: 1.04, 1.40, 1.52 Tasks: 1643 total, 1 running, 1642 sleeping, 0 stopped, 0 zombie Cpu(s): 0.7%us, 0.7%sy, 0.0%ni, 93.3%id, 5.1%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 3924432k total, 3779236k used, 145196k free, 23740k buffers Swap: 4063228k total, 3181164k used, 882064k free, 156420k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 16738 root 20 0 399m 182m 436 S 0.0 4.8 5:50.58 config 18790 dovenull 20 0 54528 11m 2036 S 0.0 0.3 0:00.05 imap-login 22277 dovenull 20 0 54528 11m 2156 S 0.0 0.3 0:00.04 imap-login 22796 dovenull 20 0 54528 11m 2140 S 0.0 0.3 0:00.03 imap-login 22749 dovenull 20 0 54528 11m 2148 S 0.0 0.3 0:00.03 imap-login 22825 dovenull 20 0 54528 11m 2156 S 0.0 0.3 0:00.03 imap-login This is with 1015 imap processes and 484 imap-login processes. The old version was 1.0.rc15 and running on a CentOS 4 box with 2GB of RAM. It was overloaded but the RAM usage wasn't maxed out like this one is (not much swap usage and a large amount of cached memory). Is this amount of memory usage normal? What could be causing it, if it isn't? Thanks for your help, Hanna Here is the output of dovecot -n: # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) nfs auth_verbose = yes disable_plaintext_auth = no listen = * login_greeting = Ready. mail_debug = yes mail_location = maildir:/mail/%1Lu/%u:INDEX=/var/cache/dovecot/indexes/%1Lu/%u mail_plugins = " mail_log notify" maildir_very_dirty_syncs = yes mbox_lazy_writes = no namespace { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap-pass.conf driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = imap pop3 service anvil { client_limit = 1250 } service auth { client_limit = 4300 user = root } service imap-login { process_limit = 600 service_count = 1 } service imap { process_limit = 1536 } service pop3-login { process_limit = 600 service_count = 1 } service pop3 { process_limit = 1536 } ssl_ca = </etc/pki/tls/certs/ca-bundle.crt ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap-user.conf driver = ldap } version_ignore = yes protocol imap { mail_max_userip_connections = 10 ssl_cert = </etc/pki/dovecot/certs/imap.ecuad.ca.pem ssl_key = </etc/pki/dovecot/private/imap.ecuad.ca.pem } protocol pop3 { pop3_uidl_format = %08Xv%08Xu ssl_cert = </etc/pki/dovecot/certs/pop.ecuad.ca.pem ssl_key = </etc/pki/dovecot/private/pop.ecuad.ca.pem } protocol lda { postmaster_address = _postmaster(a)example.com_ } -- Hanna Ruotsalainen Coordinator of Information Applications, IT Services | tel 604 844 3875 local 3875 emily carr university of art + design <http://www.ecuad.ca> | 1399 Johnston Street, Vancouver BC V6H 3R9

1 0

replicating (manage)sieve - no symlink on replica?
by Philipp 17 Oct '14

17 Oct '14

2.2.13/0.4.2, both from source-tarball, no fancy options. tcp-replication between two nodes and following plugin parameters (same on both sides, except the ip-addr) plugin { sieve_before = /home/ssfn/etc/dovecot/sieve.default sieve_global_dir = /home/ssfn/etc/dovecot sieve = ~/.dovecot.sieve sieve_dir = ~/.sieve sieve_extensions = +vnd.dovecot.duplicate sieve_duplicate_period = 6h mail_replica = tcp:10.66.50.15:9993 } mail_location = sdbox:/home/ssfn/var/mail/boxes/%n mail_home = /home/ssfn/var/mail/boxes/%n Testbed: user getting first email delivered, the subdir %n/.sieve/tmp is automatically created as expected. User logs into managesieve on node1 and uploads a script. The %n/.sieve/managesieve.sieve and the symlink .dovecot.sieve -> .sieve/managesieve.sieve is created as expected. One node2 (via replica) happens: automatically create %n/.sieve/tmp automatically create/sync %n/.sieve/managesieve.sieve just NOT the symlink .dovecot.sieve -> .sieve/managesieve.sieve Logfiles for lmtp,doveadm,dsync-local dont indicate an error why it could not create the link or such, to me it looks like it wouldnt even try. mail-syncing works flawlessly (for delivery, just read about the expunge .. hum.) I tried to force a full-resync via doveadm replicator afterwards, to no change on the matter. Any input? Log from node2 after the moment managesieve saves everything correctly on node1: Sep 11 15:20:12 doveadm(10.66.30.15,ssfntest): Debug: auth input: user=ssfntest Sep 11 15:20:12 doveadm(10.66.30.15,ssfntest): Debug: auth input: ssfntest Sep 11 15:20:12 doveadm(10.66.30.15,ssfntest): Debug: Added userdb setting: plugin/=yes Sep 11 15:20:12 doveadm(10.66.30.15,ssfntest): Debug: Effective uid=993, gid=993, home=/home/ssfn/var/mail/boxes/ssfntest Sep 11 15:20:12 doveadm(10.66.30.15,ssfntest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=sdbox:/home/ssfn/var/mail/boxes/ssfntest:INDEX=/home/ssfn/var/mail/index/ssfntest:ALT=/home/ssfn/var/mail/alternate/ssfntest Sep 11 15:20:12 doveadm(10.66.30.15,ssfntest): Debug: fs: root=/home/ssfn/var/mail/boxes/ssfntest, index=/home/ssfn/var/mail/index/ssfntest, indexpvt=, control=, inbox=, alt=/home/ssfn/var/mail/alternate/ssfntest Sep 11 15:20:12 dsync-server(ssfntest): Debug: Namespace : Using permissions from /home/ssfn/var/mail/boxes/ssfntest: mode=0700 gid=default Sep 11 15:20:12 dsync-server(ssfntest): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve: Pigeonhole version 0.4.2 initializing Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve-storage: using active sieve script path: /home/ssfn/var/mail/boxes/ssfntest/.dovecot.sieve Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve-storage: using sieve script storage directory: /home/ssfn/var/mail/boxes/ssfntest/.sieve Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve-storage: permission lookup failed from /home/ssfn/var/mail/boxes/ssfntest/.sieve Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve-storage: created storage directory /home/ssfn/var/mail/boxes/ssfntest/.sieve/tmp Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve-storage: relative path to sieve storage in active link: .sieve/ Sep 11 15:20:12 dsync-server(ssfntest): Debug: sieve: script file /home/ssfn/var/mail/boxes/ssfntest/.sieve/managesieve.sieve not found Sep 11 15:20:12 dsync-server(ssfntest): Debug: doveadm-sieve: Value missing for key `vendor/vendor.dovecot/pvt/server/sieve/files/managesieve' (last change: 1970-01-01 01:00:00) Sep 11 15:20:13 dsync-server(ssfntest): Debug: doveadm-sieve: Assigned value for key `vendor/vendor.dovecot/pvt/server/sieve/files/managesieve' FS tree node2 after that: [..boxes]# find ssfntest -ls|grep -v mailb 24442 4 drwx------ 4 ssfnimp dlagimp 4096 Sep 11 15:20 ssfntest 21048 4 drwx------ 3 ssfnimp dlagimp 4096 Sep 11 15:20 ssfntest/.sieve 21052 4 drwx------ 2 ssfnimp dlagimp 4096 Sep 11 15:20 ssfntest/.sieve/tmp 21056 4 -rw------- 1 ssfnimp dlagimp 69 Sep 11 15:20 ssfntest/.sieve/managesieve.sieve 21016 0 lrwxrwxrwx 1 ssfnimp dlagimp 38 Sep 11 15:05 ssfntest/dbox-alt-root -> /home/dlag/var/mail/alternate/ssfntest FS tree node1 "before": [..boxes]# find ssfntest -ls|grep -v mailb 1835841 4 drwx------ 4 ssfnimp ssfnimp 4096 Sep 11 15:20 ssfntest 1835861 0 lrwxrwxrwx 1 ssfnimp ssfnimp 24 Sep 11 15:05 ssfntest/.dovecot.sieve -> .sieve/managesieve.sieve 1835854 4 -rw------- 1 ssfnimp ssfnimp 8 Sep 11 15:05 ssfntest/dovecot-uidvalidity 1835856 4 drwx------ 3 ssfnimp ssfnimp 4096 Sep 11 15:20 ssfntest/.sieve 1835857 4 drwx------ 2 ssfnimp ssfnimp 4096 Sep 11 15:20 ssfntest/.sieve/tmp 1835860 4 -rw------- 1 ssfnimp ssfnimp 69 Sep 11 15:20 ssfntest/.sieve/managesieve.sieve 1835846 0 lrwxrwxrwx 1 ssfnimp ssfnimp 38 Sep 11 15:05 ssfntest/dbox-alt-root -> /home/ssfn/var/mail/alternate/ssfntest

3 2

Re: Unable to get virtual users set up with database auth
by Gedalya 17 Oct '14

17 Oct '14

When you reply, try to make sure you reply to the list (dovecot(a)dovecot.org) in Thunderbird you should have a Reply List button, or just use reply to all. btw I meant: the output of the 'dovecot -n' command, sorry :D OK so, first of all, lda doesn't do any passdb lookup at all, only userdb. It doesn't have any password to check. Just as a comment, your password query could just as well be: password_query = SELECT uPass as password FROM users WHERE uName = '%n' AND uDomain = '%d' auth_username_format = %Lu -- which I believe is set by default, would lowercase the username for you on the way in from the client, so if all your usernames are meant to be purely lowercase, there is no need to return these fields from the database. Set it explicitly just to be sure. You're using a somewhat old version, I'm not sure how it was back then. Either way, this shouldn't be affecting lda. Now, this seems to be wrong. mail_location = mbox:/var/spool/mail/%u:INBOX=/var/spool/mail/%u You probably want it more like: mbox:~/mail:INBOX=/var/spool/mail/%u The first value, ~/mail, will be a /mail directory under the 'home' directory set in your static userdb, and that's where mailboxes (folders) other than INBOX will be stored. It must be a directory, not a file! I'm a little confused about some of the other details in your log. Let's try to correct your mail_location first and see where that takes us. Also, under namespace, you might as well comment out the 'location = ' and 'prefix = ' lines, though I'm not sure it matters. On 10/16/2014 12:51 PM, Cliff Hayes wrote: > Here it is: > > [root@avalon dovecot]# dovecot -n output > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 Scientific Linux release > 6.5 (Carbon) ext4 > auth_debug = yes > auth_debug_passwords = yes > auth_verbose = yes > auth_verbose_passwords = plain > first_valid_uid = 496 > mail_access_groups = mail > mail_debug = yes > mail_location = mbox:/var/spool/mail/%u:INBOX=/var/spool/mail/%u > mbox_write_locks = fcntl > namespace { > hidden = no > inbox = yes > list = yes > location = > prefix = > separator = . > subscriptions = yes > type = private > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > protocols = imap > ssl_ca = </etc/pki/dovecot/certs/intermediate.crt > ssl_cert = </etc/pki/dovecot/certs/avalon20140929.crt > ssl_key = </etc/pki/dovecot/private/avalon20140929.key > userdb { > args = uid=497 gid=12 home=/var/email/%u > driver = static > } > verbose_ssl = yes > protocol imap { > imap_id_log = * > } > > [root@avalon dovecot]# cat dovecot-sql.conf.ext > > driver = mysql > > default_pass_scheme = PLAIN > > connect = host=localhost dbname=email user=blah password=blah > > password_query = SELECT uName as user, uNameDomain as username, > uDomain as domain, uPass as password FROM users WHERE uName = '%n' AND > uDomain = '%d' > > > > > On 10/16/2014 11:24 AM, Gedalya wrote: >> On 10/16/2014 12:19 PM, Cliff Hayes wrote: >>> well i don't want to use system users. but how do I assign a UID and >>> GID to virtual users to eliminate the permissions errors? >>> >> For dovecot, it is like this: >> http://wiki2.dovecot.org/AuthDatabase/SQL#User_database_lookups >> >> I don't know if this would be OK with your sendmail side. >> >> You should really post your complete dovecot config (dovecot -n output + >> any ".ext" files) so that we can get a better picture. >>

2 1

ssd for /home
by Luciano Gabriel Andino 16 Oct '14

16 Oct '14

Hi, I am thinking to change sata disk for /home and I want to know if change to a SSD hd, is a good option. I have 30-40 accounts with 30-50K email in boxes. -- Saludos!! Luciano Andino GNU/Linux user #185103 Santa Fe - Argentina -----------------------------------------------

4 3

dsync duplicating identical messages
by Bruno Viaris 16 Oct '14

16 Oct '14

Dear list, I’m trying to setup dovecot/postfix/getmail for a single user application: I have 3 email accounts and 2 main computers (home and work). I want to have an imap server on each machine with sync/replication so that read/reply/delete status and sorting into folders are identical. This part is achieved via dsync and seems to work. Messages are fetched from my ISPs with getmail on both machines. The problem I have is that dsync does not see that a message already fetched on both sides is identical, and duplicates it. I use sdbox (because I like the compression option and the separate storage of attachments). The content of the messages (headers and body) are identical, but the sdbox files have differences in the header and/or trailer. In fact, for each incoming message, I get 4 slightly different files (2 on each machine). Do you know where I could find documentation on how dsync compares messages, and also on sdbox file format ? If you explain me that there is no way to prevent this duplication and that anyway dsync does the right thing, I will probably use getmail only on one machine and let dsync propagate the messages to the other site, but I will miss the redundancy (and some speed) of my wished setup... Thanks -- Bruno P.S. : Background: Home has DSL with 10 MBit/s down and 1 MBit/s up, and full control over NAT / firewall Work internet access is limited by the 100 MBit LAN and has strict firewall and complicated VPN. So it is not easy for my home email client to access the imap server of my work machine, and it is slow for my work email client to access my home imap server. I don’t trust any company for archiving my emails.

2 1

Re: Unable to get virtual users set up with database auth
by Gedalya 16 Oct '14

16 Oct '14

On 10/16/2014 12:19 PM, Cliff Hayes wrote: > well i don't want to use system users. but how do I assign a UID and > GID to virtual users to eliminate the permissions errors? > For dovecot, it is like this: http://wiki2.dovecot.org/AuthDatabase/SQL#User_database_lookups I don't know if this would be OK with your sendmail side. You should really post your complete dovecot config (dovecot -n output + any ".ext" files) so that we can get a better picture.

1 0

Unable to get virtual users set up with database auth
by Cliff Hayes 16 Oct '14

16 Oct '14

I can't seem to get imap virtual users to work with database authentication on new Scientific Linux 6.5 server with Sendmail 8.14.4 & Dovecot 2.0.9 Apparently I have to also create system users because sendmail/procmail/dovecot want to check directory ownership. So I created a database, made entry for the user, created the user chayes on the system. But %u always contains just chayes instead of chayes(a)domain.tld even though I tried different select statements to put the entire username in the username field. This causes the following errors: Oct 16 00:10:26 avalon sendmail[2536]: s9G5APo9002535: forward /var/spool/mail/chayes(a)domain.tld/chayes/.forward.avalon: Group writable directory Oct 16 00:10:26 avalon sendmail[2536]: s9G5APo9002535: forward /var/spool/mail/chayes(a)domain.tld/chayes/.forward: Group writable directory Oct 16 00:10:26 avalon dovecot: lda(chayes): Debug: Effective uid=497, gid=497, home=/var/spool/mail/chayes(a)domain.tld/chayes Oct 16 00:10:26 avalon dovecot: lda(chayes): Debug: Home dir not found: /var/spool/mail/chayes(a)domain.tld/chayes Oct 16 00:10:26 avalon dovecot: lda(chayes): Debug: Namespace : type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:/var/spool/mail/chayes:INBOX=/var/spool/mail/chayes Oct 16 00:10:26 avalon dovecot: lda(chayes): Debug: fs: root=/var/spool/mail/chayes, index=, control=, inbox=/var/spool/mail/chayes Oct 16 00:10:26 avalon dovecot: lda(chayes): Error: user chayes: Initialization failed: Namespace '': mbox: mbox root directory can't be a file: /var/spool/mail/chayes (http://wiki.dovecot.org/MailLocation/Mbox) Oct 16 00:10:26 avalon dovecot: lda(chayes): Fatal: Invalid user settings. Refer to server log for more information. protocols = imap mail_location = mbox:/var/spool/mail/%u:INBOX=/var/spool/mail/%u password_query = SELECT uNameDomain as username, uDomain as domain, uPass as password FROM users WHERE uName = '%n' AND uDomain = '%d' also tried password_query = SELECT uName as username, uDomain as domain, uPass as password FROM users WHERE uName = '%n' AND uDomain = '%d' # file: /etc/procmailrc # system-wide settings for procmail SHELL="/bin/bash" SENDMAIL="/usr/sbin/sendmail -oi -t" LOGFILE="/var/log/procmail.log" DROPPRIVS=yes DELIVER="/usr/libexec/dovecot/deliver" #MAILDIR=$HOME/ #DEFAULT=$HOME/ :0 w * ^X-Spam-Status: Yes | $DELIVER -m spam :0 w | $DELIVER Thanks in advance

3 4